Skip to content
Snippets Groups Projects
Commit d152a516 authored by Sylvester Joosten's avatar Sylvester Joosten
Browse files

Macos support

parent 2dbfadf8
Branches
Tags
No related merge requests found
Hide whitespace changes
Inline Side-by-side
README.md
+ 42
− 32
View file @ d152a516
...@@ -3,6 +3,12 @@ EIC software container ...@@ -3,6 +3,12 @@ EIC software container
Simple Installation Simple Installation
------------ ------------
*The environment has been tested on linux (requires singularity v3+) and MacOS (requires
docker)*
Please follow the steps below to setup and run the container in your environment.
1. Create a local directory that you want to work in, e.g `$HOME/eic`, and go into this 1. Create a local directory that you want to work in, e.g `$HOME/eic`, and go into this
directory. directory.
```bash ```bash
...@@ -25,44 +31,19 @@ eic-shell ...@@ -25,44 +31,19 @@ eic-shell
4. Within your development environment (`eic-shell`), you can install software to the 4. Within your development environment (`eic-shell`), you can install software to the
internal `$ATHENA_PREFIX` internal `$ATHENA_PREFIX`
Singularity Container Dowload for Development Usage Singularity Container setup for Development Usage
------------- -------------
**Note: this container download script is meant for expert usage. If it is unclear to you **Note: this container download script is meant for expert usage. If it is unclear to you
why you would want to do this, you are probably looking for the single installation why you would want to do this, you are probably looking for the simple `jug_xl` installation
above.** above.**
To download the `jug_dev:testing` base image, do
```bash You can use the same install scripts to setup other container setups, including `jug_dev`
curl https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/download_dev.sh | bash (the main development container). Note that for `jug_dev` there is no nighlty release, and
``` the appropriate version (tag) would be `testing`. To setup the `jug_dev:testing` environment, do
To download the `jug_xl:nightly` image, do
```bash ```bash
curl https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/download_dev.sh | bash -s -- -c jug_xl -v nightly curl https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/install.sh | bash -s -- -c jug_dev -v testing
``` ```
Using the docker container for your CI purposes
-----------------------------------------------
The docker containers are publicly accessible from
[Dockerhub](https://hub.docker.com/u/eicweb). You probably want to use the default
`jug_xl` container. Relevant versions are:
- `eicweb/jug_xl:nightly`: nightly release, with latest detector and reconstruction
version. This is probably what you want to use unless you are dispatching a large
simulation/reconstruciton job
- `eicweb/jug_xl:3.0-stable`: latest stable release, what you want to use for large
simulation jobs (for reproducibility). Please coordinate with the software group to
ensure all desired software changes are present in this container.
1. To load the container environment in your run scripts, you have to do nothing special.
The environment is already setup with good defaults, so you can use all the programs
in the container as usual and assume everything needed to run the included software
is already setup.
2. If using this container as a basis for a new container, you can direction access
the full container environment from a docker `RUN` shell command with no further
action needed. For the most optimal experience, you can install your software to
`/usr/local` to fully integrate with the existing environment. (Note that, internally,
`/usr/local` is a symlink to `/opt/view`).
Included software: Included software:
------------------ ------------------
- Included software (for the exact versions, check the file [spack.yaml](spack.yaml) or use the command `eic-info` inside the container): - Included software (for the exact versions, check the file [spack.yaml](spack.yaml) or use the command `eic-info` inside the container):
...@@ -103,3 +84,32 @@ Included software: ...@@ -103,3 +84,32 @@ Included software:
- igprof - igprof
- The singularity build exports the following applications: - The singularity build exports the following applications:
- eic-shell: a development shell in the image - eic-shell: a development shell in the image
Using the docker container for your CI purposes
-----------------------------------------------
**These instructions are old and need updating. In general we recommend using
`eicweb/juggler:latest` for most CI usages. This image is functionally identical to
`jug_xl:nightly`**
The docker containers are publicly accessible from
[Dockerhub](https://hub.docker.com/u/eicweb). You probably want to use the default
`jug_xl` container. Relevant versions are:
- `eicweb/jug_xl:nightly`: nightly release, with latest detector and reconstruction
version. This is probably what you want to use unless you are dispatching a large
simulation/reconstruciton job
- `eicweb/jug_xl:3.0-stable`: latest stable release, what you want to use for large
simulation jobs (for reproducibility). Please coordinate with the software group to
ensure all desired software changes are present in this container.
1. To load the container environment in your run scripts, you have to do nothing special.
The environment is already setup with good defaults, so you can use all the programs
in the container as usual and assume everything needed to run the included software
is already setup.
2. If using this container as a basis for a new container, you can direction access
the full container environment from a docker `RUN` shell command with no further
action needed. For the most optimal experience, you can install your software to
`/usr/local` to fully integrate with the existing environment. (Note that, internally,
`/usr/local` is a symlink to `/opt/view`).
install.sh
+ 244
− 112
View file @ d152a516
#!/bin/bash #!/bin/bash
## Simple setup script that installs the container
## in your local environment under $PREFIX/local/lib
## and creates a simple top-level launcher script
## that launches the container for this working directory
## with the $ATHENA_PREFIX variable pointing
## to the $PREFIX/local directory
CONTAINER="jug_xl" CONTAINER="jug_xl"
VERSION="nightly" VERSION="nightly"
PREFIX="$PWD" PREFIX="$PWD"
...@@ -10,6 +17,7 @@ function print_the_help { ...@@ -10,6 +17,7 @@ function print_the_help {
echo " -p,--prefix Working directory to deploy the environment (D: $PREFIX)" echo " -p,--prefix Working directory to deploy the environment (D: $PREFIX)"
echo " -t,--tmpdir Change tmp directory (D: $([[ -z "$TMPDIR" ]] && echo "/tmp" || echo "$TMPDIR"))" echo " -t,--tmpdir Change tmp directory (D: $([[ -z "$TMPDIR" ]] && echo "/tmp" || echo "$TMPDIR"))"
echo " -n,--no-cvmfs Disable check for local CVMFS (D: enabled)" echo " -n,--no-cvmfs Disable check for local CVMFS (D: enabled)"
echo " -c,--container Container version (D: $CONTAINER)"
echo " -v,--version Version to install (D: $VERSION)" echo " -v,--version Version to install (D: $VERSION)"
echo " -h,--help Print this message" echo " -h,--help Print this message"
echo "" echo ""
...@@ -23,7 +31,7 @@ while [ $# -gt 0 ]; do ...@@ -23,7 +31,7 @@ while [ $# -gt 0 ]; do
key=$1 key=$1
case $key in case $key in
-p|--prefix) -p|--prefix)
PREFIX=$2 PREFIX=$(realpath $2)
shift shift
shift shift
;; ;;
...@@ -37,6 +45,11 @@ while [ $# -gt 0 ]; do ...@@ -37,6 +45,11 @@ while [ $# -gt 0 ]; do
DISABLE_CVMFS_USAGE=true DISABLE_CVMFS_USAGE=true
shift shift
;; ;;
-c|--container)
CONTAINER=$2
shift
shift
;;
-v|--version) -v|--version)
VERSION=$2 VERSION=$2
shift shift
...@@ -54,7 +67,9 @@ while [ $# -gt 0 ]; do ...@@ -54,7 +67,9 @@ while [ $# -gt 0 ]; do
esac esac
done done
## create prefix if needed
mkdir -p $PREFIX || exit 1 mkdir -p $PREFIX || exit 1
pushd $PREFIX
if [ ! -d $PREFIX ]; then if [ ! -d $PREFIX ]; then
echo "ERROR: not a valid directory: $PREFIX" echo "ERROR: not a valid directory: $PREFIX"
...@@ -64,125 +79,123 @@ fi ...@@ -64,125 +79,123 @@ fi
echo "Setting up development environment for eicweb/$CONTAINER:$VERSION" echo "Setting up development environment for eicweb/$CONTAINER:$VERSION"
## Simple setup script that installs the container mkdir -p $PREFIX/local/lib || exit 1
## in your local environment under $PREFIX/local/lib
## and creates a simple top-level launcher script
## that launches the container for this working directory
## with the $ATHENA_PREFIX variable pointing
## to the $PREFIX/local directory
mkdir -p local/lib || exit 1 function install_linux() {
SINGULARITY=
## check for a singularity install
## default singularity if new enough
if [ $(type -P singularity ) ]; then
SINGULARITY=$(which singularity)
SINGULARITY_VERSION=`$SINGULARITY --version`
if [ ${SINGULARITY_VERSION:0:1} = 2 ]; then
## too old, look for something else
SINGULARITY=
fi
fi
if [ -z $SINGULARITY ]; then
## first priority: a known good install (this one is on JLAB)
if [ -d "/apps/singularity/3.7.1/bin/" ]; then
SINGULARITY="/apps/singularity/3.7.1/bin/singularity"
## whatever is in the path is next
elif [ $(type -P singularity ) ]; then
SINGULARITY=$(which singularity)
## cvmfs singularity is last resort (sandbox mode can cause issues)
elif [ -f "/cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity" ]; then
SINGULARITY="/cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity"
## not good...
else
echo "ERROR: no singularity found, please make sure you have singularity in your \$PATH"
exit 1
fi
fi
echo " - Found singularity at $SINGULARITY"
SINGULARITY= ## get singularity version
## check for a singularity install ## we only care if is 2.x or not, so we can use singularity --version
## default singularity if new enough ## which returns 2.xxxxx for version 2
if [ $(type -P singularity ) ]; then
SINGULARITY=$(which singularity)
SINGULARITY_VERSION=`$SINGULARITY --version` SINGULARITY_VERSION=`$SINGULARITY --version`
SIF=
if [ ${SINGULARITY_VERSION:0:1} = 2 ]; then if [ ${SINGULARITY_VERSION:0:1} = 2 ]; then
## too old, look for something else SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}.simg"
SINGULARITY=
fi echo "WARNING: your singularity version $SINGULARITY_VERSION is ancient, we strongly recommend using version 3.x"
fi echo "We will attempt to use a fall-back SIMG image to be used with this singularity version"
if [ -z $SINGULARITY ]; then if [ -f /gpfs02/eic/athena/${CONTAINER}-${VERSION}.simg ]; then
## first priority: a known good install (this one is on JLAB) ln -sf /gpfs02/eic/athena/${CONTAINER}-${VERSION}.simg ${SIF}
if [ -d "/apps/singularity/3.7.1/bin/" ]; then else
SINGULARITY="/apps/singularity/3.7.1/bin/singularity" echo "Attempting last-resort singularity pull for old image"
## whatever is in the path is next echo "This may take a few minutes..."
elif [ $(type -P singularity ) ]; then INSIF=`basename ${SIF}`
SINGULARITY=$(which singularity) singularity pull --name "${INSIF}" docker://eicweb/$CONTAINER:$VERSION
## cvmfs singularity is last resort (sandbox mode can cause issues) mv ${INSIF} $SIF
elif [ -f "/cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity" ]; then chmod +x ${SIF}
SINGULARITY="/cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity" unset INSIF
## not good... fi
else ## we are in sane territory, yay!
echo "ERROR: no singularity found, please make sure you have singularity in your \$PATH"
exit 1
fi
fi
echo " - Found singularity at $SINGULARITY"
## get singularity version
## we only care if is 2.x or not, so we can use singularity --version
## which returns 2.xxxxx for version 2
SINGULARITY_VERSION=`$SINGULARITY --version`
SIF=
if [ ${SINGULARITY_VERSION:0:1} = 2 ]; then
SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}.simg"
echo "WARNING: your singularity version $SINGULARITY_VERSION is ancient, we strongly recommend using version 3.x"
echo "We will attempt to use a fall-back SIMG image to be used with this singularity version"
if [ -f /gpfs02/eic/athena/${CONTAINER}-${VERSION}.simg ]; then
ln -sf /gpfs02/eic/athena/${CONTAINER}-${VERSION}.simg ${SIF}
else else
echo "Attempting last-resort singularity pull for old image" ## check if we can just use cvmfs for the image
echo "This may take a few minutes..." SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}.sif"
INSIF=`basename ${SIF}` if [ -z "$DISABLE_CVMFS_USAGE" -a -d /cvmfs/singularity.opensciencegrid.org/eicweb/${CONTAINER}:${VERSION} ]; then
singularity pull --name "${INSIF}" docker://eicweb/$CONTAINER:$VERSION SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}"
mv ${INSIF} $SIF ## need to cleanup in this case, else it will try to make a subdirectory
chmod +x ${SIF} rm -rf ${SIF}
unset INSIF ln -sf /cvmfs/singularity.opensciencegrid.org/eicweb/${CONTAINER}:${VERSION} ${SIF}
elif [ -f /cvmfs/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ]; then
ln -sf /cvmfs/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ${SIF}
elif [ -f /gpfs02/cvmfst0/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ]; then
ln -sf /gpfs02/cvmfst0/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ${SIF}
## check if we have an internal CI image we will use for testing purposes
elif [ -f $PWD/.gitlab-ci/${CONTAINER}-${VERSION}.sif ]; then
ln -sf $PWD/.gitlab-ci/${CONTAINER}-${VERSION}.sif ${SIF}
## if not, download the container to the system
else
## get the python installer and run the old-style install
## work in temp directory
tmp_dir=$(mktemp -d -t ci-XXXXXXXXXX)
pushd $tmp_dir
wget https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/install.py
chmod +x install.py
./install.py -f -c $CONTAINER -v $VERSION .
INSIF=lib/`basename ${SIF}`
mv $INSIF $SIF
chmod +x ${SIF}
## cleanup
popd
rm -rf $tmp_dir
unset INSIF
fi
fi fi
## we are in sane territory, yay!
else echo $SIF
## check if we can just use cvmfs for the image ls $SIF 2>&1 > /dev/null && GOOD_SIF=1
SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}.sif" if [ -z "$SIF" -o -z "$GOOD_SIF" ]; then
if [ -z "$DISABLE_CVMFS_USAGE" -a -d /cvmfs/singularity.opensciencegrid.org/eicweb/${CONTAINER}:${VERSION} ]; then echo "ERROR: no singularity image found"
SIF="$PREFIX/local/lib/${CONTAINER}-${VERSION}" exit 1
## need to cleanup in this case, else it will try to make a subdirectory
rm -rf ${SIF}
ln -sf /cvmfs/singularity.opensciencegrid.org/eicweb/${CONTAINER}:${VERSION} ${SIF}
elif [ -f /cvmfs/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ]; then
ln -sf /cvmfs/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ${SIF}
elif [ -f /gpfs02/cvmfst0/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ]; then
ln -sf /gpfs02/cvmfst0/eic.opensciencegrid.org/singularity/athena/${CONTAINER}_v${VERSION}.sif ${SIF}
## check if we have an internal CI image we will use for testing purposes
elif [ -f $PWD/.gitlab-ci/${CONTAINER}-${VERSION}.sif ]; then
ln -sf $PWD/.gitlab-ci/${CONTAINER}-${VERSION}.sif ${SIF}
## if not, download the container to the system
else else
## get the python installer and run the old-style install echo " - Deployed ${CONTAINER} image: $SIF"
## work in temp directory
tmp_dir=$(mktemp -d -t ci-XXXXXXXXXX)
pushd $tmp_dir
wget https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/install.py
chmod +x install.py
./install.py -f -c $CONTAINER -v $VERSION .
INSIF=lib/`basename ${SIF}`
mv $INSIF $SIF
chmod +x ${SIF}
## cleanup
popd
rm -rf $tmp_dir
unset INSIF
fi fi
fi
echo $SIF ## We want to make sure the root directory of the install directory
ls $SIF 2>&1 > /dev/null && GOOD_SIF=1 ## is always bound. We also check for the existence of a few standard
if [ -z "$SIF" -o -z "$GOOD_SIF" ]; then ## locations (/scratch /volatile /cache) and bind those too if found
echo "ERROR: no singularity image found" echo " - Determining additional bind paths"
exit 1 PREFIX_ROOT="/$(realpath $PREFIX | cut -d "/" -f2)"
else BINDPATH=$PREFIX_ROOT
echo " - Deployed ${CONTAINER} image: $SIF" echo " --> $PREFIX_ROOT"
fi for dir in /work /scratch /volatile /cache; do
## only add directories once
if [[ ${BINDPATH} =~ $(basename $dir) ]]; then
continue
fi
if [ -d $dir ]; then
echo " --> $dir"
BINDPATH="${BINDPATH},$dir"
fi
done
## We want to make sure the root directory of the install directory ## create a new top-level eic-shell launcher script
## is always bound. We also check for the existence of a few standard ## that sets the ATHENA_PREFIX and then starts singularity
## locations (/scratch /volatile /cache) and bind those too if found
echo " - Determining additional bind paths"
PREFIX_ROOT="/$(realpath $PREFIX | cut -d "/" -f2)"
BINDPATH=$PREFIX_ROOT
echo " --> $PREFIX_ROOT"
for dir in /work /scratch /volatile /cache; do
if [ -d $dir ]; then
echo " --> $dir"
BINDPATH="${BINDPATH},$dir"
fi
done
## create a new top-level eic-shell launcher script
## that sets the ATHENA_PREFIX and then starts singularity
cat << EOF > eic-shell cat << EOF > eic-shell
#!/bin/bash #!/bin/bash
...@@ -270,8 +283,127 @@ export SINGULARITY_BINDPATH=$BINDPATH ...@@ -270,8 +283,127 @@ export SINGULARITY_BINDPATH=$BINDPATH
$SINGULARITY exec $SIF eic-shell \$@ $SINGULARITY exec $SIF eic-shell \$@
EOF EOF
chmod +x eic-shell chmod +x eic-shell
echo " - Created custom eic-shell excecutable"
}
function install_macos() {
## check for docker install
DOCKER=$(which docker)
if [ -z ${DOCKER} ]; then
echo "ERROR: no docker install found, docker is required for running on MacOS"
fi
echo " - Found docker at ${DOCKER}"
IMG=eicweb/${CONTAINER}:${VERSION}
docker pull ${IMG}
echo " - Deployed ${CONTAINER} image: ${IMG}"
## We want to make sure the root directory of the install directory
## is always bound. We also check for the existence of a few standard
## locations (/Volumes /Users /tmp) and bind those too if found
echo " - Determining mount paths"
PREFIX_ROOT="/$(realpath $PREFIX | cut -d "/" -f2)"
MOUNT=""
echo " --> $PREFIX_ROOT"
for dir in /Volumes /Users /tmp; do
## only add directories once
if [[ ${MOUNT} =~ $(basename $dir) ]]; then
continue
fi
if [ -d $dir ]; then
echo " --> $dir"
MOUNT="$MOUNT -v $dir:$dir"
fi
done
echo " - Docker mount directive: '$MOUNT'"
## create a new top-level eic-shell launcher script
## that sets the ATHENA_PREFIX and then starts singularity
cat << EOF > eic-shell
#!/bin/bash
## capture environment setup for upgrades
CONTAINER=$CONTAINER
TMPDIR=$TMPDIR
VERSION=$VERSION
PREFIX=$PREFIX
DISABLE_CVMFS_USAGE=${DISABLE_CVMFS_USAGE}
function print_the_help {
echo "USAGE: ./eic-shell [OPTIONS] [ -- COMMAND ]"
echo "OPTIONAL ARGUMENTS:"
echo " -u,--upgrade Upgrade the container to the latest version"
echo " -h,--help Print this message"
echo ""
echo " Start the eic-shell containerized software environment."
echo ""
echo "EXAMPLES: "
echo " - Start an interactive shell: ./eic-shell"
echo " - Upgrade the container: ./eic-shell --upgrade"
echo " - Execute a single command: ./eic-shell -- <COMMAND>"
echo ""
exit
}
UPGRADE=
while [ \$# -gt 0 ]; do
key=\$1
case \$key in
-u|--upgrade)
UPGRADE=1
shift
;;
-h|--help)
print_the_help
exit 0
;;
--)
shift
break
;;
*)
echo "ERROR: unknown argument: \$key"
echo "use --help for more info"
exit 1
;;
esac
done
if [ ! -z \${UPGRADE} ]; then
echo "Upgrading eic-shell..."
docker pull $IMG || exit 1
echo "eic-shell upgrade sucessful"
exit 0
fi
docker run $MOUNT -w=$PWD -it --rm -e ATHENA_PREFIX=$PREFIX/local $IMG eic-shell \$@
EOF
chmod +x eic-shell
echo " - Created custom eic-shell excecutable"
}
## detect OS
OS=`uname -s`
case ${OS} in
Linux)
echo " - Detected OS: Linux"
install_linux
;;
Darwin)
echo " - Detected OS: MacOS"
install_macos
;;
*)
echo "ERROR: OS '${OS}' not currently supported"
exit 1
;;
esac
echo " - Created custom eic-shell excecutable" popd
echo "Environment setup succesfull" echo "Environment setup succesfull"
echo "You can start the development environment by running './eic-shell'" echo "You can start the development environment by running './eic-shell'"
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment