Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
E
eic_container
Manage
Activity
Members
Labels
Plan
Issues
31
Issue boards
Milestones
Wiki
Code
Merge requests
29
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
containers
eic_container
Commits
bdba99f7
Commit
bdba99f7
authored
1 year ago
by
Wouter Deconinck
Browse files
Options
Downloads
Patches
Plain Diff
fix: re-enable eic-spack S3 build cache; increase db lock timeout to 10s / job
parent
02b2ac69
No related branches found
No related tags found
1 merge request
!561
fix: re-enable eic-spack S3 build cache; increase db lock timeout to 10s / job
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
.gitlab-ci.yml
+3
-2
3 additions, 2 deletions
.gitlab-ci.yml
containers/jug/dev.Dockerfile
+8
-7
8 additions, 7 deletions
containers/jug/dev.Dockerfile
mirrors.yaml.in
+18
-0
18 additions, 0 deletions
mirrors.yaml.in
with
29 additions
and
9 deletions
.gitlab-ci.yml
+
3
−
2
View file @
bdba99f7
...
...
@@ -294,6 +294,8 @@ jug_dev:default:
echo "FORCE_NOCACHE set"
export CACHE_FLAG="--no-cache"
fi
## Setup mirrors.yaml with rw mirror secrets
-
cat mirrors.yaml.in | envsubst > mirrors.yaml
## Build targets
-
for target in ${TARGETS} ; do
if [ ! ${target} = export ] ; then
...
...
@@ -319,11 +321,10 @@ jug_dev:default:
--build-arg EICSPACK_CHERRYPICKS="${EICSPACK_CHERRYPICKS}"
--build-arg S3_ACCESS_KEY="${S3_ACCESS_KEY}"
--build-arg S3_SECRET_KEY="${S3_SECRET_KEY}"
--build-arg S3RW_ACCESS_KEY="${S3RW_ACCESS_KEY}"
--build-arg S3RW_SECRET_KEY="${S3RW_SECRET_KEY}"
--build-arg INTERNAL_TAG=${INTERNAL_TAG}
--build-arg JUG_VERSION=${INTERNAL_TAG}-$(git rev-parse HEAD)
--build-context spack=.
--secret id=mirrors,src=mirrors.yaml
containers/jug
; done
...
...
This diff is collapsed.
Click to expand it.
containers/jug/dev.Dockerfile
+
8
−
7
View file @
bdba99f7
...
...
@@ -54,7 +54,7 @@ RUN declare -A arch=( \
&&
spack config blame packages
\
&&
spack config
--scope
site add
"config:suppress_gpg_warnings:true"
\
&&
spack config
--scope
site add
"config:build_jobs:
${
jobs
}
"
\
&&
spack config
--scope
site add
"config:db_lock_timeout:
120"
\
&&
spack config
--scope
site add
"config:db_lock_timeout:
${
jobs
}
0"
\
&&
spack config
--scope
site add
"config:install_tree:root:/opt/software"
\
&&
spack config blame config
\
&&
spack compiler find
--scope
site
\
...
...
@@ -69,7 +69,9 @@ RUN --mount=type=cache,target=/var/cache/spack-mirror \
&&
spack buildcache update-index
-d
/var/cache/spack-mirror
\
&&
spack mirror list
## Setup eic-spack buildcache mirrors (FIXME: leaks credentials into layer)
## Setup eic-spack buildcache mirrors
## - this always adds the read-only mirror to the container
## - the write-enabled mirror is provided later as a secret mount
ARG
S3_ACCESS_KEY=""
ARG
S3_SECRET_KEY=""
RUN
--mount
=
type
=
cache,target
=
/var/cache/spack-mirror
\
...
...
@@ -101,19 +103,18 @@ RUN git clone https://github.com/${EICSPACK_ORGREPO}.git ${EICSPACK_ROOT} \
fi
\
&&
spack repo add
--scope
site
"
${
EICSPACK_ROOT
}
"
## Setup our custom environment
## Setup our custom environment
(secret mount for write-enabled mirror)
COPY
--from=spack spack-environment/ /opt/spack-environment/
ARG
ENV=dev
RUN
--mount
=
type
=
cache,target
=
/var/cache/spack-mirror,sharing
=
locked
\
--mount
=
type
=
secret,id
=
mirrors,target
=
/opt/spack/etc/spack/mirrors.yaml
\
cd
/opt/spack-environment
\
&&
source
$SPACK_ROOT
/share/spack/setup-env.sh
\
&&
spack
env
activate
--dir
/opt/spack-environment/
${
ENV
}
\
&&
make
--jobs
${
jobs
}
--keep-going
--directory
/opt/spack-environment
\
SPACK_ENV
=
${
ENV
}
\
BUILDCACHE_DIR
=
/var/cache/spack-mirror
# FIXME disabled S3 buildcache until multipart upload fixed
# \
# BUILDCACHE_MIRROR=eic-spack
BUILDCACHE_DIR
=
/var/cache/spack-mirror
\
BUILDCACHE_MIRROR
=
eic-spack
## Create view at /usr/local
RUN
--mount
=
type
=
cache,target
=
/var/cache/spack-mirror,sharing
=
locked
\
...
...
This diff is collapsed.
Click to expand it.
mirrors.yaml.in
0 → 100644
+
18
−
0
View file @
bdba99f7
mirrors:
eic-spack:
fetch:
url: s3://eictest/EPIC/spack
access_pair:
- ${S3RW_ACCESS_KEY}
- ${S3RW_SECRET_KEY}
access_token: null
profile: null
endpoint_url: https://eics3.sdcc.bnl.gov:9000
push:
url: s3://eictest/EPIC/spack
access_pair:
- ${S3RW_ACCESS_KEY}
- ${S3RW_SECRET_KEY}
access_token: null
profile: null
endpoint_url: https://eics3.sdcc.bnl.gov:9000
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment