Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
spack
spack-tutorial-container
Commits
22d9fc1c
Unverified
Commit
22d9fc1c
authored
Apr 08, 2021
by
Greg Becker
Committed by
GitHub
Apr 08, 2021
Browse files
Merge pull request #4 from spack/validate-rev-aws20
Validate pipeline for aws20 container
parents
a7e5c96c
fae46e0d
Changes
5
Hide whitespace changes
Inline
Side-by-side
UPDATING.md
View file @
22d9fc1c
...
...
@@ -12,11 +12,11 @@ There are two build rules (see this [section](./DOCKERHUB_SETUP.md#configure-bui
for more info) governing when DockerHub will rebuild the container. The
first specifies that any time you push to the
`master`
branch on this repo,
DockerHub will build an image and tag it with
`latest`
. The second build rull
specifies that any time you push a
tag
matching the regular expression
specifies that any time you push a
branch
matching the regular expression
`^rev-(.+)$`
, DockerHub will build an image and use the matched group
(everything in the
tag
following the
`rev-`
) to tag the image. So if,
for
example, you want a container to be tagged
`sc20`
, you would
tag this
repository
`rev-sc20`
and push it
.
(everything in the
branch name
following the
`rev-`
) to tag the image. So if,
for
example, you want a container to be tagged
`sc20`
, you would
push a branch
to this
repository
named
`rev-sc20`
.
## Steps to build an updated container
...
...
@@ -36,12 +36,15 @@ the helper script, run the following command from the root of the repository:
$ ./keyhelp/export_pipeline_key.sh ./docker "name-of-key-to-export"
1.
Create a new mirror (this is required until the sync process between the
mirror and the container build cache is controlled by/limited to the spack
environment in which it runs). The mirror url currently needs to be updated
1.
Either a) create a new mirror, or else b) clean out the existing one. This
is required until the sync process between the mirror and the container build
cache is controlled by/limited to the spack environment in which it runs). If
you choose to create a new mirror, the mirror url currently needs to be updated
in two locations: The
`spack.yaml`
and the DockerHub autobuild environment
variables (see
[
here
](
./DOCKERHUB_SETUP.md#configure-builds
)
for more
information).
information). If you choose to clean out the old mirror, just use the web
interface to remove the entire "build_cache" directory, and then there is
nothing else to change.
1.
Create a new buildgroup in CDash to track the builds from the version to be
(done by editing the
`spack.yaml`
and changing the
`build-group`
property)
...
...
@@ -54,3 +57,28 @@ the packages build.
1.
Merge the PR into master and/or tag the result using the tagging scheme
described in the section above.
## Items that may require updates as OS/Spack version change
Check what version of
`gcc`
gets installed by
`apt-get`
by default if you
don't specify a version. That should be the compiler the tutorial picks
to install most packages. Then an older
`gcc`
should be installed and used
to install zlib with the "custom" system compiler. If either of those
compilers aren't in the image used to build the pipeline, they need to
be bootstrapped in the
`spack.yaml`
. Then a newer
`gcc`
should also be
added to the
`spack.yaml`
and used for the demonstration of installing
a
`gcc`
with spack (and building all the packages with it in the subsequent
steps). That compiler will need to be bootstrapped as well.
Check if
`tcl`
is still a 1-dependency package, if it's not, it should
be replaced with one.
Check the hash of:
`spack spec zlib @1.2.8 cppflags=-O3`
, that hash should
be used in the line that installs
`tcl`
with a specific
`zlib`
dependency.
Check the version of
`clang`
we get with
`apt-get install clang`
. If it is
no longer clang-6.0.0, then update the version of clang we bootstrap
in the
`spack.yaml`
and the packages that built against clang to match.
Make sure to update the release branch of spack checked out in the
tutorial script.
docker/Dockerfile
View file @
22d9fc1c
...
...
@@ -8,17 +8,18 @@ ARG AWS_SECRET_ACCESS_KEY="None"
RUN
apt-get update
-y
&&
\
DEBIAN_FRONTEND
=
noninteractive apt-get
install
-y
--no-install-recommends
\
autoconf
\
build-essential
\
ca-certificates
\
curl
\
clang
\
clang-3.7
\
emacs
\
file
\
g++
\
g++-
4.8
\
g++-
6
\
gcc
\
gcc-
4.8
\
gcc-
6
\
gfortran
\
gfortran-
4.8
\
gfortran-
6
\
git
\
gnupg2
\
iproute2
\
...
...
@@ -37,22 +38,20 @@ RUN apt-get update -y && \
apt-get autoremove
--purge
&&
\
apt-get clean
RUN
mkdir
-p
/mirror/build_cache
&&
\
aws s3
sync
${
REMOTE_BUILDCACHE_URL
}
/mirror/build_cache
COPY
/public.key /mirror/public.key
#
COPY /packages.yaml /etc/spack/packages.yaml
COPY
/packages.yaml /etc/spack/packages.yaml
COPY
/tutorial-test.sh /tutorial/.test/tutorial-test.sh
### TODO: Find another way to set perms without increasing the image download size
RUN
useradd
-ms
/bin/bash spack
&&
\
mkdir
-p
/mirror/build_cache
&&
\
aws s3
sync
${
REMOTE_BUILDCACHE_URL
}
/mirror
&&
\
chmod
-R
go+r /mirror
&&
\
#
chmod -R go+r /etc/spack &&
\
chmod
-R
go+r /etc/spack
&&
\
chmod
go+rx /tutorial/.test/tutorial-test.sh
USER
spack
WORKDIR
/home/spack
CMD
["bash"]
i
CMD
["bash"]
docker/packages.yaml
0 → 100644
View file @
22d9fc1c
packages
:
all
:
target
:
[
x86_64
]
docker/tutorial-test.sh
View file @
22d9fc1c
...
...
@@ -15,7 +15,7 @@ rm -rf ~/.spack/*
# clone a new spack
git clone https://github.com/spack/spack
cd
spack
git checkout releases/v0.1
2
git checkout releases/v0.1
5
# shell init
.
share/spack/setup-env.sh
...
...
@@ -28,13 +28,13 @@ spack mirror add tutorial /mirror
spack gpg trust /mirror/public.key
spack
install
zlib %clang
spack
install
zlib @1.2.8
spack
install
zlib %gcc@
4.8
spack
install
zlib %gcc@
6.5.0
spack
install
zlib @1.2.8
cppflags
=
-O3
spack find
spack find
-lf
spack
install
tcl
spack
install
tcl ^zlib @1.2.8 %clang
spack
install
tcl ^/
64mn
spack
install
tcl ^/
h6i5
spack find
-ldf
spack
install
hdf5
spack
install
hdf5~mpi
...
...
@@ -51,10 +51,10 @@ spack find -lf zlib
spack find ^mpich
spack find
cppflags
=
-O3
spack find
-px
spack
install
gcc@
7.2
.0
spack
install
gcc@
8.3
.0
spack find
-p
gcc
spack compiler add
`
spack location
-i
gcc@
7.2
.0
`
spack compiler remove gcc@
7.2
.0
spack compiler add
`
spack location
-i
gcc@
8.3
.0
`
spack compiler remove gcc@
8.3
.0
# Packagin
spack
install
mpileaks
...
...
@@ -62,13 +62,13 @@ spack install mpileaks
# Modules
spack
install
lmod
source
`
spack location
-i
lmod
`
/lmod/7.8/init/bash
spack compiler add
`
spack location
-i
gcc@
7.2
.0
`
spack
install
netlib-scalapack ^openmpi ^openblas %gcc@
7.2
.0
spack
install
netlib-scalapack ^openmpi ^netlib-lapack %gcc@
7.2
.0
spack
install
netlib-scalapack ^mpich ^openblas %gcc@
7.2
.0
spack
install
netlib-scalapack ^mpich ^netlib-lapack %gcc@
7.2
.0
spack
install
py-scipy ^openblas %gcc@
7.2
.0
spack compiler remove gcc@
7.2
.0
spack compiler add
`
spack location
-i
gcc@
8.3
.0
`
spack
install
netlib-scalapack ^openmpi ^openblas %gcc@
8.3
.0
spack
install
netlib-scalapack ^openmpi ^netlib-lapack %gcc@
8.3
.0
spack
install
netlib-scalapack ^mpich ^openblas %gcc@
8.3
.0
spack
install
netlib-scalapack ^mpich ^netlib-lapack %gcc@
8.3
.0
spack
install
py-scipy ^openblas %gcc@
8.3
.0
spack compiler remove gcc@
8.3
.0
# Advanced packaging
spack
install
netlib-lapack
...
...
spack.yaml
View file @
22d9fc1c
...
...
@@ -42,7 +42,7 @@ spack:
-
$gcc_spack_built_packages
mirrors
:
cloud_mirror
:
'
s3://spack-tutorial-container/mirror
-v0.15.1
'
cloud_mirror
:
'
s3://spack-tutorial-container/mirror'
gitlab-ci
:
bootstrap
:
...
...
@@ -88,8 +88,6 @@ spack:
modules
:
enable
:
[]
packages
:
suite-sparse
:
version
:
[
5.3.0
,
5.4.0
,
5.5.0
,
5.6.0
]
all
:
target
:
[
x86_64
]
repos
:
[]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment