ecp21: Add macsio/scr to build cache, update for newer Spack instance
The ecp21 tutorial will be the first Spack tutorial based on the 0.16.X series of releases.
The ecp21 tutorial will require a complete rewrite of the dev workflows section, which requires new packages
along with their dependencies.
Other changes are updating the CI pipeline for changes to Spack from 0.15 to 0.16