feat: use spack ci generate to setup build pipeline

Rather than having a single long build, we can split it up in different jobs with spack ci generate. This will be more resilient against failures. We can also run the builds inside a regular job instead of inside the container, and then only pull from the build cache once we build the container.