Instead of compressing & decompressing again to get the underlying tar
hash, use a similar mechanism as for store path layers for the symlink
layer and only compress it once while uploading.
Docker expects hashes of compressed tarballs in the manifest (as these
are used to fetch from the content-addressable layer store), but for
some reason it expects hashes in the configuration layer to be of
uncompressed tarballs.
To achieve this an additional SHA256 hash is calculcated while
creating the layer tarballs, but before passing them to the gzip
writer.
In the current constellation the symlink layer is first compressed and
then decompressed again to calculate its hash. This can be refactored
in a future change.
This introduces a new Nix derivation that, given an attribute set of
layer hashes mapped to store paths, will create a layer tarball for
each of the store paths.
This is going to be used by the builder to create layers that are not
present in the cache.
Relates to #50.
Simplifies the wrapper script used to invoke Nix builds from Nixery to
just contain the essentials, since the layer grouping logic is moving
into the server itself.
This is the first step towards a more granular build process where
some of the build responsibility moves into the server component.
Rather than assembling all layers inside of Nix, it will only create
the symlink forest and return information about the runtime paths
required by the image.
The server is then responsible for grouping these paths into layers,
and assembling the layers themselves.
Relates to #50.
Introduces three new types representing each of the possible package
sources and moves the logic for specifying the package source to the
server.
Concrete changes:
* Determining whether a specified git reference is a commit vs. a
branch/tag is now done in the server, and is done more precisely by
using a regular expression.
* Package sources now have a new `CacheKey` function which can be used
to retrieve a key under which a build manifest can be cached *if*
the package source is not a moving target (i.e. a full git commit
hash of either nixpkgs or a private repository).
This function is not yet used.
* Users *must* now specify a package source, Nixery no longer defaults
to anything and will fail to launch if no source is configured.
Moves the relevant parts of the customisation layer construction from
dockerTools.mkCustomisationLayer into the Nixery code base.
The version in dockerTools builds additional files (including via
hashing of potentially large files) which are not required when
serving an image over the registry protocol.
Some upcoming changes might require the Nix build to be split into
multiple separate nix-build invocations of different expressions, thus
splitting this out is useful.
It also fixes an issue where `build-image/default.nix` might be called
in an environment where no Nix channels are configured.
ALl the ones except for build-image.nix are considered trivial. On the
latter, nixfmt makes some useful changes but by-and-large it is not
ready for that code yet.
Removes usage of the old layering algorithm and replaces it with the
new one.
Apart from the new layer layout this means that each layer is now
built in a separate derivation, which hopefully leads to better
cacheability.
Instead of requiring the server component to be made aware of the
location of the Nix builder via environment variables, this commit
introduces a wrapper script for the builder that can simply exist on
the builders $PATH.
This is one step towards a slightly nicer out-of-the-box experience
when using `nix-build -A nixery-bin`.
