This will be required for making a standalone, Nixery-style image
builder function usable from Nix.
Change-Id: I5e36348bd4c32d249d56f6628cd046916691319f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5601
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Cleans up a whole bunch of things I wanted to get out of the door
right away:
* depot internal references to //third_party/nixery have been replaced
with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks
Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
When serving a manifest, it is important to set the content-type
correctly (otherwise pulling an image is likely to give a cryptic
error message, "Error response from daemon: missing signature key").
This makes sure that we set the content-type properly for both
manifests and layers.
Extends storage.Persist to accept a Content-Type argument, which in
the GCS backend is persisted with the object to ensure that the object
is served back with this content-type.
This is not yet implemented for the filesystem backend, where the
parameter is simply ignored.
This should help in the case of clients which expect the returned
objects to have content-types set when, for example, fetching layers
by digest.
To ensure that registry clients which attempt to pull manifests by
their content hash can interact with Nixery, this change implements
persisting image manifests in the CAS in the same way as image layers.
In combination with the previous refactorings this means that Nixery's
serving flow is now compatible with containerd.
I have verified this locally, but CI currently only runs against
Docker and not containerd, which is something I plan to address in a
subsequent PR.
This fixes#102
Modifies the layer serving endpoint to be a generic blob-serving
endpoint that can handle both manifest and layer object "types".
Note that this commit does not yet populate the CAS with any
manifests.
This gets rid of the package called "server" and instead moves
everything into the project root, such that Go actually builds us a
binary called `nixery`.
This is the first step towards factoring out CLI-based functionality
for Nixery.
Google Cloud Storage supports granting access to protected objects via
time-restricted URLs that are cryptographically signed.
This makes it possible to store private data in buckets and to
distribute it to eligible clients without having to make those clients
aware of GCS authentication methods.
Nixery now uses this feature to sign URLs for GCS buckets when
returning layer URLs to clients on image pulls. This means that a
private Nixery instance can run a bucket with restricted access just
fine.
Under the hood Nixery uses a key provided via environment
variables to sign the URL with a 5 minute expiration time.
This can be set up by adding the following two environment variables:
* GCS_SIGNING_KEY: Path to the PEM file containing the signing key.
* GCS_SIGNING_ACCOUNT: Account ("e-mail" address) to use for signing.
If the variables are not set, the previous behaviour is not modified.
Previously the acknowledgement calls from Docker were receiving a
404 (which apparently doesn't bother it?!). This corrects the URL,
which meant that acknowledgement had to move inside of the
registryHandler.
Uses the structured errors feature introduced in the Nix code to
return more sensible errors to clients. For now this is quite limited,
but already a lot better than before:
* packages that could not be found result in 404s
* all other errors result in 500s
This way the registry clients will not attempt to interpret the
returned garbage data/empty response as something useful.
Adds environment variables with which users can configure the package
set source to use. Not setting a source lets Nix default to a recent
NixOS channel (currently nixos-19.03).
Instead of just dispatching on URL regexes, use handlers to split the
routes into registry-related handlers and otherwise(tm).
For now the otherwise(tm) consists of a file server serving the static
directory, rather than just a plain match on the index route.
Previously the code had hardcoded paths to runtime data (the Nix
builder & web files), which have now been moved into configuration
options.
Additionally configuration for the application is now centralised in a
single config struct, an instance of which is passed around the
application.
This makes it possible to implement a wrapper in Nix that will
configure the runtime data locations automatically.
