Commit Graph

18 Commits

Author SHA1 Message Date
Vincent Ambo
2f448d5d4a refactor(server): Use logrus convenience functions for logs
Makes use of the `.WithError` and `.WithField` convenience functions
in logrus to simplify log statement construction.

This has the added benefit of making it easier to correctly log
errors.
2019-10-06 23:05:23 +01:00
Vincent Ambo
f312c7cf13 refactor(server): Convert existing log entries to structured format
This rewrites all existing log statements into the structured logrus
format. For consistency, all errors are always logged separately from
the primary message in a field called `error`.

Only the "info", "error" and "warn" severities are used.
2019-10-06 23:05:23 +01:00
Vincent Ambo
a1ef4a4217 feat(server): Add log level to severity mapping
The output format now writes a `severity` field that follows that
format that should be recognised by Stackdriver Logging.
2019-10-06 23:05:23 +01:00
Vincent Ambo
2268e6fa88 feat(server): Use hash of Nixery source as version
Uses a hash of Nixery's sources as the version displayed when Nixery
launches or logs an error. This makes it possible to distinguish
between errors logged from different versions.

The source hashes should be reproducible between different checkouts
of the same source tree.
2019-10-06 23:05:23 +01:00
Vincent Ambo
fb27326074 fix(server): Amend package path for Go tooling compatibility
With these changes it is possible to keep Nixery in $GOPATH and build
the server in there, while still having things work correctly via Nix.
2019-10-06 23:05:23 +01:00
Vincent Ambo
fd359338ea refactor(server): Replace log calls with logrus
This introduces a structured logging library that can be used (next
step) to attach additional metadata to log entries.
2019-10-06 23:05:23 +01:00
Vincent Ambo
d921925645 feat(server): Fetch popularity data on launch
The last missing puzzle piece for #50!
2019-10-03 13:21:04 +01:00
Vincent Ambo
1d3035df39 feat(server): Reimplement local manifest cache backed by files
Implements a local manifest cache that uses the temporary directory to
cache manifest builds.

This is necessary due to the size of manifests: Keeping them entirely
in-memory would quickly balloon the memory usage of Nixery, unless
some mechanism for cache eviction is implemented.
2019-10-03 13:21:04 +01:00
Vincent Ambo
27ca158889 refactor(server): Clean up cache implementation
A couple of minor fixes and improvements to the cache implementation.
2019-10-03 13:21:04 +01:00
Vincent Ambo
e3f39c11e0 fix(server): Minor fixes to updated new builder code 2019-10-03 13:21:04 +01:00
Vincent Ambo
8489c6e16a refactor(server): Introduce a state type to carry runtime state
The state type contains things such as the bucket handle and Nixery's
configuration which need to be passed around in the builder.

This is only added for convenience.
2019-10-03 13:21:04 +01:00
Vincent Ambo
c9237845ec feat(server): Cache built manifests to the GCS bucket
Caches manifests under `manifests/$cacheKey` in the GCS bucket and
introduces two-tiered retrieval of manifests from the caches (local
first, bucket second).

There is some cleanup to be done in this code, but the initial version
works.
2019-09-10 11:32:37 +01:00
Vincent Ambo
72fac25c97 refactor(server): Move package source management logic to server
Introduces three new types representing each of the possible package
sources and moves the logic for specifying the package source to the
server.

Concrete changes:

* Determining whether a specified git reference is a commit vs. a
  branch/tag is now done in the server, and is done more precisely by
  using a regular expression.

* Package sources now have a new `CacheKey` function which can be used
  to retrieve a key under which a build manifest can be cached *if*
  the package source is not a moving target (i.e. a full git commit
  hash of either nixpkgs or a private repository).

  This function is not yet used.

* Users *must* now specify a package source, Nixery no longer defaults
  to anything and will fail to launch if no source is configured.
2019-09-10 11:32:37 +01:00
Vincent Ambo
c9586e23dd fix(server): Print correct project name during startup
They grow up so fast :')
2019-08-14 20:18:41 +01:00
Vincent Ambo
55a465b071 feat(builder): Implement build cache for manifests & layers
Implements a cache that keeps track of:

a) Manifests that have already been built (for up to 6 hours)
b) Layers that have already been seen (and uploaded to GCS)

This significantly speeds up response times for images that are full
or partial matches with previous images served by an instance.
2019-08-14 20:18:41 +01:00
Vincent Ambo
a6d76d7985 refactor(server): Extract build logic into separate module
This module is going to get more complex as the implementation of #32
progresses.
2019-08-14 20:18:41 +01:00
Vincent Ambo
b2f179b47a refactor(server): Use wrapper script to avoid path dependency
Instead of requiring the server component to be made aware of the
location of the Nix builder via environment variables, this commit
introduces a wrapper script for the builder that can simply exist on
the builders $PATH.

This is one step towards a slightly nicer out-of-the-box experience
when using `nix-build -A nixery-bin`.
2019-08-14 00:02:04 +01:00
Vincent Ambo
32a48fd446 refactor: Move registry server to a subfolder 2019-08-14 00:02:04 +01:00