This commit adds support for OpenCensus tracing
and metrics collection. This required support for
context.Context propogation throughout the cluster
codebase, and in particular, the ipfscluster component
interfaces.
The tracing propogates across RPC and HTTP boundaries.
The current default tracing backend is Jaeger.
The metrics currently exports the metrics exposed by
the opencensus http plugin as well as the pprof metrics
to a prometheus endpoint for scraping.
The current default metrics backend is Prometheus.
Metrics are currently exposed by default due to low
overhead, can be turned off if desired, whereas tracing
is off by default as it has a much higher performance
overhead, though the extent of the performance hit can be
adjusted with smaller sampling rates.
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
This changes the current strategy to extract headers from the IPFS daemon to
use them for hijacked endpoints in the proxy. The ipfs daemon is a bit of a
mess and what we were doing is not really reliable, specially when it comes to
setting CORS headers right (which we were not doing).
The new approach is:
* For every hijacked request, make an OPTIONS request to the same path, with
the given Origin, to the IPFS daemon and extract some CORS headers from
that. Use those in the hijacked response
* Avoid hijacking OPTIONS request, they should always go through so the IPFS
daemon controls all the CORS-preflight things as it wants.
* Similar to before, have a only-once-triggered request to extract other
interesting or custom headers from a fixed IPFS endpoint. This allows us to
have the proxy forward other custom headers and to catch
`Access-Control-Expose-Methods`. The difference is that the endpoint use for
this and the additional headers are configurable by the user (but with hidden
configuration options because this is quite exotic from regular usage).
Now the implementation:
* Replaced the standard Muxer with gorilla/mux (I have also taken the change
to update the gxed version to the latest tag). This gives us much better
matching control over routes and allows us to not handle OPTIONS requests.
* This allows also to remove the extractArgument code and have proper handlers
for the endpoints passing command arguments as the last segment of the URL. A
very simple handler that wraps the default ones can be used to extract the
argument from the url and put it in the query. Overall much cleaner this way.
* No longer capture interesting headers from any random proxied request. This
made things complicated with a wrapping handler. We will just trigger the one
request to do it when we need it.
* When preparing the headers for the hijacked responses:
* Trigger the OPTIONS request and figure out which CORS things we should set
* Set the additional headers (perhaps triggering a POST request to fetch them)
* Set our own headers.
* Moved all the headers stuff to a new headers.go file.
* Added configuration options (hidden by default) to:
* Customize the extract headers endpoint
* Customize what additional headers are extracted
* Use HTTPs when talking to the IPFS API
* I haven't tested this, but I did not want to have hardcoded 'http://' urls
around, as before.
* Added extra testing for this, and tested manually a lot comparing the
daemon original output with our hijacked endpoint outputs while looking
at the API traffic with ngrep and making sure the requets happen as expected.
Also tested with IPFS companion in FF and Chrome.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This patch modifies the RPC protocol tag to use Major and Minor parts of the
version and not all of it.
This means all peers on the 0.5.x can run in the same cluster.
As cluster has become more mature and I see less risks in letting peers from
similar versions run together. This is useful when upgrading too.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
Unfortunately, there are still some data races in yamux
https://github.com/libp2p/go-libp2p/issues/396 so we can't
enable this by default.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
