Align with previous behaviour and make sure it is logged that
the configuration file was written.
Do not say "peerstore written with 0 entries" as that might be
taken like an error.
Fixes#865.
This makes the necessary changes so that consensu is selected on "init" with a
flag set, by default, to "crdt". This generates only a "crdt" or a "raft"
section, not both.
If the configuration file has a "raft" section, "raft" will be used to start
the daemon. If it has a "crdt" section, "crdt" will be used. If it has none or
both sections, an error will happen.
This also affects "state *" commands, which will now autoselect how to work
from the existing configuration.
* Daemon: support remote configuration
This:
* Adds support for fetching the configuration from a remote HTTP location:
`ipfs-cluster-service init http://localhost:8080/ipfs/Qm...` will instruct
cluster to read the configuration file from ipfs on start (potentially making
use of ipns and dnslink).
This is done by creating a `service.json` like `{ "source": <url> }`.
The source is then read when loading that configuration every time the daemon starts.
This allows to let users always use a mutating remote configuration, potentially
adding/removing trusted peers from the list or adjusting other things.
* Configuration and state helpers from ipfs-cluster-service have been extracted
to its own cmdutils package. This will help supporting something like an
`ipfs-cluster-follow` command in the next releases.
* Allows to disable the rest api by not defining it in the configuration (I thought
this was already so, but apparently only affected the ipfsproxy).
* Removes informer/allocator configurations from the daemon (--alloc). No one used
a non default pair. In fact, it was potentially buggy to use the reposize one.
It has a few implications to launch a raft peer when you wanted to do crdt and vice-versa.
Same when exporting and exporting states.
Users starting cluster peers should be explicit about their consensus choice.
Also, if we ever want to make `crdt` the default, we can't do that before
making `raft` non-default first. I don't like to break things but otherwise
the experience for new users wanting to try crdts might be aweful.
* Init should take a list of peers
This commit adds `--peers` option to `ipfs-cluster-service init`
`ipfs-cluster-service init --peers <multiaddress,multiaddress>`
- Adds and writes the given peers to the peerstore file
- For raft config section, adds the peer IDs to the `init_peerset`
- For crdt config section, add the peer IDs to the `trusted_peers`
* Fix error messages (they must be in the form "doing something")
* Improve/reword some error messages
* Document the identity.json existance in the cli docs
* Fix a bunch of typos
* Fix missing folder path in the --help
* Fix cluster not locking when configuration is not there but folder is
* Fix force flag not overriding the config overwrite prompt
* Fix deletion of Raft state on re-init (not necessary if identity persists)
* Fix overwriting on identity (should not be overwritten if already exists)
Much of this paves the way to be able to run without service.json:
* Either taking default values (and using env vars) - maybe someday
* Either by getting a configuration template it from somewhere (ipfs, http)
at runtime - sooner.
This adds a new "crdt" consensus component using go-ds-crdt.
This implies several refactors to fully make cluster consensus-component
independent:
* Delete mapstate and fully adopt dsstate (after people have migrated).
* Return errors from state methods rather than ignoring them.
* Add a new "datastore" modules so that we can configure datastores in the
main configuration like other components.
* Let the consensus components fully define the "state.State". Thus, they do
not receive the state, they receive the storage where we put the state (a
go-datastore).
* Allow to customize how the monitor component obtains Peers() (the current
peerset), including avoiding using the current peerset. At the moment the
crdt consensus uses the monitoring component to define the current peerset.
Therefore the monitor component cannot rely on the consensus component to
produce a peerset.
* Re-factor/re-implementation of "ipfs-cluster-service state"
operations. Includes the dissapearance of the "migrate" one.
The CRDT consensus component defines creates a crdt-datastore (with ipfs-lite)
and uses it to intitialize a dssate. Thus the crdt-store is elegantly
wrapped. Any modifications to the state get automatically replicated to other
peers. We store all the CRDT DAG blocks in the local datastore.
The consensus components only expose a ReadOnly state, as any modifications to
the shared state should happen through them.
DHT and PubSub facilities must now be created outside of Cluster and passed in
so they can be re-used by different components.
Remove basic monitor
This commit removes `basic` monitor component, because it is not being
used by default since few releases ago pubsub monitor was introduced.
Issue #689
* cluster and restapi configs can also get values from environment variables
* other config components don't read any values from the environment
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
This commit adds support for OpenCensus tracing
and metrics collection. This required support for
context.Context propogation throughout the cluster
codebase, and in particular, the ipfscluster component
interfaces.
The tracing propogates across RPC and HTTP boundaries.
The current default tracing backend is Jaeger.
The metrics currently exports the metrics exposed by
the opencensus http plugin as well as the pprof metrics
to a prometheus endpoint for scraping.
The current default metrics backend is Prometheus.
Metrics are currently exposed by default due to low
overhead, can be turned off if desired, whereas tracing
is off by default as it has a much higher performance
overhead, though the extent of the performance hit can be
adjusted with smaller sampling rates.
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
This commit makes the proxy extract useful fixed headers (like CORS) from
the IPFS daemon API responses and then apply them to the responses
from hijacked endpoints like /add or /repo/stat.
It does this by caching a list of headers from the first IPFS API
response which has them. If we have not performed any proxied request or
managed to obtain the headers we're interested in, this will try triggering a
request to "/api/v0/version" to obtain them first.
This should fix the issues with using Cluster proxy with IPFS Companion and
Chrome.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
Prompt on init and skip the prompt in init -f (and not have a -y flag)
This would be in line with other state subcommands which ask by default
and -f skips the prompt.
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
This commit moves `ipfs-cluster-service` and `ipfs-cluster-ctl` to
`cmd/` directory to follow "standard" project structure.
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
