* Daemon: support remote configuration
This:
* Adds support for fetching the configuration from a remote HTTP location:
`ipfs-cluster-service init http://localhost:8080/ipfs/Qm...` will instruct
cluster to read the configuration file from ipfs on start (potentially making
use of ipns and dnslink).
This is done by creating a `service.json` like `{ "source": <url> }`.
The source is then read when loading that configuration every time the daemon starts.
This allows to let users always use a mutating remote configuration, potentially
adding/removing trusted peers from the list or adjusting other things.
* Configuration and state helpers from ipfs-cluster-service have been extracted
to its own cmdutils package. This will help supporting something like an
`ipfs-cluster-follow` command in the next releases.
* Allows to disable the rest api by not defining it in the configuration (I thought
this was already so, but apparently only affected the ipfsproxy).
* Removes informer/allocator configurations from the daemon (--alloc). No one used
a non default pair. In fact, it was potentially buggy to use the reposize one.
Peers configured with follower_mode = true fail to add/pin/unpin.
Additionally they do not contact other peers when doing Status, Sync or
Recover and report on themselves.
They still contact other peers when doing "peers ls", as this is an OpenRPC
endpoint.
This is merely improving user interaction with a cluster peer and avoids
getting into confusing places:
* pin/unpin seems to work even no one trusts them
* status will query all peers in the peerset only to get auth errors and
ignore them, becoming way slower than it could be
This is not a security feature.
As raised in #793, sometimes the user wants to simply disallow unpinning
altogether in a Cluster peer so that no content can be removed from IPFS
through cluster, even when crendentials are compromised (including access to
RPC endpoint).
This introduces an unpin_disable parameter for the ipfshttp connector. There
is no way that a cluster will effecitvely unpin something if the connector
refuses to ask ipfs.
With this commit
- If cid in `DELETE /pins/{cid}` isn't part of the pinset, it would
return 404
- If path in `DELETE /pins/{keyType}/{path}` resolves to a cid that
isn't part of the pinset, it would return 404
It has a few implications to launch a raft peer when you wanted to do crdt and vice-versa.
Same when exporting and exporting states.
Users starting cluster peers should be explicit about their consensus choice.
Also, if we ever want to make `crdt` the default, we can't do that before
making `raft` non-default first. I don't like to break things but otherwise
the experience for new users wanting to try crdts might be aweful.
With this commit, cluster peer will observe on events of peer removal
from cluster. On occurence of the event, the cluster peer will clear
the removed peer from its peerstore.
* Init should take a list of peers
This commit adds `--peers` option to `ipfs-cluster-service init`
`ipfs-cluster-service init --peers <multiaddress,multiaddress>`
- Adds and writes the given peers to the peerstore file
- For raft config section, adds the peer IDs to the `init_peerset`
- For crdt config section, add the peer IDs to the `trusted_peers`
* Improve pin/unpin method signatures:
These changes the following Cluster Go API methods:
* -> Cluster.Pin(ctx, cid, options) (pin, error)
* -> Cluster.Unpin(ctx, cid) (pin, error)
* -> Cluster.PinPath(ctx, path, opts) (pin,error)
Pin and Unpin now return the pinned object.
The signature of the methods now matches that of the API Client, is clearer as
to what options the user can set and is aligned with PinPath, UnpinPath, which
returned pin methods.
The REST API now returns the Pinned/Unpinned object rather than 204-Accepted.
This was necessary for a cleaner pin/update approach, which I'm working on in
another branch.
Most of the changes here are updating tests to the new signatures
* Adapt load-balancing client to new Pin/Unpin signatures
* cluster.go: Fix typo
Co-Authored-By: Kishan Sagathiya <kishansagathiya@gmail.com>
* cluster.go: Fix typo
Co-Authored-By: Kishan Sagathiya <kishansagathiya@gmail.com>
This adds a LoadBalancing rest client implementation which is initialized with a set of client configurations and can use two strategies: failover and roundrobin (more strategies can be added by implementing the LBStrategy interface).
* Do not load API components removed from the config
This commit introduces a map that would keep track of whether components
for a component were missing or not from the JSON config file. This map
can be check while creating cluster to avoid loading a component.
It would consider component only if the component is fully missing from the
config.
Say the component in question is `ipfsproxy` which is under `api`
section.
This would use defaults for `ipfsproxy` and load IPFS proxy.
```
{
"api":{
"ipfsproxy": {}
}
}
```
However, this would not load IPFS proxy
```
{
"api":{}
}
```
