GlobalPinInfo objects carried redundant information (Cid, Peer) that takes
space and time to serialize.
This has been addressed by having GlobalPinInfo embed PinInfoShort rather than
PinInfo. This new types ommits redundant fields.
* Libp2p protectors no longer needed, use PSK directly
* Generate cluster 32-byte secret here (helper gone from pnet)
* Switch to go-log/v2 in all places
* DHT bootstrapping not needed. Adjust DHT options for tests.
* Do not rely on dissappeared CidToDsKey and DsKeyToCid functions fro dshelp.
* Disable QUIC (does not support private networks)
* Fix tests: autodiscovery started working properly
* add ipv6 listening addresses to the default config
* ipfsproxy: support multiple listeners. Add default ipv6.
* mm
* restapi: support multiple listen addresses. enable ipv6
* cluster_config: format default listen addresses
* commands: update for multiple listeners. Fix randomports for udp and ipv6.
* ipfs-cluster-service: fix randomports test
* multiple listeners: fix remaining tests
* golint
* Disable ipv6 in defaults
It is not supported by docker by default. It is not supported in travis-CI
build environments. User can enable it now manually.
* proxy: disable ipv6 in test
* ipfshttp: fix test
Co-authored-by: @RubenKelevra <cyrond@gmail.com>
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.
Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
This adds a new PinOption: ExpireAt.
The StateSync ticker will check and unpin expired pins from the Cluster.
ipfs-cluster-ctl supports an "expire-in" which gives a duration.
- Human-sizes for freespace metrics. Display whether if metric is
expires in something like "expires in 3m".
- When not passing metric name `ipfs-cluster-ctl health metrics` hits
the the metrics endpoint which returns a list of available metrics and
displays to user
- Humanize metrics output
- Sort metrics output
- cluster method, ipfs connector method, rpc and rest apis,
command, etc for repo gc
- Remove extra space from policy generator
- Added special timeout for `/repo/gc` call to IPFS
- Added `RepoGCLocal` cluster rpc method, which will be used to run gc
on local IPFS daemon
- Added peer name to the repo gc struct
- Sorted with peer ids, while formatting(only affects cli
results)
- Special timeout setting where timeout gets checked from last update
- Added `local` argument, which would run gc only on contacted peer
- Instead of using a boolean config element(send logs to std out or
file), use a string element that would store path of the log file.
That way user has freedom to choose the filepath. Use std out if this
path is empty.
This commit introduces logging for Cluster HTTP APIs.
It adds a config element `send_logs_to_file`, which tells whether logs
should be saved in a file or shown in standard output.
Requests are logged as per Apache Common Log Format (CLF)
http://httpd.apache.org/docs/2.2/logs.html#commonFixes#574
With this commit
- If cid in `DELETE /pins/{cid}` isn't part of the pinset, it would
return 404
- If path in `DELETE /pins/{keyType}/{path}` resolves to a cid that
isn't part of the pinset, it would return 404
* Improve pin/unpin method signatures:
These changes the following Cluster Go API methods:
* -> Cluster.Pin(ctx, cid, options) (pin, error)
* -> Cluster.Unpin(ctx, cid) (pin, error)
* -> Cluster.PinPath(ctx, path, opts) (pin,error)
Pin and Unpin now return the pinned object.
The signature of the methods now matches that of the API Client, is clearer as
to what options the user can set and is aligned with PinPath, UnpinPath, which
returned pin methods.
The REST API now returns the Pinned/Unpinned object rather than 204-Accepted.
This was necessary for a cleaner pin/update approach, which I'm working on in
another branch.
Most of the changes here are updating tests to the new signatures
* Adapt load-balancing client to new Pin/Unpin signatures
* cluster.go: Fix typo
Co-Authored-By: Kishan Sagathiya <kishansagathiya@gmail.com>
* cluster.go: Fix typo
Co-Authored-By: Kishan Sagathiya <kishansagathiya@gmail.com>
This adds a LoadBalancing rest client implementation which is initialized with a set of client configurations and can use two strategies: failover and roundrobin (more strategies can be added by implementing the LBStrategy interface).
Currently, unless doing Join() (--bootstrap), we do not connect to any peers on startup.
We however loaded up the peerstore file and Raft will automatically connect
older peers to figure out who is the leader etc. DHT bootstrap, after Raft
was working, did the rest.
For CRDTs we need to connect to people on a normal boot as otherwise, unless
bootstrapping, this does not happen, even if the peerstore contains known peers.
This introduces a number of changes:
* Move peerstore file management back inside the Cluster component, which was
already in charge of saving the peerstore file.
* We keep saving all "known addresses" but we load them with a non permanent
TTL, so that there will be clean up of peers we're not connected to for long.
* "Bootstrap" (connect) to a small number of peers during Cluster component creation.
* Bootstrap the DHT asap after this, so that other cluster components can
initialize with a working peer discovery mechanism.
* CRDT Trust() method will now:
* Protect the trusted Peer ID in the conn manager
* Give top priority in the PeerManager to that Peer (see below)
* Mark addresses as permanent in the Peerstore
The PeerManager now attaches priorities to peers when importing them and is
able to order them according to that priority. The result is that peers with
high priority are saved first in the peerstore file. When we load the peerstore
file, the first entries in it are given the highest priority.
This means that during startup we will connect to "trusted peers" first
(because they have been tagged with priority in the previous run and saved at
the top of the list). Once connected to a small number of peers, we let the
DHT bootstrap process in the background do the rest and discover the network.
All this makes the peerstore file a "bootstrap" list for CRDTs and we will attempt
to connect to peers on that list until some of those connections succeed.
