The restapi component supports filters for the pinset. This was done to keep
expected output when sharding was fully supported by filtering out "internal"
pins.
However this filter requires looping on the full pinset and re-allocating and
usually does nothing. The useless copy is significant for really big pinsets.
Additionally, ipfs-cluster-ctl set the filter by default to "pins". By setting
it to "all" instead we can skip the whole filtering step and, in practice, get the
same results.
When using SSL and not talking to libp2p-http endpoints, we should not
resolve the dns names in the multiaddresses as otherwise we cannot
verify the https certificates used by the remote endpoint.
This commit adds a new add option: "format".
This option specifies how IPFS Cluster is expected to build the DAG when
adding content. By default, it takes a "unixfs", which chunks and DAG-ifies as
it did before, resulting in a UnixFSv1 DAG.
Alternatively, it can be set to "car". In this case, Cluster will directly
read blocks from the CAR file and add them.
Adding CAR files or doing normal processing is independent from letting
cluster do sharding or not. If sharding is ever enabled, Cluster could
potentially shard a large CAR file among peers.
Currently, importing CAR files is limited to a single CAR file with a single
root (the one that is pinned). Future iterations may support multiple CARs
and/or multiple roots by transparently wrapping them.
A "simplest thing that could work" implementation of adding a `--wait` flag to the ipfs-cluster-ctl add command.
Allows CI to wait for cluster to fully replicate the files just added before continuting, or fail if replication fails.
Fixes#1285
License: MIT
Signed-off-by: Oli Evans <oli@tableflip.io>
This adds a new pin option: Mode that can be set to "direct" or "recursive".
The Mode is used to set the MaxDepth Pin field accordingly. When set to 0, we
will call pin/add using the type=direct.
* Libp2p protectors no longer needed, use PSK directly
* Generate cluster 32-byte secret here (helper gone from pnet)
* Switch to go-log/v2 in all places
* DHT bootstrapping not needed. Adjust DHT options for tests.
* Do not rely on dissappeared CidToDsKey and DsKeyToCid functions fro dshelp.
* Disable QUIC (does not support private networks)
* Fix tests: autodiscovery started working properly
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.
Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
This adds a new PinOption: ExpireAt.
The StateSync ticker will check and unpin expired pins from the Cluster.
ipfs-cluster-ctl supports an "expire-in" which gives a duration.