11124ee224
Long story: Since #1768 there has been a recurring repinning test failure with Raft consensus. Per the test, if a pin is allocated to a peer that has been shutdown, submitting the pin again should re-allocate it to a peer that is still running. Investigation on why this test fails and why it fails only in Raft lead to realizing that this and other similar tests, were passing by chance. The needed re-allocations were made not by the new submission of the pin, but by the automatic-repinning feature. The actual resubmitted pin was carrying the same allocations (one of them being the peer that was down), but it was silently failing because the RedirectToLeader() code path was using cc.ctx and hitting the peer that had been shutdown, which caused it to error. Fixing the context propagation, meant that we would re-overwrite the pin with the old allocations, thus the actual behaviour did not pass the test. So, on one side, this fix an number of tests that had not disabled automatic repinning and was probably getting in the way of things. On the other side, this removes a condition that prevents re-allocation of pins if they exists and options have not changed. I don't fully understand why this was there though, since the Allocate() code does return the old allocations anyways when they are enough, so it should not re-allocate randomly. I suspect this was preventing some misbehaviour in the Allocate() code from the time before it was improved with multiple allocators etc. |
||
|---|---|---|
| .github | ||
| adder | ||
| allocator/balanced | ||
| api | ||
| cmd | ||
| cmdutils | ||
| config | ||
| consensus | ||
| datastore | ||
| docker | ||
| informer | ||
| ipfsconn/ipfshttp | ||
| monitor | ||
| observations | ||
| pintracker | ||
| pstoremgr | ||
| rpcutil | ||
| sharness | ||
| state | ||
| test | ||
| version | ||
| .codeclimate.yml | ||
| .codecov.yml | ||
| .dockerignore | ||
| .gitignore | ||
| add_test.go | ||
| allocate.go | ||
| CHANGELOG.md | ||
| cluster_config_test.go | ||
| cluster_config.go | ||
| cluster_test.go | ||
| cluster.go | ||
| clusterhost.go | ||
| config_test.go | ||
| connect_graph.go | ||
| CONTRIBUTING.md | ||
| COPYRIGHT | ||
| docker-compose.yml | ||
| Dockerfile | ||
| Dockerfile-bundle | ||
| Dockerfile-test | ||
| go.mod | ||
| go.sum | ||
| ipfscluster_test.go | ||
| ipfscluster.go | ||
| LICENSE | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| logging.go | ||
| Makefile | ||
| peer_manager_test.go | ||
| pnet_test.go | ||
| README.md | ||
| release.sh | ||
| rpc_api.go | ||
| rpc_policy.go | ||
| util.go | ||
IPFS Cluster
Pinset orchestration for IPFS
IPFS Cluster provides data orchestration across a swarm of IPFS daemons by allocating, replicating and tracking a global pinset distributed among multiple peers.
There are 3 different applications:
- A cluster peer application:
ipfs-cluster-service, to be run along withgo-ipfsas a sidecar. - A client CLI application:
ipfs-cluster-ctl, which allows easily interacting with the peer's HTTP API. - An additional "follower" peer application:
ipfs-cluster-follow, focused on simplifying the process of configuring and running follower peers.
Are you using IPFS Cluster?
Please participate in the IPFS Cluster user registry.
Table of Contents
Documentation
Please visit https://ipfscluster.io/documentation/ to access user documentation, guides and any other resources, including detailed download and usage instructions.
News & Roadmap
We regularly post project updates to https://ipfscluster.io/news/ .
The most up-to-date Roadmap is available at https://ipfscluster.io/roadmap/ .
Install
Instructions for different installation methods (including from source) are available at https://ipfscluster.io/download .
Usage
Extensive usage information is provided at https://ipfscluster.io/documentation/ , including:
Contribute
PRs accepted. As part of the IPFS project, we have some contribution guidelines.
License
This library is dual-licensed under Apache 2.0 and MIT terms.
© 2022. Protocol Labs, Inc.