Commit Graph

1163 Commits

Author SHA1 Message Date
Hector Sanjuan
4c3f16d7aa
Merge pull request #233 from ipfs/feat/raft-state-backups
Raft: do not ever remove state, rename it and leave it around
2017-11-14 18:13:15 +01:00
Hector Sanjuan
33c325b865 cluster-service: Fix too many arguments for panic
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-14 18:01:32 +01:00
Hector Sanjuan
18ff7a79cc cluster-service: Upstream snap patch
Snaps set a custom $HOME, but we were using /etc/passwd.
There might be other cases were using a custom $HOME might be
handy.

In UNIX systems, $HOME should be always set. For all the rest,
we fall back to the original os/user.HomeDir method.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-14 13:23:43 +01:00
Hector Sanjuan
dc3f5b202e
Merge pull request #231 from ipfs/fix/224-pin-progress
Fix #224: Better handling of progress updates when proxy-adding file
2017-11-14 12:33:09 +01:00
Hector Sanjuan
2837170d51 Raft: improve test error message
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-14 12:31:58 +01:00
Hector Sanjuan
b6ba6d5a1e Issue #219: Clean up peer manager. Rename Peers RPC call
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-14 12:26:42 +01:00
Hector Sanjuan
ef86f718e7 Enable travis for snapcraft builds
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-14 01:12:36 +01:00
Hector Sanjuan
4ca12026dc
Merge pull request #232 from elopio/snapcraft
Add the packaging metadata to build the ipfs-cluster snap
2017-11-14 00:29:27 +01:00
Hector Sanjuan
1a06baeb23 Raft: do not ever remove state, rename it and leave it around
This commit changes the way that consensus.Clean() works. Before
it deleted the whole data folder. Now it renames it as <name>.old.0
and leaves it. When Clean() is called again, it renames <name>.old.0
as <name>.old.1, and the actual data becomes <name>.old.0. Higher number
means older. The number of backups is fixed to 5. When 5 backups exists
and a new one comes up again, the last one is discarded.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-13 18:18:52 +01:00
Leo Arias
4a45e5def5 Add the packaging metadata to build the ipfs-cluster snap 2017-11-13 16:43:48 +00:00
Hector Sanjuan
f14f2f4863 Fix #224: Better handling of progress updates when proxy-adding file
Up to now, we hardcoded progress to "false" in the proxy, regardless
of what the original request said. We now leave it as it is, and
just ignore any progress updates when processing the response.

Since the response is buffered and sent back all together, they are
still useless, but at least the clients (ipfs cli) won't show a 0%
progress bar when successfully adding a file.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-13 14:09:20 +01:00
Hector Sanjuan
2323deac6c Issue #219: Fix TestClustersPeerRemoveReallocsPins
This test failed if the leader was randomly selected to be
the node on which we wait for leader. Needed to remove
the shutdown-leader from the clusters slice.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 21:26:37 +01:00
Hector Sanjuan
f37c6c0620 Issue #219: Fix api types tests
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 18:23:38 +01:00
Hector Sanjuan
a6ab9fab47 Issue #219: Fix PeerAdd test again
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 17:32:07 +01:00
Hector Sanjuan
c0310ea995 Issue #219: Fix PeerAdd test
ClusterPeers now contains ourselves.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 17:09:08 +01:00
Hector Sanjuan
d1473fd3be Issue #219: Fix waiting for save on shutdown
Improve the logic

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 16:55:39 +01:00
Hector Sanjuan
01fc55550a Issue #219: make sure peers are saved after Join
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 16:22:31 +01:00
Hector Sanjuan
2a616aeddb Issue #219: Provide a list of peers and a list of addresses in the ID object.
Fix cluster-ctl to show right number of peers

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-10 16:11:09 +01:00
Hector Sanjuan
435012a7fd
Merge pull request #228 from elopio/patch-2
Add the instructions to install the snap
2017-11-10 03:28:52 +01:00
Leo Arias
a4248092bf Add instructions to install the snap to the guide 2017-11-10 02:09:35 +00:00
Leo Arias
66e9747733
Add the instructions to install the snap
I have just pushed an initial version of ipfs-cluster to the snapcraft store:
https://discuss.ipfs.io/t/start-testing-the-ipfs-cluster-snap/1422
2017-11-08 18:46:07 -06:00
Hector Sanjuan
9dd239666c
Merge pull request #227 from elopio/patch-1
Fix typo: ins/is
2017-11-09 01:35:56 +01:00
Leo Arias
6d3e96c7b1
Fix typo: ins/is 2017-11-08 18:29:57 -06:00
Hector Sanjuan
289890ad6c Back to 6 testing peers
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-09 01:27:26 +01:00
Hector Sanjuan
b852dfa892 Fix #219: WIP: Remove duplicate peer accounting
This change removes the duplicities of the PeerManager component:

* No more commiting PeerAdd and PeerRm log entries
* The Raft peer set is the source of truth
* Basic broadcasting is used to communicate peer multiaddresses
  in the cluster
* A peer can only be added in a healthy cluster
* A peer can be removed from any cluster which can still commit
* This also adds support for multiple multiaddresses per peer

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-08 20:04:04 +01:00
Hector Sanjuan
aced97cfa1
Merge pull request #199 from ipfs/feat/131-new-raft
Fix #139: Update cluster to Raft 1.0.0
2017-11-02 16:06:05 +01:00
Hector Sanjuan
bff1ec3635 Issue #131: rename addFromMultiaddrs to setFromMultiaddrs
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 19:38:46 +01:00
Hector Sanjuan
f64d3edb91 config: save 1 time per second at most.
This prevents hammering and saving multiple times for example
when bootstrapping.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 14:42:23 +01:00
Hector Sanjuan
073c43e291 Issue #131: Make sure peers are moved to bootstrap when leaving
Also, do not shutdown when seeing our own departure during bootstrap.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 13:58:57 +01:00
Hector Sanjuan
c912cfd205 Issue #131: Destroy raft data when the peer has been removed
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 13:25:28 +01:00
Hector Sanjuan
7a5f8f184b Issue #131: Improvements adding and removing
This works on remove+shutdown procedure and fixes a few small
issues.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 13:00:32 +01:00
Hector Sanjuan
10c7afbd59 Raft: re-enable: do not start with unconsistent peers.
Improved error messages

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
74ed634653 Raft: add cachestore for the log store
Just like consul does it

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
107ad3bdfc Shutdown: do not save peers if we did not become ready
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
199dbb944a Raft/PeerRm: attempt more orderly peer removal
Wait until FSM has applied the operation.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
7540e7b056 Leave on shutdown: only attempt when cluster reached ready state.
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
7df2277684 Consensus: only log pins committed on the leader.
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
09e9b05ed0 Peer add: do not print "new peer" info messages on boot
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
d21cc2b32f Issue #213: Always wait for configuration to be saved
Even when cluster dies

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
18dbf1a93b Issue #131: Do not abort on bad peerset
Print warning instead.

Shutdown raft on peerRm.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
b7ed9e5c25 Issue #131: Be more verbose when raft initialization fails
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
7bfb3c45d7 Issue #131: Remove some left-over logging
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
0980a5de70 Issue #131: Do not crash when shutting down after consensus start error
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
848023e381 Fix #139: Update cluster to Raft 1.0.0
The main differences is that the new version of Raft is more strict
about starting raft peers which already contain configurations.

For a start, cluster will fail to start if the configured cluster
peers are different from the Raft peers. The user will have to
manually cleanup Raft (TODO: an ipfs-cluster-service command for it).

Additionally, this commit adds extra options to the consensus/raft
configuration section, adds tests and improves existing ones and
improves certain code sections.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-11-01 12:17:33 +01:00
Hector Sanjuan
71a37e8303
Merge pull request #217 from ipfs/fix/216-docker
Fix #216: Docker does not like an env variable in CMD (anymore?)
2017-10-31 11:55:51 +01:00
Hector Sanjuan
fb7be4137f Fix #216: Docker does not like an env variable in CMD (anymore?)
This used to work. Even if it's been surfaced by 4a8759939 we were
extensively testing the Dockerfile before that release.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-10-31 11:46:32 +01:00
Hector Sanjuan
e51f771391
Merge pull request #214 from ipfs/fix/213-save
Issue #213: Fix save in cluster component config
2017-10-27 23:46:03 +02:00
Hector Sanjuan
746ce00d31 Issue #213: Peer addresses should never be a nil slice
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-10-27 22:11:14 +02:00
Hector Sanjuan
828236dcc0 Issue #213: Make sure we wait for configuration to be saved
There might be a case where the program is terminated before
configuration is saved.

Also, avoid calling save() multiple times on shutdowns.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-10-27 21:44:02 +02:00
Hector Sanjuan
849581681a Issue #213: Fix save in cluster component config
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-10-27 20:22:14 +02:00