Commit Graph

101 Commits

Author SHA1 Message Date
Hector Sanjuan
8a045ae2d2 Update to latest go-libp2p-pubsub (with renames)
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-29 17:25:52 +01:00
Hector Sanjuan
1f29abf754
Merge pull request #596 from ipfs/feat/env-cfg
add env config; refactor LoadJSON tests
2018-10-29 16:01:16 +01:00
Hector Sanjuan
0b923f8d49 Set envconfig version (more or less), and move setDefaults down a bit.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-29 14:53:59 +01:00
Hector Sanjuan
de2848fb23 Upgrade ipld-related dependencies. Remove impoters DAG forced-Batching.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-29 12:50:36 +01:00
Adrian Lanzafame
3ab3c2afa0
add env config; refactor LoadJSON tests
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-26 19:38:30 +10:00
Hector Sanjuan
64bd1cb924 Upgrade to libp2p 6.0.23
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-25 12:08:37 +02:00
Hector Sanjuan
b86f3cedb4 Files package has been extracted from cmdkit
Related: https://github.com/ipfs/go-ipfs-cmdkit/pull/31

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-24 01:28:05 +02:00
Hector Sanjuan
72e19b4fa1 Use better version for gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:33:44 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
58d35f34ee Upgrade to libp2p-6.0.19. Update deps.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:40:40 +02:00
Hector Sanjuan
3e11445516
gx publish 0.6.0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-03 15:21:55 +02:00
Hector Sanjuan
12e7695b1e
gx publish 0.6.0-rc1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 16:35:35 +02:00
Hector Sanjuan
b27ae72ef0 Update to go-1.11
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 12:03:08 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Hector Sanjuan
59714f69d4 Allow running peers with different cluster versions in the same cluster
This patch modifies the RPC protocol tag to use Major and Minor parts of the
version and not all of it.

This means all peers on the 0.5.x can run in the same cluster.

As cluster has become more mature and I see less risks in letting peers from
similar versions run together. This is useful when upgrading too.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-09-27 07:35:57 +02:00
Hector Sanjuan
275a8b2881
Merge pull request #544 from ipfs/fix/update-go-cid
update go-cid and go-libp2p
2018-09-24 13:54:30 +02:00
Adrian Lanzafame
31474f6490
update go-cid and go-libp2p
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-24 11:35:38 +10:00
Adrian Lanzafame
a34154ff25
gx update go-dot
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-13 13:29:34 +10:00
Hector Sanjuan
753322cdc1
gx publish 0.5.0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-24 15:04:17 +02:00
Hector Sanjuan
2559655c81
gx publish 0.5.0-rc1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 20:51:49 +02:00
Hector Sanjuan
d6e516b84a
Merge pull request #513 from ipfs/remove-go-ipfs-dep
AddParams: support "cid-version" and "hash" function.
2018-08-21 20:49:02 +02:00
Hector Sanjuan
0b60e330a3 Fix duplicate dep
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 20:06:12 +02:00
Hector Sanjuan
b266ff2fbf
Merge pull request #514 from ipfs/fix/466-proxy-repo-stat
Fix #466: Hijack repo/stat in the proxy and return aggregates from the cluster
2018-08-21 19:39:03 +02:00
Hector Sanjuan
05102c13a0 AddParams: support "cid-version" and "hash" function.
Additionally, remove dependency to go-ipfs after extraction from
"go-mfs".

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 19:38:41 +02:00
Hector Sanjuan
9c3826fdc5 Use cmdkit WebFile.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 02:12:36 +02:00
Hector Sanjuan
2ffa3d80de Fix 466: Hijack repo/stat in the proxy and return aggregates from the cluster.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 20:43:27 +02:00
Hector Sanjuan
00a757fc0b Proxy/add: flush AddedOutput
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
5bbc699bb4 Issue #340: Fix some data races
Unfortunately, there are still some data races in yamux
https://github.com/libp2p/go-libp2p/issues/396 so we can't
enable this by default.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-15 12:27:01 +02:00
Hector Sanjuan
df768b0b17 Align go-libp2p-kad-dht version
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 12:09:13 +02:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
7b826a5335 Dependency upgrades
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:34 +02:00
Hector Sanjuan
f8ff9883b0 wip
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
68ec9316f7 Finish rebase of sharding branch. It builds!
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Wyatt Daviau
2d47e09d7c Resolving dep conflicts and small rebase inconsistancies
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
9f74f6f47d Addressing third round of comments
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
c12685a518 Bring importer interface to cluster
dependence on dex replaced with ipld-importer subpackage
slight improvement to importer tests and formatting
gx deps update: new raft, ipld format and ipfs added

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
a73763b5fe update to newest dex dep
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
4ce1958bf9 update libp2p gx deps
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
a77f0fb3ee adding dex as dependency
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
9521885b63 ipfs-cluster-ctl add
parse file path arguments to cmdkit.File objects
no support for directories, symlinks or recursive add

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:13 +02:00
Hector Sanjuan
d3d1f960f5 Feat: Enable DHT-based peer discovery and routing for cluster peers
This uses go-libp2p-kad-dht as routing provider for the Cluster Peers.

This means that:

* A cluster peer can discover other Cluster peers even if they are
not in their peerstore file.
* We remove a bunch of code sending and receiving peers multiaddresses
when a new peer was added to the Cluster.
* PeerAdd now takes an ID and not a multiaddress. We do not need to
ask the new peer which is our external multiaddress nor broadcast
the new multiaddress to everyone. This will fix problems when bootstrapping
a new peer to the Cluster while not all the other peers are online.
* Adding a new peer does not mean to open connections to all peers
anymore. The number of connections will be made according to the DHT
parameters (this is good to have for future work)

The that detecting a peer addition in the watchPeers() function does
no longer mean that we have connected to it or that we know its
multiaddresses. Therefore it's no point to save the peerstore in these
events anymore.

Here a question opens, should we save the peerstore at all, and should we
save multiaddresses only for cluster peers, or for everyone known?
Currently, the peerstore is only updated on clean shutdown,
and it is updated with all the multiaddresses known, and not limited to
peer IDs in the cluster, (because, why not).

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-07-24 15:33:41 +02:00
Hector Sanjuan
df5544952a Update the libp2p dependency to version 6.0.5
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-06-28 13:27:02 +02:00
Hector Sanjuan
a0b0b56529 gx: update go-libp2p
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-06-01 17:47:53 +02:00
Hector Sanjuan
510ed23686
gx publish 0.4.0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-30 13:26:31 +02:00
Hector Sanjuan
5a7c31a7a1 Fix #433: Use new floodsub version
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-29 20:53:45 +02:00
Hector Sanjuan
c10494f522
gx publish 0.4.0-rc2
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-28 12:46:04 +02:00
Hector Sanjuan
059fcdaa42 Go-libp2p: update to 5.0.17
Among other things, this fixes race condition test failures in libp2p and
random panics in go-log.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-27 17:30:47 +02:00
Hector Sanjuan
884f9d199a
gx publish 0.4.0-rc1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-14 11:57:55 +02:00
Hector Sanjuan
73b962f799 Basic Monitor: test Publish()
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-07 14:26:06 +02:00
Hector Sanjuan
3c3341e491 Monitor: add PublishMetric() to component interface
The monitor component should be in charge of deciding how it is
best to send metrics to other peers and what that means.

This adds the PublishMetric() method to the component interface
and moves that functionality from Cluster main component to the
basic monitor.

There is a behaviour change. Before, the metrics where sent only to
the leader, while the leader was the only peer to broadcast them everywhere.
Now, all peers broadcast all metrics everywhere. This is mostly
because we should not rely on the consensus layer providing a Leader(), so
we are taking the chance to remove this dependency.

Note that in any-case, pubsub monitoring should replace the
existing basic monitor. This is just paving the ground.

Additionally, in order to not duplicate the multiRPC code
in the monitor, I have moved that functionality to go-libp2p-gorpc
and added an rpcutil library to cluster which includes useful
methods to perform multiRPC requests (some of them existed in
util.go, others are new and help handling multiple contexts etc).

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-07 14:26:06 +02:00