Commit Graph

1883 Commits

Author SHA1 Message Date
Hector Sanjuan
4ea830f74e Feat: ipfs-cluster-follow
This adds a new cluster command: ipfs-cluster-follow.

This command allows initializing and running follower peers as configured by a
remote-source configuration. The command can list configured peers
and obtain information for each of them.

Peers are launched with the rest API listening on a local unix socket. The
command can be run to list the items in the cluster pinset using this
endpoint. Alternatively, if no socket is present, the peer will be assumed to
be offline and the pin list will be directly read from the datastore.

Cluster peers launched with this command (and their configurations) are
compatible with ipfs-cluster-ctl and ipfs-cluster-service. We purposely do not
support most configuration options here. Using ipfs-cluster-ctl or launching
the peers using ipfs-cluster-service is always an option when the usecase
deviates from that supported by ipfs-cluster-follow.

Examples:

$ ipfs-cluster-follow -> list configured peers
$ ipfs-cluster-follow --help
$ ipfs-cluster-follow <clusterName> init <url>
$ ipfs-cluster-follow <clusterName> info
$ ipfs-cluster-follow <clusterName> run
$ ipfs-cluster-follow <clusterName> list
2019-12-07 15:38:59 +01:00
Hector Sanjuan
ce3c50187e
config manager: Improve source-fetch error handling (#967)
config manager: Improve source-fetch error handling

* Allow detection of source-download errors by adding IsSourceDownloadError method.
* Treat non-success response codes as a failure to fetch the source.
* Set the Source url before errors happens so it can be consulted even in that case.
2019-12-07 12:32:43 +01:00
Kishan Sagathiya
e1faf12bae ipfsproxy: hijack repo/gc and trigger cluster-wide GC
This adds hijacking of the repo/gc endpoint to the proxy to do cluster-wide gc.
2019-12-06 13:08:57 +01:00
Hector Sanjuan
db352534df
Merge pull request #975 from ipfs/update/quic2
Update QUIC transport dependency
2019-12-06 11:27:26 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils
Service: extract some methods to cmdutils
2019-12-06 11:08:58 +01:00
Hector Sanjuan
61778f50fd
Merge pull request #965 from ipfs/fix/pprof
Use index endpoint with a trailing slash
2019-12-06 11:07:09 +01:00
Hector Sanjuan
a6c83f72ba Fix #973: update quic-transport
Did not build in windows before...
2019-12-06 10:58:30 +01:00
Hector Sanjuan
d0b2c80a15 Dockerfile formatting 2019-12-06 10:56:17 +01:00
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt
Fix: disable default libp2p API endpoint with CRDT consensus
2019-12-05 16:14:35 +01:00
Kishan Sagathiya
30fd5ee6a0 Feat: Allow and run multiple informer components (#962)
This introduces the possiblity of running Cluster with multiple informer components. The first one in the list is the used for "allocations". This is just groundwork for working with several informers in the future.
2019-12-05 15:08:43 +01:00
Hector Sanjuan
281653bdbd
Merge pull request #971 from ipfs/feat/peer-addresses
config: Add PeerAddresses
2019-12-03 13:16:37 +01:00
Hector Sanjuan
dc75343dc4
Merge pull request #969 from ipfs/feat/restapi-unix-sockets
RestAPI client: support unix sockets
2019-12-03 13:15:48 +01:00
Hector Sanjuan
f6ea95bccf Service/cmdutils: fix small typos (review) 2019-12-02 16:08:47 +01:00
Hector Sanjuan
d2bf1bc7b6 config: Add PeerAddresses
This adds a PeerAddresses entry to the main cluster configuration.

The peer will ingest and potentially connect to those peer addresses during
the start (similarly to the ones in the peerstore).

This allows to provide "bootstrap" (as in "peers we connect to") addresses
directly in the configuration, which is useful when distributing a single
configuration template that will allow a cluster peer to know where to connect
on the first boot.
2019-12-02 15:08:37 +01:00
Hector Sanjuan
5cc9ed0c4f ipfs-cluster-service: extract some helpers to cmdutils
This will allow better re-use from other CLIs.
2019-12-02 11:57:24 +01:00
Hector Sanjuan
454efe83fd ipfs-cluster-service: rename cluster node -> cluster peer in help.
We use "peer" everywhere. This is an old leftover.
2019-12-02 11:57:13 +01:00
Hector Sanjuan
ba021e8bb4 restapi: fix logging
Logging with Infof when there is nothing to format.
2019-12-02 11:56:25 +01:00
Hector Sanjuan
71217ac5d1 restapi client: support talking to unix sockets
Setting APIAddr to a Unix socket multiaddress will just work.
2019-12-02 11:56:16 +01:00
Hector Sanjuan
578ee29bdf Fix: disable default libp2p API endpoint with CRDT consensus 2019-12-02 11:54:51 +01:00
Kishan Mohanbhai Sagathiya
42b1413e23 Use index endpoint with a trailing slash
On using `/debug/pprof` as index endpoint, profiles contained by it
would point to wrong url. For example, link for `allocs` would be
`/debug/allocs` (results in 404), whereas it should actually be
`/debug/pprof/allocs`.

Use a trailing slash to get correct links. `/debug/pprof` will
automatically be redirected to `/debug/pprof/`
2019-11-28 00:39:35 +05:30
Hector Sanjuan
d628558aeb
Merge pull request #964 from ipfs/fix/checksum
Remove indirect btcd dependency
2019-11-21 11:48:06 -06:00
Hector Sanjuan
f9411539f0 go.mod: remove btcd indirect dependency
It causes a checksum mismatch error and it is not needed directly.
2019-11-21 11:46:22 -06:00
Kishan Mohanbhai Sagathiya
9bef8ea04e Bump btcd to v0.20.1-beta 2019-11-21 14:05:44 +05:30
Kishan Sagathiya
a178e5988a Avoid out of index error in failover (#951)
Fixes panic when using the load balancing client.
2019-11-13 12:15:29 +01:00
Hector Sanjuan
005bd9e2d8
Merge pull request #947 from ipfs/fix/broadcast-ops
"Sync/Status/Recover" operations should not broadcast to peers with remote pins
2019-11-13 12:02:13 +01:00
Hector Sanjuan
3a22da58ee
Merge pull request #955 from marten-seemann/update-quic
update go-libp2p-quic-transport to v0.2.1
2019-11-09 14:55:32 +01:00
Marten Seemann
dc4dc977f3 update go-libp2p-quic-transport to v0.2.1 2019-11-09 18:39:24 +07:00
Kishan Mohanbhai Sagathiya
a9910a77e2 remove unused var 2019-11-08 22:41:14 +05:30
Kishan Mohanbhai Sagathiya
cef9129d0e Fix subtract function 2019-11-08 22:35:42 +05:30
Kishan Mohanbhai Sagathiya
95016492c3 Merge branch 'master' into fix/broadcast-ops 2019-11-08 20:50:19 +05:30
Kishan Mohanbhai Sagathiya
d42c0fd651 move comment with variables 2019-11-08 20:49:06 +05:30
Kishan Mohanbhai Sagathiya
0e7ed97e59 Make it clearer 2019-11-08 19:45:01 +05:30
Hector Sanjuan
10de80a396
Merge pull request #954 from ipfs/fix/graph-all-ipfs
Fix graph not displaying all ipfs nodes when all-ipfs-peers enabled
2019-11-08 14:17:34 +01:00
Hector Sanjuan
7c5d86ee86 Fix graph not displaying all ipfs nodes when all-ipfs-peers enabled 2019-11-08 12:48:02 +01:00
Hector Sanjuan
9db16092cb
Merge pull request #953 from ipfs/fix/tests2
Multiple fixes to tests
2019-11-08 12:47:03 +01:00
Hector Sanjuan
7b4d647267 Tests: multiple fixes, increase timings 2019-11-08 12:46:11 +01:00
Hector Sanjuan
11eb08afe5
Merge pull request #950 from ipfs/fix/949-activation-threshold
Fix #949: Cluster peers should advertise their external earlier
2019-11-08 10:08:00 +01:00
Kishan Mohanbhai Sagathiya
108fcff8a9 temp 2019-11-08 12:43:01 +05:30
Hector Sanjuan
9649664dbd Run tests by default with crdt and not with raft 2019-11-07 20:19:34 +01:00
Hector Sanjuan
92bdc2e2e3 Improve comment about discovery 2019-11-07 20:19:24 +01:00
Hector Sanjuan
f788a83d23 Fix lbclient test with new multiaddresses 2019-11-07 20:19:07 +01:00
Hector Sanjuan
f9b52e02b9 Fix graph tests failing and randomly failing 2019-11-07 20:18:50 +01:00
Hector Sanjuan
a04f7fd785
Merge pull request #945 from ipfs/feat/cluster-gc
Run Garbage Collector(ipfs repo gc) on all peers
2019-11-07 18:50:21 +01:00
Hector Sanjuan
249d9007d2 Merge branch 'master' into feat/cluster-gc 2019-11-07 18:35:42 +01:00
Hector Sanjuan
df08d76c2c Fix #949: Cluster peers should advertise their external earlier
This lowers the activation threshold for an observed addressed
to 1 and increases the OwnObservedAddressTTL to infinite.

This should let cluster peers (particularly in small clusters)
advertise their known external addresses asap so that other peers
can include them in their peerstores (and save them on shutdown).

By default this only happens when an external address has received 4
connections in the last 40 minutes. In NATed environments this may
result in cluster peers advertising right away their NAT-translated address/ports
which may not be dialable, but in other cases that address seems to be
indeed dialable (docker).
2019-11-07 13:05:31 +01:00
Hector Sanjuan
834a95b44a update go-dot to latest tag 2019-11-07 12:52:17 +01:00
Hector Sanjuan
e55db51ab3
Merge pull request #925 from ipfs/feat/trust-graph
Improve connection graph output
2019-11-07 12:50:34 +01:00
Hector Sanjuan
5f265b017a Change colors and shapes 2019-11-07 12:49:51 +01:00
Hector Sanjuan
71bf98ac86 go.mod 2019-11-07 10:51:46 +01:00
Kishan Mohanbhai Sagathiya
4d8ef92b3d health/graph: Improve graph
Mark local, trusted peers. Add peernames. Improve display.
2019-11-07 10:47:29 +01:00