Commit Graph

2148 Commits

Author SHA1 Message Date
Kishan Sagathiya
5258a4d428 Remove map pintracker (#944)
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.

Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
2019-12-12 21:22:54 +01:00
Hector Sanjuan
2da2dc8c86
Merge pull request #956 from ipfs/tests/re-enable-quic
Tests: re-enable quic listen address on tests
2019-12-09 09:32:56 +01:00
Hector Sanjuan
5121a30770
Release 0.12.0-alpha1 2019-12-07 15:46:04 +01:00
Hector Sanjuan
4ea830f74e Feat: ipfs-cluster-follow
This adds a new cluster command: ipfs-cluster-follow.

This command allows initializing and running follower peers as configured by a
remote-source configuration. The command can list configured peers
and obtain information for each of them.

Peers are launched with the rest API listening on a local unix socket. The
command can be run to list the items in the cluster pinset using this
endpoint. Alternatively, if no socket is present, the peer will be assumed to
be offline and the pin list will be directly read from the datastore.

Cluster peers launched with this command (and their configurations) are
compatible with ipfs-cluster-ctl and ipfs-cluster-service. We purposely do not
support most configuration options here. Using ipfs-cluster-ctl or launching
the peers using ipfs-cluster-service is always an option when the usecase
deviates from that supported by ipfs-cluster-follow.

Examples:

$ ipfs-cluster-follow -> list configured peers
$ ipfs-cluster-follow --help
$ ipfs-cluster-follow <clusterName> init <url>
$ ipfs-cluster-follow <clusterName> info
$ ipfs-cluster-follow <clusterName> run
$ ipfs-cluster-follow <clusterName> list
2019-12-07 15:38:59 +01:00
Hector Sanjuan
ce3c50187e
config manager: Improve source-fetch error handling (#967)
config manager: Improve source-fetch error handling

* Allow detection of source-download errors by adding IsSourceDownloadError method.
* Treat non-success response codes as a failure to fetch the source.
* Set the Source url before errors happens so it can be consulted even in that case.
2019-12-07 12:32:43 +01:00
Hector Sanjuan
cf0afdcbc3 Tests: re-enable quic listen address on tests 2019-12-07 12:26:55 +01:00
Kishan Sagathiya
e1faf12bae ipfsproxy: hijack repo/gc and trigger cluster-wide GC
This adds hijacking of the repo/gc endpoint to the proxy to do cluster-wide gc.
2019-12-06 13:08:57 +01:00
Hector Sanjuan
db352534df
Merge pull request #975 from ipfs/update/quic2
Update QUIC transport dependency
2019-12-06 11:27:26 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils
Service: extract some methods to cmdutils
2019-12-06 11:08:58 +01:00
Hector Sanjuan
61778f50fd
Merge pull request #965 from ipfs/fix/pprof
Use index endpoint with a trailing slash
2019-12-06 11:07:09 +01:00
Hector Sanjuan
a6c83f72ba Fix #973: update quic-transport
Did not build in windows before...
2019-12-06 10:58:30 +01:00
Hector Sanjuan
d0b2c80a15 Dockerfile formatting 2019-12-06 10:56:17 +01:00
Kishan Mohanbhai Sagathiya
a136509522 No need to verify log levels 2019-12-06 11:22:08 +05:30
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt
Fix: disable default libp2p API endpoint with CRDT consensus
2019-12-05 16:14:35 +01:00
Kishan Sagathiya
30fd5ee6a0 Feat: Allow and run multiple informer components (#962)
This introduces the possiblity of running Cluster with multiple informer components. The first one in the list is the used for "allocations". This is just groundwork for working with several informers in the future.
2019-12-05 15:08:43 +01:00
Kishan Mohanbhai Sagathiya
df36995213 detail description and example for how to customize logging 2019-12-04 23:22:38 +05:30
Kishan Mohanbhai Sagathiya
794646ff12 Addressed reviews 2019-12-04 23:10:35 +05:30
Hector Sanjuan
281653bdbd
Merge pull request #971 from ipfs/feat/peer-addresses
config: Add PeerAddresses
2019-12-03 13:16:37 +01:00
Hector Sanjuan
dc75343dc4
Merge pull request #969 from ipfs/feat/restapi-unix-sockets
RestAPI client: support unix sockets
2019-12-03 13:15:48 +01:00
Hector Sanjuan
f6ea95bccf Service/cmdutils: fix small typos (review) 2019-12-02 16:08:47 +01:00
Hector Sanjuan
d2bf1bc7b6 config: Add PeerAddresses
This adds a PeerAddresses entry to the main cluster configuration.

The peer will ingest and potentially connect to those peer addresses during
the start (similarly to the ones in the peerstore).

This allows to provide "bootstrap" (as in "peers we connect to") addresses
directly in the configuration, which is useful when distributing a single
configuration template that will allow a cluster peer to know where to connect
on the first boot.
2019-12-02 15:08:37 +01:00
Hector Sanjuan
5cc9ed0c4f ipfs-cluster-service: extract some helpers to cmdutils
This will allow better re-use from other CLIs.
2019-12-02 11:57:24 +01:00
Hector Sanjuan
454efe83fd ipfs-cluster-service: rename cluster node -> cluster peer in help.
We use "peer" everywhere. This is an old leftover.
2019-12-02 11:57:13 +01:00
Hector Sanjuan
ba021e8bb4 restapi: fix logging
Logging with Infof when there is nothing to format.
2019-12-02 11:56:25 +01:00
Hector Sanjuan
71217ac5d1 restapi client: support talking to unix sockets
Setting APIAddr to a Unix socket multiaddress will just work.
2019-12-02 11:56:16 +01:00
Hector Sanjuan
578ee29bdf Fix: disable default libp2p API endpoint with CRDT consensus 2019-12-02 11:54:51 +01:00
Kishan Mohanbhai Sagathiya
42b1413e23 Use index endpoint with a trailing slash
On using `/debug/pprof` as index endpoint, profiles contained by it
would point to wrong url. For example, link for `allocs` would be
`/debug/allocs` (results in 404), whereas it should actually be
`/debug/pprof/allocs`.

Use a trailing slash to get correct links. `/debug/pprof` will
automatically be redirected to `/debug/pprof/`
2019-11-28 00:39:35 +05:30
Kishan Mohanbhai Sagathiya
e99abf7688 Do not modify global var ipfscluster.LoggingFacilities 2019-11-25 17:18:50 +05:30
Kishan Mohanbhai Sagathiya
3d625184a3 Keep debug first to avoid iterations 2019-11-25 17:10:46 +05:30
Kishan Mohanbhai Sagathiya
a384057d39 Merge branch 'master' into feat/log-env-flags 2019-11-25 16:58:43 +05:30
Kishan Mohanbhai Sagathiya
f0c83f59b4 Don't error if a loglevel is passed twice, but warn 2019-11-25 16:57:48 +05:30
Kishan Mohanbhai Sagathiya
047d3fe615 Use a single log-level flag 2019-11-25 16:41:49 +05:30
Kishan Mohanbhai Sagathiya
361b819139 Try to set all loglevels with a single flag 2019-11-25 12:57:59 +05:30
Hector Sanjuan
d628558aeb
Merge pull request #964 from ipfs/fix/checksum
Remove indirect btcd dependency
2019-11-21 11:48:06 -06:00
Hector Sanjuan
f9411539f0 go.mod: remove btcd indirect dependency
It causes a checksum mismatch error and it is not needed directly.
2019-11-21 11:46:22 -06:00
Kishan Mohanbhai Sagathiya
9bef8ea04e Bump btcd to v0.20.1-beta 2019-11-21 14:05:44 +05:30
Kishan Mohanbhai Sagathiya
1fd4e41c99 Use urfave/cli's environment variable support for loglevels
- introduce a new flag for loglevels for individual components
- handle all log related flags together so that we can maintain priority
among log level flags (loglevel < component-loglevel < debug)
2019-11-16 15:37:57 +05:30
Kishan Mohanbhai Sagathiya
d8929aa5a6 Environment variable to set loglevels for individual identifiers
- Example: export LOG_FACS="cluster:debug,raft:debug"
2019-11-14 22:54:26 +05:30
Kishan Sagathiya
a178e5988a Avoid out of index error in failover (#951)
Fixes panic when using the load balancing client.
2019-11-13 12:15:29 +01:00
Hector Sanjuan
005bd9e2d8
Merge pull request #947 from ipfs/fix/broadcast-ops
"Sync/Status/Recover" operations should not broadcast to peers with remote pins
2019-11-13 12:02:13 +01:00
Hector Sanjuan
3a22da58ee
Merge pull request #955 from marten-seemann/update-quic
update go-libp2p-quic-transport to v0.2.1
2019-11-09 14:55:32 +01:00
Marten Seemann
dc4dc977f3 update go-libp2p-quic-transport to v0.2.1 2019-11-09 18:39:24 +07:00
Kishan Mohanbhai Sagathiya
a9910a77e2 remove unused var 2019-11-08 22:41:14 +05:30
Kishan Mohanbhai Sagathiya
cef9129d0e Fix subtract function 2019-11-08 22:35:42 +05:30
Kishan Mohanbhai Sagathiya
95016492c3 Merge branch 'master' into fix/broadcast-ops 2019-11-08 20:50:19 +05:30
Kishan Mohanbhai Sagathiya
d42c0fd651 move comment with variables 2019-11-08 20:49:06 +05:30
Kishan Mohanbhai Sagathiya
0e7ed97e59 Make it clearer 2019-11-08 19:45:01 +05:30
Hector Sanjuan
10de80a396
Merge pull request #954 from ipfs/fix/graph-all-ipfs
Fix graph not displaying all ipfs nodes when all-ipfs-peers enabled
2019-11-08 14:17:34 +01:00
Hector Sanjuan
7c5d86ee86 Fix graph not displaying all ipfs nodes when all-ipfs-peers enabled 2019-11-08 12:48:02 +01:00
Hector Sanjuan
9db16092cb
Merge pull request #953 from ipfs/fix/tests2
Multiple fixes to tests
2019-11-08 12:47:03 +01:00