Commit Graph

1941 Commits

Author SHA1 Message Date
Hector Sanjuan
212b3b00d2 follow: handle error from ApplyEnvVars 2019-12-19 19:55:19 +01:00
Hector Sanjuan
10e90616f5 follow: rename listCmd to listClustersCmd and pinsetCmd to listCmd 2019-12-19 19:53:30 +01:00
Hector Sanjuan
130965d07f follow: show the notInitialized message when not initialized. 2019-12-19 19:50:46 +01:00
Hector Sanjuan
bf2c950016 Fix #986: Ensure ctrl-c always kills ipfs-cluster-follow 2019-12-19 18:34:18 +01:00
Hector Sanjuan
d12bc7daf7 Merge branch 'master' into feat/ipfs-cluster-follow 2019-12-18 13:49:00 +01:00
Hector Sanjuan
bc506fda62
Merge pull request #985 from ipfs/fix/peer-addresses-null
cluster config: initialize peer addresses to [] rather than null.
2019-12-18 13:48:07 +01:00
Hector Sanjuan
b97efd146e
Merge pull request #984 from ipfs/fix/983-add-paths
Fix #983: Correctly print paths when adding
2019-12-18 13:47:45 +01:00
Hector Sanjuan
293086d798 cluster config: initialize peer addresses to [] rather than null.
User should see this is an array.
2019-12-18 11:41:34 +01:00
Hector Sanjuan
6f3a551b33 Adder: improve comment 2019-12-18 11:32:28 +01:00
Hector Sanjuan
6e4cbc7b6e Sharness: compare files and folder labels on adding against ipfs 2019-12-18 00:24:45 +01:00
Hector Sanjuan
442b441985 Fix #983: Make sure single-file labels correspond to their name 2019-12-18 00:24:16 +01:00
Hector Sanjuan
9c61bf6f26 Fix #983: Correctly print paths when adding 2019-12-17 23:01:25 +01:00
Hector Sanjuan
66abca4e32
Release 0.12.0-rc1 2019-12-16 17:58:30 +01:00
Hector Sanjuan
2391d2d246
Merge pull request #960 from ipfs/feat/log-env-flags
Environment variable to set loglevels for individual identifiers
2019-12-16 17:20:17 +01:00
Hector Sanjuan
5cd4abe58b consensus: fix panic when getting state 2019-12-16 15:43:38 +01:00
Hector Sanjuan
7f1926d386 follow: make sure travis knows how to build the app 2019-12-16 15:13:07 +01:00
Hector Sanjuan
01973a72b1 Docker: add ipfs-cluster-follow to containers 2019-12-16 15:10:11 +01:00
Hector Sanjuan
1e47f74ef8 Follow: simplify tracing/metrics [no] setup a bit 2019-12-16 15:07:03 +01:00
Hector Sanjuan
0d5857a95a follow: support custom gateway url 2019-12-16 15:02:07 +01:00
Hector Sanjuan
a79994ed44 follow: improve ipfs connector handling in info 2019-12-16 14:55:36 +01:00
Hector Sanjuan
6aef2e8648 Follow: wait for IPFS to be running 2019-12-16 14:47:14 +01:00
Hector Sanjuan
ad7b57aec0 follow: do not show help when "info" called explicitally 2019-12-16 14:46:24 +01:00
Hector Sanjuan
1dd9207422 follow: support custom restapi endpoint in http client
by using CLUSTER_RESTAPI_HTTPLISTENMULTIADDRESS.
2019-12-16 14:22:45 +01:00
Hector Sanjuan
35c1a4895e crdt consensus: do not panic when getting the state while not ready.
Wait instead.
2019-12-16 14:22:09 +01:00
Hector Sanjuan
ad1e739bfb Cluster: follower peers should ignore alerts/re-allocations. 2019-12-16 13:42:35 +01:00
Hector Sanjuan
d2042b8092 follow: set shorhand for setting loglevels 2019-12-16 13:42:35 +01:00
Hector Sanjuan
6a35c19acf follow: better text formatting 2019-12-16 13:42:35 +01:00
Hector Sanjuan
f578145e59 follow: hide blockstore error 2019-12-16 13:42:35 +01:00
Hector Sanjuan
0671159bee Crdt: fix OfflineState comment 2019-12-16 13:42:35 +01:00
Hector Sanjuan
78f160aeb8 follow: better errors when doing "list" 2019-12-16 13:42:35 +01:00
Hector Sanjuan
47b2aadbba Cluster config: Slightly increase default pin_recover interval 2019-12-16 13:42:35 +01:00
Hector Sanjuan
f8a1e0f4b9 pintracker: take care of tests
Simplify the tests, remove things that are not used at all, align the
behaviour of the mocks, add methods to test the correct behaviour of Status
etc.
2019-12-16 13:42:35 +01:00
Hector Sanjuan
9f660ba38e Fix: stateless: cluster should pin items that are in the state but not on ipfs
StateSync() used to take care of this by issuing Track() calls. But this
functionality was removed.

This starts returning items that are in the state but not on IPFS as
PIN_ERRORs. It ensures that the Recover methods see them so that they can
trigger repinnings for missing items. This covers cases where the user
modifies the ipfs state manually, or resets the ipfs daemon but keeps the
cluster state, and cases where cluster was stopped half-way through a pinning.
2019-12-16 13:42:35 +01:00
Hector Sanjuan
c5d1de9b85
Merge pull request #979 from ipfs/fix/stateless-sync
Fix: stateless: cluster should pin items that are in the state but not on ipfs
2019-12-16 12:10:19 +01:00
Hector Sanjuan
e3cf51ac0e Cluster config: Slightly increase default pin_recover interval 2019-12-16 12:09:08 +01:00
Hector Sanjuan
09d933fde1 pintracker: take care of tests
Simplify the tests, remove things that are not used at all, align the
behaviour of the mocks, add methods to test the correct behaviour of Status
etc.
2019-12-13 12:03:01 +01:00
Hector Sanjuan
8b6fd1fabe Fix: stateless: cluster should pin items that are in the state but not on ipfs
StateSync() used to take care of this by issuing Track() calls. But this
functionality was removed.

This starts returning items that are in the state but not on IPFS as
PIN_ERRORs. It ensures that the Recover methods see them so that they can
trigger repinnings for missing items. This covers cases where the user
modifies the ipfs state manually, or resets the ipfs daemon but keeps the
cluster state, and cases where cluster was stopped half-way through a pinning.
2019-12-13 12:00:03 +01:00
Hector Sanjuan
26f553f3db follow: fix typos in comments/docs 2019-12-13 09:51:15 +01:00
Hector Sanjuan
7646cffa38 follow: update stateless tracker creation to new signature 2019-12-13 09:50:44 +01:00
Hector Sanjuan
6ad8974bde cluster: Shutdown informer components 2019-12-13 09:48:12 +01:00
Hector Sanjuan
31f4afadea cluster: Improve Shutdown() docstring 2019-12-13 09:47:42 +01:00
Hector Sanjuan
c8b7409685 follow: makefile: fix "all" rule 2019-12-13 09:47:01 +01:00
Hector Sanjuan
3bf77055df Merge remote-tracking branch 'origin/master' into feat/ipfs-cluster-follow 2019-12-12 21:32:02 +01:00
Kishan Sagathiya
5258a4d428 Remove map pintracker (#944)
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.

Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
2019-12-12 21:22:54 +01:00
Hector Sanjuan
2da2dc8c86
Merge pull request #956 from ipfs/tests/re-enable-quic
Tests: re-enable quic listen address on tests
2019-12-09 09:32:56 +01:00
Hector Sanjuan
5121a30770
Release 0.12.0-alpha1 2019-12-07 15:46:04 +01:00
Hector Sanjuan
4ea830f74e Feat: ipfs-cluster-follow
This adds a new cluster command: ipfs-cluster-follow.

This command allows initializing and running follower peers as configured by a
remote-source configuration. The command can list configured peers
and obtain information for each of them.

Peers are launched with the rest API listening on a local unix socket. The
command can be run to list the items in the cluster pinset using this
endpoint. Alternatively, if no socket is present, the peer will be assumed to
be offline and the pin list will be directly read from the datastore.

Cluster peers launched with this command (and their configurations) are
compatible with ipfs-cluster-ctl and ipfs-cluster-service. We purposely do not
support most configuration options here. Using ipfs-cluster-ctl or launching
the peers using ipfs-cluster-service is always an option when the usecase
deviates from that supported by ipfs-cluster-follow.

Examples:

$ ipfs-cluster-follow -> list configured peers
$ ipfs-cluster-follow --help
$ ipfs-cluster-follow <clusterName> init <url>
$ ipfs-cluster-follow <clusterName> info
$ ipfs-cluster-follow <clusterName> run
$ ipfs-cluster-follow <clusterName> list
2019-12-07 15:38:59 +01:00
Hector Sanjuan
ce3c50187e
config manager: Improve source-fetch error handling (#967)
config manager: Improve source-fetch error handling

* Allow detection of source-download errors by adding IsSourceDownloadError method.
* Treat non-success response codes as a failure to fetch the source.
* Set the Source url before errors happens so it can be consulted even in that case.
2019-12-07 12:32:43 +01:00
Hector Sanjuan
cf0afdcbc3 Tests: re-enable quic listen address on tests 2019-12-07 12:26:55 +01:00
Kishan Sagathiya
e1faf12bae ipfsproxy: hijack repo/gc and trigger cluster-wide GC
This adds hijacking of the repo/gc endpoint to the proxy to do cluster-wide gc.
2019-12-06 13:08:57 +01:00