Commit Graph

2834 Commits

Author SHA1 Message Date
Hector Sanjuan
9c61bf6f26 Fix #983: Correctly print paths when adding 2019-12-17 23:01:25 +01:00
Hector Sanjuan
8d38f3a288 CHANGELOG for v0.12.0 2019-12-17 00:15:59 +01:00
Hector Sanjuan
66abca4e32
Release 0.12.0-rc1 2019-12-16 17:58:30 +01:00
Hector Sanjuan
2391d2d246
Merge pull request #960 from ipfs/feat/log-env-flags
Environment variable to set loglevels for individual identifiers
2019-12-16 17:20:17 +01:00
Hector Sanjuan
5cd4abe58b consensus: fix panic when getting state 2019-12-16 15:43:38 +01:00
Hector Sanjuan
7f1926d386 follow: make sure travis knows how to build the app 2019-12-16 15:13:07 +01:00
Hector Sanjuan
01973a72b1 Docker: add ipfs-cluster-follow to containers 2019-12-16 15:10:11 +01:00
Hector Sanjuan
1e47f74ef8 Follow: simplify tracing/metrics [no] setup a bit 2019-12-16 15:07:03 +01:00
Hector Sanjuan
0d5857a95a follow: support custom gateway url 2019-12-16 15:02:07 +01:00
Hector Sanjuan
a79994ed44 follow: improve ipfs connector handling in info 2019-12-16 14:55:36 +01:00
Hector Sanjuan
6aef2e8648 Follow: wait for IPFS to be running 2019-12-16 14:47:14 +01:00
Hector Sanjuan
ad7b57aec0 follow: do not show help when "info" called explicitally 2019-12-16 14:46:24 +01:00
Hector Sanjuan
1dd9207422 follow: support custom restapi endpoint in http client
by using CLUSTER_RESTAPI_HTTPLISTENMULTIADDRESS.
2019-12-16 14:22:45 +01:00
Hector Sanjuan
35c1a4895e crdt consensus: do not panic when getting the state while not ready.
Wait instead.
2019-12-16 14:22:09 +01:00
Hector Sanjuan
ad1e739bfb Cluster: follower peers should ignore alerts/re-allocations. 2019-12-16 13:42:35 +01:00
Hector Sanjuan
d2042b8092 follow: set shorhand for setting loglevels 2019-12-16 13:42:35 +01:00
Hector Sanjuan
6a35c19acf follow: better text formatting 2019-12-16 13:42:35 +01:00
Hector Sanjuan
f578145e59 follow: hide blockstore error 2019-12-16 13:42:35 +01:00
Hector Sanjuan
0671159bee Crdt: fix OfflineState comment 2019-12-16 13:42:35 +01:00
Hector Sanjuan
78f160aeb8 follow: better errors when doing "list" 2019-12-16 13:42:35 +01:00
Hector Sanjuan
47b2aadbba Cluster config: Slightly increase default pin_recover interval 2019-12-16 13:42:35 +01:00
Hector Sanjuan
f8a1e0f4b9 pintracker: take care of tests
Simplify the tests, remove things that are not used at all, align the
behaviour of the mocks, add methods to test the correct behaviour of Status
etc.
2019-12-16 13:42:35 +01:00
Hector Sanjuan
9f660ba38e Fix: stateless: cluster should pin items that are in the state but not on ipfs
StateSync() used to take care of this by issuing Track() calls. But this
functionality was removed.

This starts returning items that are in the state but not on IPFS as
PIN_ERRORs. It ensures that the Recover methods see them so that they can
trigger repinnings for missing items. This covers cases where the user
modifies the ipfs state manually, or resets the ipfs daemon but keeps the
cluster state, and cases where cluster was stopped half-way through a pinning.
2019-12-16 13:42:35 +01:00
Hector Sanjuan
c5d1de9b85
Merge pull request #979 from ipfs/fix/stateless-sync
Fix: stateless: cluster should pin items that are in the state but not on ipfs
2019-12-16 12:10:19 +01:00
Hector Sanjuan
e3cf51ac0e Cluster config: Slightly increase default pin_recover interval 2019-12-16 12:09:08 +01:00
Hector Sanjuan
09d933fde1 pintracker: take care of tests
Simplify the tests, remove things that are not used at all, align the
behaviour of the mocks, add methods to test the correct behaviour of Status
etc.
2019-12-13 12:03:01 +01:00
Hector Sanjuan
8b6fd1fabe Fix: stateless: cluster should pin items that are in the state but not on ipfs
StateSync() used to take care of this by issuing Track() calls. But this
functionality was removed.

This starts returning items that are in the state but not on IPFS as
PIN_ERRORs. It ensures that the Recover methods see them so that they can
trigger repinnings for missing items. This covers cases where the user
modifies the ipfs state manually, or resets the ipfs daemon but keeps the
cluster state, and cases where cluster was stopped half-way through a pinning.
2019-12-13 12:00:03 +01:00
Hector Sanjuan
26f553f3db follow: fix typos in comments/docs 2019-12-13 09:51:15 +01:00
Hector Sanjuan
7646cffa38 follow: update stateless tracker creation to new signature 2019-12-13 09:50:44 +01:00
Hector Sanjuan
6ad8974bde cluster: Shutdown informer components 2019-12-13 09:48:12 +01:00
Hector Sanjuan
31f4afadea cluster: Improve Shutdown() docstring 2019-12-13 09:47:42 +01:00
Hector Sanjuan
c8b7409685 follow: makefile: fix "all" rule 2019-12-13 09:47:01 +01:00
Kishan Mohanbhai Sagathiya
618ebd23f4 Check expiry in alert 2019-12-13 12:25:28 +05:30
Kishan Mohanbhai Sagathiya
d21860eee7 Merge branch 'master' into feat/alerts 2019-12-13 10:22:06 +05:30
Hector Sanjuan
3bf77055df Merge remote-tracking branch 'origin/master' into feat/ipfs-cluster-follow 2019-12-12 21:32:02 +01:00
Kishan Sagathiya
5258a4d428 Remove map pintracker (#944)
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.

Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
2019-12-12 21:22:54 +01:00
Kishan Mohanbhai Sagathiya
04069b8c81 Introduction of ctl alerts
- basic version, just alerts if peers are down
2019-12-13 00:21:28 +05:30
Hector Sanjuan
2da2dc8c86
Merge pull request #956 from ipfs/tests/re-enable-quic
Tests: re-enable quic listen address on tests
2019-12-09 09:32:56 +01:00
Hector Sanjuan
5121a30770
Release 0.12.0-alpha1 2019-12-07 15:46:04 +01:00
Hector Sanjuan
4ea830f74e Feat: ipfs-cluster-follow
This adds a new cluster command: ipfs-cluster-follow.

This command allows initializing and running follower peers as configured by a
remote-source configuration. The command can list configured peers
and obtain information for each of them.

Peers are launched with the rest API listening on a local unix socket. The
command can be run to list the items in the cluster pinset using this
endpoint. Alternatively, if no socket is present, the peer will be assumed to
be offline and the pin list will be directly read from the datastore.

Cluster peers launched with this command (and their configurations) are
compatible with ipfs-cluster-ctl and ipfs-cluster-service. We purposely do not
support most configuration options here. Using ipfs-cluster-ctl or launching
the peers using ipfs-cluster-service is always an option when the usecase
deviates from that supported by ipfs-cluster-follow.

Examples:

$ ipfs-cluster-follow -> list configured peers
$ ipfs-cluster-follow --help
$ ipfs-cluster-follow <clusterName> init <url>
$ ipfs-cluster-follow <clusterName> info
$ ipfs-cluster-follow <clusterName> run
$ ipfs-cluster-follow <clusterName> list
2019-12-07 15:38:59 +01:00
Hector Sanjuan
ce3c50187e
config manager: Improve source-fetch error handling (#967)
config manager: Improve source-fetch error handling

* Allow detection of source-download errors by adding IsSourceDownloadError method.
* Treat non-success response codes as a failure to fetch the source.
* Set the Source url before errors happens so it can be consulted even in that case.
2019-12-07 12:32:43 +01:00
Hector Sanjuan
cf0afdcbc3 Tests: re-enable quic listen address on tests 2019-12-07 12:26:55 +01:00
Kishan Sagathiya
e1faf12bae ipfsproxy: hijack repo/gc and trigger cluster-wide GC
This adds hijacking of the repo/gc endpoint to the proxy to do cluster-wide gc.
2019-12-06 13:08:57 +01:00
Hector Sanjuan
db352534df
Merge pull request #975 from ipfs/update/quic2
Update QUIC transport dependency
2019-12-06 11:27:26 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils
Service: extract some methods to cmdutils
2019-12-06 11:08:58 +01:00
Hector Sanjuan
61778f50fd
Merge pull request #965 from ipfs/fix/pprof
Use index endpoint with a trailing slash
2019-12-06 11:07:09 +01:00
Hector Sanjuan
a6c83f72ba Fix #973: update quic-transport
Did not build in windows before...
2019-12-06 10:58:30 +01:00
Hector Sanjuan
d0b2c80a15 Dockerfile formatting 2019-12-06 10:56:17 +01:00
Kishan Mohanbhai Sagathiya
a136509522 No need to verify log levels 2019-12-06 11:22:08 +05:30
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt
Fix: disable default libp2p API endpoint with CRDT consensus
2019-12-05 16:14:35 +01:00