IPFS Cluster v1.0.6 is a maintenance release with some small fixes. The main
change in this release is that `pebble` becomes the default datastore backend,
as we mentioned in the last release.
Pebble is the datastore backend used by CockroachDB and is inspired in
RocksDB. Upon testing, Pebble has demonstrated good performance and optimal
disk usage. Pebble incorporates modern datastore-backend features such as
compression, caching and bloom filters. Pebble is actively maintained by the
CockroachDB team and therefore seems like the best default choice for IPFS
Cluster.
Badger3, a very good alternative choice, becomes the new default for platforms
not supported by Pebble (mainly 32bit architectures). Badger and LevelDB are
still supported, but we heavily disencourage their usage for new Cluster peers.
#### List of changes
##### Breaking changes
There are no breaking changes on this release.
##### Features
* Pebble: make the new default | [ipfs/ipfs-cluster#1881](https://github.com/ipfs/ipfs-cluster/issues/1881) | [ipfs/ipfs-cluster#1847](https://github.com/ipfs/ipfs-cluster/issues/1847)
* ctl: support `ipfs-cluster-ctl add --no-pin` flag | [ipfs/ipfs-cluster#1852](https://github.com/ipfs/ipfs-cluster/issues/1852)
##### Bug fixes
* PinStatus information not up-to-date for re-pinned items with ongoing operations | [ipfs/ipfs-cluster#1785](https://github.com/ipfs/ipfs-cluster/issues/1785) | [ipfs/ipfs-cluster#1876](https://github.com/ipfs/ipfs-cluster/issues/1876)
* Fix error building docker container with newer git | [ipfs/ipfs-cluster#1836](https://github.com/ipfs/ipfs-cluster/issues/1836)
* Fix building master due to pinned go-mfs dependency | [ipfs/ipfs-cluster#1817](https://github.com/ipfs/ipfs-cluster/issues/1817) | [ipfs/ipfs-cluster#1818](https://github.com/ipfs/ipfs-cluster/issues/1818)
IPFS Cluster v1.0.4 is a maintenance release addressing a couple of bugs and
adding more "state crdt" commands.
One of the bugs has potential to cause a panic, while a second one can
potentially dead-lock pinning operations and hang new pinning requests. We
recommend all users to upgrade as soon as possible.
#### List of changes
##### Breaking changes
There are no breaking changes on this release.
##### Features
* service: add "state crdt info/mark-dirty/mark-clean" commands | [ipfs/ipfs-cluster#1771](https://github.com/ipfs/ipfs-cluster/issues/1771)
##### Bug fixes
* Fix panic returned when request is canceled while rate-limited | [ipfs/ipfs-cluster#1770](https://github.com/ipfs/ipfs-cluster/issues/1770)
* Fix operationtracker not respecting context cancellations and never returning locks | [ipfs/ipfs-cluster#1768](https://github.com/ipfs/ipfs-cluster/issues/1768)
* Fix repinning of a CID not re-allocating errored peers as expected | [ipfs/ipfs-cluster#1774](https://github.com/ipfs/ipfs-cluster/issues/1774)
##### Other changes
No other changes.
#### Upgrading notices
##### Configuration changes
There are no configuration changes for this release.
IPFS Cluster v1.0.3 is a maintenance release addressing some bugs and bringing
some improvements to error handling behavior, as well as a couple of small
features.
This release upgrades to the latest libp2p release (v0.22.0).
#### List of changes
##### Breaking changes
There are no breaking changes on this release.
##### Features
* ipfs proxy api: intercept and trigger cluster-pins on `/block/put` and `/dag/put` requests | [ipfs/ipfs-cluster#1738](https://github.com/ipfs/ipfs-cluster/issues/1738) | [ipfs/ipfs-cluster#1756](https://github.com/ipfs/ipfs-cluster/issues/1756)
* "ipfs-cluster-service state crdt dot" command generates a DOT file from the CRDT DAG | [ipfs/ipfs-cluster#1516](https://github.com/ipfs/ipfs-cluster/issues/1516)
##### Bug fixes
* Fix leaking goroutines on aborted /add requests | [ipfs/ipfs-cluster#1732](https://github.com/ipfs/ipfs-cluster/issues/1732)
* Fix ARM build panics because of unaligned structs | [ipfs/ipfs-cluster#1735](https://github.com/ipfs/ipfs-cluster/issues/1735) | [ipfs/ipfs-cluster#1736](https://github.com/ipfs/ipfs-cluster/issues/1736) | [ipfs/ipfs-cluster#1754](https://github.com/ipfs/ipfs-cluster/issues/1754)
* Fix "blockPut response CID XXX does not match the multihash of any blocks sent" warning wrongly displayed | [ipfs/ipfs-cluster#1706](https://github.com/ipfs/ipfs-cluster/issues/1706) | [ipfs/ipfs-cluster#1755](https://github.com/ipfs/ipfs-cluster/issues/1755)
* Fix behavior and error handling when IPFS is unavailable | [ipfs/ipfs-cluster#1733](https://github.com/ipfs/ipfs-cluster/issues/1733) | [ipfs/ipfs-cluster#1762](https://github.com/ipfs/ipfs-cluster/issues/1762)
* ctl: fix "Exp" label showing the pin timestamp instead of the experiation date | [ipfs/ipfs-cluster#1666](https://github.com/ipfs/ipfs-cluster/issues/1666) | [ipfs/ipfs-cluster#1716](https://github.com/ipfs/ipfs-cluster/issues/1716)
* Fix: state import can result in different CRDT-heads | [ipfs-cluster/ipfs-cluster#1547](https://github.com/ipfs-cluster/ipfs-cluster/issues/1547) | [ipfs-cluster/ipfs-cluster#1664](https://github.com/ipfs-cluster/ipfs-cluster/issues/1664)
This enables the Pinqueue Informer, which broadcasts metrics containing the size of the pinqueue with the metric weight divided by `weight_bucket_size`. The new metric is not used for allocations by default, and it needs to be manually added to the `allocate_by` option in the allocator, usually like:
* restapi/client: support querying status for multiple CIDs | [ipfs-cluster/ipfs-cluster#1564](https://github.com/ipfs-cluster/ipfs-cluster/issues/1564) | [ipfs-cluster/ipfs-cluster#1592](https://github.com/ipfs-cluster/ipfs-cluster/issues/1592)
* Pinning Services API | [ipfs-cluster/ipfs-cluster#1213](https://github.com/ipfs-cluster/ipfs-cluster/issues/1213) | [ipfs-cluster/ipfs-cluster#1483](https://github.com/ipfs-cluster/ipfs-cluster/issues/1483)
* ipfs-cluster-ctl should exit with status 1 when an argument error happens | [ipfs-cluster/ipfs-cluster#1633](https://github.com/ipfs-cluster/ipfs-cluster/issues/1633) | [ipfs-cluster/ipfs-cluster#1634](https://github.com/ipfs-cluster/ipfs-cluster/issues/1634)
* CRDT: update with RepairInterval option and more workers | [ipfs-cluster/ipfs-cluster#1561](https://github.com/ipfs-cluster/ipfs-cluster/issues/1561) | [ipfs-cluster/ipfs-cluster#1576](https://github.com/ipfs-cluster/ipfs-cluster/issues/1576)
* Add `?cids` query parameter to /pins: limit status request to several CIDs | [ipfs-cluster/ipfs-cluster#1562](https://github.com/ipfs-cluster/ipfs-cluster/issues/1562)
* Fix: logging was too noisy | [ipfs-cluster/ipfs-cluster#1581](https://github.com/ipfs-cluster/ipfs-cluster/issues/1581) | [ipfs-cluster/ipfs-cluster#1579](https://github.com/ipfs-cluster/ipfs-cluster/issues/1579)
* Remove warning message about informer metrics | [ipfs-cluster/ipfs-cluster#1543](https://github.com/ipfs-cluster/ipfs-cluster/issues/1543)
* Fix: faster shutdown by aborting state list on context cancellation | [ipfs-cluster/ipfs-cluster#1555](https://github.com/ipfs-cluster/ipfs-cluster/issues/1555)
* Leave peername empty when unknown on status response | [ipfs-cluster/ipfs-cluster#1569](https://github.com/ipfs-cluster/ipfs-cluster/issues/1569) | [ipfs-cluster/ipfs-cluster#1575](https://github.com/ipfs-cluster/ipfs-cluster/issues/1575)
* Fix comment in graphs.go | [ipfs-cluster/ipfs-cluster#1570](https://github.com/ipfs-cluster/ipfs-cluster/issues/1570) | [ipfs-cluster/ipfs-cluster#1574](https://github.com/ipfs-cluster/ipfs-cluster/issues/1574)
* Make `/add?local=true` requests forcefully allocate to local peer | [ipfs-cluster/ipfs-cluster#1560](https://github.com/ipfs-cluster/ipfs-cluster/issues/1560)
* ipfs-cluster-service: buffered i/o on state import/export | [ipfs-cluster/ipfs-cluster#1517](https://github.com/ipfs-cluster/ipfs-cluster/issues/1517)
* Performance improvements to state list and logging for large states | [ipfs-cluster/ipfs-cluster#1510](https://github.com/ipfs-cluster/ipfs-cluster/issues/1510)
* Add timestamps to pin objects | [ipfs-cluster/ipfs-cluster#1484](https://github.com/ipfs-cluster/ipfs-cluster/issues/1484) | [ipfs-cluster/ipfs-cluster#989](https://github.com/ipfs-cluster/ipfs-cluster/issues/989)
* Support priority pinning for recent pins with small number of retries | [ipfs-cluster/ipfs-cluster#1469](https://github.com/ipfs-cluster/ipfs-cluster/issues/1469) | [ipfs-cluster/ipfs-cluster#1490](https://github.com/ipfs-cluster/ipfs-cluster/issues/1490)
* Improve support for pre-0.14.0 peers | [ipfs-cluster/ipfs-cluster#1409](https://github.com/ipfs-cluster/ipfs-cluster/issues/1409) | [ipfs-cluster/ipfs-cluster#1446](https://github.com/ipfs-cluster/ipfs-cluster/issues/1446)
* ctl: --wait returns as soon as replication-factor-min is reached | [ipfs-cluster/ipfs-cluster#1427](https://github.com/ipfs-cluster/ipfs-cluster/issues/1427) | [ipfs-cluster/ipfs-cluster#1444](https://github.com/ipfs-cluster/ipfs-cluster/issues/1444)
* Fix some data races in tests | [ipfs-cluster/ipfs-cluster#1428](https://github.com/ipfs-cluster/ipfs-cluster/issues/1428)
* Do not block peer startup while waiting for RecoverAll | [ipfs-cluster/ipfs-cluster#1436](https://github.com/ipfs-cluster/ipfs-cluster/issues/1436) | [ipfs-cluster/ipfs-cluster#1438](https://github.com/ipfs-cluster/ipfs-cluster/issues/1438)
* Use HTTP 307-redirects on restapi paths ending with "/" | [ipfs-cluster/ipfs-cluster#1415](https://github.com/ipfs-cluster/ipfs-cluster/issues/1415) | [ipfs-cluster/ipfs-cluster#1445](https://github.com/ipfs-cluster/ipfs-cluster/issues/1445)
* Set disable_repinning to true by default, for new configurations | [ipfs-cluster/ipfs-cluster#1398](https://github.com/ipfs-cluster/ipfs-cluster/issues/1398)
* Efficient status queries with filters | [ipfs-cluster/ipfs-cluster#1360](https://github.com/ipfs-cluster/ipfs-cluster/issues/1360) | [ipfs-cluster/ipfs-cluster#1377](https://github.com/ipfs-cluster/ipfs-cluster/issues/1377) | [ipfs-cluster/ipfs-cluster#1399](https://github.com/ipfs-cluster/ipfs-cluster/issues/1399)
* Provide darwin/arm64 binaries (Apple M1). Needs testing! | [ipfs-cluster/ipfs-cluster#1369](https://github.com/ipfs-cluster/ipfs-cluster/issues/1369)
* Set the "size" field in the response when adding CARs when the archive contains a single unixfs file | [ipfs-cluster/ipfs-cluster#1362](https://github.com/ipfs-cluster/ipfs-cluster/issues/1362) | [ipfs-cluster/ipfs-cluster#1372](https://github.com/ipfs-cluster/ipfs-cluster/issues/1372)
* Support a leveldb-datastore backend | [ipfs-cluster/ipfs-cluster#1364](https://github.com/ipfs-cluster/ipfs-cluster/issues/1364) | [ipfs-cluster/ipfs-cluster#1373](https://github.com/ipfs-cluster/ipfs-cluster/issues/1373)
* Speed up pin/ls by not filtering when not needed | [ipfs-cluster/ipfs-cluster#1405](https://github.com/ipfs-cluster/ipfs-cluster/issues/1405)
* Badger datastore takes too much size | [ipfs-cluster/ipfs-cluster#1320](https://github.com/ipfs-cluster/ipfs-cluster/issues/1320) | [ipfs-cluster/ipfs-cluster#1370](https://github.com/ipfs-cluster/ipfs-cluster/issues/1370)
* Fix: error-type responses from the IPFS proxy not understood by ipfs | [ipfs-cluster/ipfs-cluster#1366](https://github.com/ipfs-cluster/ipfs-cluster/issues/1366) | [ipfs-cluster/ipfs-cluster#1371](https://github.com/ipfs-cluster/ipfs-cluster/issues/1371)
* Fix: adding with cid-version=1 does not automagically set raw-leaves | [ipfs-cluster/ipfs-cluster#1358](https://github.com/ipfs-cluster/ipfs-cluster/issues/1358) | [ipfs-cluster/ipfs-cluster#1359](https://github.com/ipfs-cluster/ipfs-cluster/issues/1359)
* Tests: close datastore on test node shutdown | [ipfs-cluster/ipfs-cluster#1389](https://github.com/ipfs-cluster/ipfs-cluster/issues/1389)
* Fix ipfs-cluster-ctl not using dns name when talking to remote https endpoints | [ipfs-cluster/ipfs-cluster#1403](https://github.com/ipfs-cluster/ipfs-cluster/issues/1403) | [ipfs-cluster/ipfs-cluster#1404](https://github.com/ipfs-cluster/ipfs-cluster/issues/1404)
* Stop using secio in REST API libp2p server and client | [ipfs-cluster/ipfs-cluster#1315](https://github.com/ipfs-cluster/ipfs-cluster/issues/1315) | [ipfs-cluster/ipfs-cluster#1316](https://github.com/ipfs-cluster/ipfs-cluster/issues/1316)
* CID status wrongly reported as REMOTE | [ipfs-cluster/ipfs-cluster#1319](https://github.com/ipfs-cluster/ipfs-cluster/issues/1319) | [ipfs-cluster/ipfs-cluster#1331](https://github.com/ipfs-cluster/ipfs-cluster/issues/1331)
* Support for multiple architectures added to the Docker container | [ipfs-cluster/ipfs-cluster#1085](https://github.com/ipfs-cluster/ipfs-cluster/issues/1085) | [ipfs-cluster/ipfs-cluster#1196](https://github.com/ipfs-cluster/ipfs-cluster/issues/1196)
* Add `--name` and `--expire` to `ipfs-cluster-ctl pin update` | [ipfs-cluster/ipfs-cluster#1184](https://github.com/ipfs-cluster/ipfs-cluster/issues/1184) | [ipfs-cluster/ipfs-cluster#1195](https://github.com/ipfs-cluster/ipfs-cluster/issues/1195)
* Failover client integrated in `ipfs-cluster-ctl` | [ipfs-cluster/ipfs-cluster#1222](https://github.com/ipfs-cluster/ipfs-cluster/issues/1222) | [ipfs-cluster/ipfs-cluster#1250](https://github.com/ipfs-cluster/ipfs-cluster/issues/1250)
*`ipfs-cluster-ctl health alerts` lists the last expired metrics seen by the peer | [ipfs-cluster/ipfs-cluster#165](https://github.com/ipfs-cluster/ipfs-cluster/issues/165) | [ipfs-cluster/ipfs-cluster#978](https://github.com/ipfs-cluster/ipfs-cluster/issues/978)
* Show expiration information in `ipfs-cluster-ctl pin ls` | [ipfs-cluster/ipfs-cluster#998](https://github.com/ipfs-cluster/ipfs-cluster/issues/998) | [ipfs-cluster/ipfs-cluster#1024](https://github.com/ipfs-cluster/ipfs-cluster/issues/1024) | [ipfs-cluster/ipfs-cluster#1066](https://github.com/ipfs-cluster/ipfs-cluster/issues/1066)
* Show pin names in `ipfs-cluster-ctl status` (and API endpoint) | [ipfs-cluster/ipfs-cluster#1129](https://github.com/ipfs-cluster/ipfs-cluster/issues/1129)
* Fix adding when using CidV1 | [ipfs-cluster/ipfs-cluster#1016](https://github.com/ipfs-cluster/ipfs-cluster/issues/1016) | [ipfs-cluster/ipfs-cluster#1006](https://github.com/ipfs-cluster/ipfs-cluster/issues/1006)
* Fix too many requests error on `ipfs-cluster-follow <cluster> list` | [ipfs-cluster/ipfs-cluster#1013](https://github.com/ipfs-cluster/ipfs-cluster/issues/1013) | [ipfs-cluster/ipfs-cluster#1129](https://github.com/ipfs-cluster/ipfs-cluster/issues/1129)
* Fix repinning not working reliably on collaborative clusters with replication factors set | [ipfs-cluster/ipfs-cluster#1064](https://github.com/ipfs-cluster/ipfs-cluster/issues/1064) | [ipfs-cluster/ipfs-cluster#1127](https://github.com/ipfs-cluster/ipfs-cluster/issues/1127)
* follow: the `info` command panics when ipfs is offline | [ipfs-cluster/ipfs-cluster#991](https://github.com/ipfs-cluster/ipfs-cluster/issues/991) | [ipfs-cluster/ipfs-cluster#993](https://github.com/ipfs-cluster/ipfs-cluster/issues/993)
* follow: the gateway url is not set on Run&Init command | [ipfs-cluster/ipfs-cluster#992](https://github.com/ipfs-cluster/ipfs-cluster/issues/992) | [ipfs-cluster/ipfs-cluster#993](https://github.com/ipfs-cluster/ipfs-cluster/issues/993)
* follow: disallow trusted peers for RepoGCLocal operation | [ipfs-cluster/ipfs-cluster#993](https://github.com/ipfs-cluster/ipfs-cluster/issues/993)
* cluster: `--local` flag for add: adds only to the local peer instead of multiple destinations | [ipfs-cluster/ipfs-cluster#848](https://github.com/ipfs-cluster/ipfs-cluster/issues/848) | [ipfs-cluster/ipfs-cluster#907](https://github.com/ipfs-cluster/ipfs-cluster/issues/907)
* api: `health/metrics` returns list of available metrics | [ipfs-cluster/ipfs-cluster#374](https://github.com/ipfs-cluster/ipfs-cluster/issues/374) | [ipfs-cluster/ipfs-cluster#924](https://github.com/ipfs-cluster/ipfs-cluster/issues/924)
* cluster: advertise external addresses as soon as known | [ipfs-cluster/ipfs-cluster#949](https://github.com/ipfs-cluster/ipfs-cluster/issues/949) | [ipfs-cluster/ipfs-cluster#950](https://github.com/ipfs-cluster/ipfs-cluster/issues/950)
* ipfs-cluster-follow: a new CLI tool to run follower cluster peers | [ipfs-cluster/ipfs-cluster#976](https://github.com/ipfs-cluster/ipfs-cluster/issues/976)
* restapi/client: Fix out of bounds error on load balanced client | [ipfs-cluster/ipfs-cluster#951](https://github.com/ipfs-cluster/ipfs-cluster/issues/951)
* service: disable libp2p restapi on CRDT clusters | [ipfs-cluster/ipfs-cluster#968](https://github.com/ipfs-cluster/ipfs-cluster/issues/968)
* observations: Fix pprof index links | [ipfs-cluster/ipfs-cluster#965](https://github.com/ipfs-cluster/ipfs-cluster/issues/965)
* A new `peer_addresses` key allows specifying additional peer addresses in the configuration (similar to the `peerstore` file). These are treated as libp2p bootstrap addreses (do not mix with Raft bootstrap process). This setting is mostly useful for CRDT collaborative clusters, as template configurations can be distributed including bootstrap peers (usually the same as trusted peers). The values are the full multiaddress of these peers: `/ip4/x.x.x.x/tcp/1234/p2p/Qmxxx...`.
*`listen_multiaddress` can now be set to be an array providing multiple listen multiaddresses, the new defaults being `/tcp/9096` and `/udp/9096/quic`.
*`enable_relay_hop` (true by default), lets the cluster peer act as a relay for other cluster peers behind NATs. This is only for the Cluster network. As a reminder, while this setting is problematic on IPFS (due to the amount of traffic the HOP peers start relaying), the cluster-peers networks are smaller and do not move huge amounts of content around.
* A new `repogc_timeout` key specifies the timeout for garbage collection operations on IPFS. It is set to 24h by default.
##### REST API
The `pin/add` and `add` endpoints support two new query parameters to indicate pin expirations: `expire-at` (with an expected value in RFC3339 format) and `expire-in` (with an expected value in Go's time format, i.e. `12h`). `expire-at` has preference.
A new `/ipfs/gc` endpoint has been added to trigger GC in the IPFS daemons attached to Cluster peers. It supports the `local` parameter to limit the operation to the local peer.
##### Go APIs
There are few changes to Go APIs. The `RepoGC` and `RepoGCLocal` methods have been added, the `mappintracker` module has been removed and the `stateless` module has changed the signature of the constructor.
##### Other
The IPFS Proxy now intercepts the `/repo/gc` endpoint and triggers a cluster-wide GC operation.
The `ipfs-cluster-follow` application is an easy to use way to run one or several cluster peers in follower mode using remote configuration templates. It is fully independent from `ipfs-cluster-service` and `ipfs-cluster-ctl` and acts as both a peer (`run` subcommand) and a client (`list` subcommand). The purpose is to facilitate IPFS Cluster usage without having to deal with the configuration and flags etc.
That said, the configuration layout and folder is the same for both `ipfs-cluster-service` and `ipfs-cluster-follow` and they can be run one in place of the other. In the same way, remote-source configurations usually used for `ipfs-cluster-follow` can be replaced with local ones usually used by `ipfs-cluster-service`.
The removal of the `map pintracker` has resulted in a simplification of some operations. `StateSync` (regularly run every `state_sync_interval`) does not trigger repinnings now, but only checks for pin expirations. `RecoverAllLocal` (regularly run every `pin_recover_interval`) will now trigger repinnings when necessary (i.e. when things that were expected to be on IPFS are not). On very large pinsets, this operation can trigger a memory spike as the full recursive pinset from IPFS is requested and loaded on memory (before this happened on `StateSync`).
* configs: separate identity and configuration | [ipfs-cluster/ipfs-cluster#760](https://github.com/ipfs-cluster/ipfs-cluster/issues/760) | [ipfs-cluster/ipfs-cluster#766](https://github.com/ipfs-cluster/ipfs-cluster/issues/766) | [ipfs-cluster/ipfs-cluster#780](https://github.com/ipfs-cluster/ipfs-cluster/issues/780)
* configs: support running with a remote `service.json` (http) | [ipfs-cluster/ipfs-cluster#868](https://github.com/ipfs-cluster/ipfs-cluster/issues/868)
* configs: support a `follower_mode` option | [ipfs-cluster/ipfs-cluster#803](https://github.com/ipfs-cluster/ipfs-cluster/issues/803) | [ipfs-cluster/ipfs-cluster#864](https://github.com/ipfs-cluster/ipfs-cluster/issues/864)
* service/configs: do not load API components if no config present | [ipfs-cluster/ipfs-cluster#452](https://github.com/ipfs-cluster/ipfs-cluster/issues/452) | [ipfs-cluster/ipfs-cluster#836](https://github.com/ipfs-cluster/ipfs-cluster/issues/836)
* service: add `ipfs-cluster-service init --peers` flag to initialize with given peers | [ipfs-cluster/ipfs-cluster#835](https://github.com/ipfs-cluster/ipfs-cluster/issues/835) | [ipfs-cluster/ipfs-cluster#839](https://github.com/ipfs-cluster/ipfs-cluster/issues/839) | [ipfs-cluster/ipfs-cluster#870](https://github.com/ipfs-cluster/ipfs-cluster/issues/870)
* cluster: support new `PinUpdate` option for new pins | [ipfs-cluster/ipfs-cluster#869](https://github.com/ipfs-cluster/ipfs-cluster/issues/869) | [ipfs-cluster/ipfs-cluster#732](https://github.com/ipfs-cluster/ipfs-cluster/issues/732)
* cluster: trigger `Recover` automatically on a configurable interval | [ipfs-cluster/ipfs-cluster#831](https://github.com/ipfs-cluster/ipfs-cluster/issues/831) | [ipfs-cluster/ipfs-cluster#887](https://github.com/ipfs-cluster/ipfs-cluster/issues/887)
* IPFSConnector: pin timeout start counting from last received block | [ipfs-cluster/ipfs-cluster#497](https://github.com/ipfs-cluster/ipfs-cluster/issues/497) | [ipfs-cluster/ipfs-cluster#738](https://github.com/ipfs-cluster/ipfs-cluster/issues/738)
* IPFSConnector: `unpin_disable` removes the ability to unpin anything from ipfs (experimental) | [ipfs-cluster/ipfs-cluster#793](https://github.com/ipfs-cluster/ipfs-cluster/issues/793) | [ipfs-cluster/ipfs-cluster#832](https://github.com/ipfs-cluster/ipfs-cluster/issues/832)
* REST API Client: Load-balancing Go client | [ipfs-cluster/ipfs-cluster#448](https://github.com/ipfs-cluster/ipfs-cluster/issues/448) | [ipfs-cluster/ipfs-cluster#737](https://github.com/ipfs-cluster/ipfs-cluster/issues/737)
* REST API: Return allocation objects on pin/unpin | [ipfs-cluster/ipfs-cluster#843](https://github.com/ipfs-cluster/ipfs-cluster/issues/843)
* REST API: Support request logging | [ipfs-cluster/ipfs-cluster#574](https://github.com/ipfs-cluster/ipfs-cluster/issues/574) | [ipfs-cluster/ipfs-cluster#894](https://github.com/ipfs-cluster/ipfs-cluster/issues/894)
* Adder: improve error handling. Keep adding while at least one allocation works | [ipfs-cluster/ipfs-cluster#852](https://github.com/ipfs-cluster/ipfs-cluster/issues/852) | [ipfs-cluster/ipfs-cluster#871](https://github.com/ipfs-cluster/ipfs-cluster/issues/871)
* Adder: support user-given allocations for the `Add` operation | [ipfs-cluster/ipfs-cluster#761](https://github.com/ipfs-cluster/ipfs-cluster/issues/761) | [ipfs-cluster/ipfs-cluster#890](https://github.com/ipfs-cluster/ipfs-cluster/issues/890)
* ctl: support adding pin metadata | [ipfs-cluster/ipfs-cluster#670](https://github.com/ipfs-cluster/ipfs-cluster/issues/670) | [ipfs-cluster/ipfs-cluster#891](https://github.com/ipfs-cluster/ipfs-cluster/issues/891)
* IPFSConnector: `pin/ls` does handle base32 and base58 cids properly | [ipfs-cluster/ipfs-cluster#808](https://github.com/ipfs-cluster/ipfs-cluster/issues/808) [ipfs-cluster/ipfs-cluster#809](https://github.com/ipfs-cluster/ipfs-cluster/issues/809)
* configs: some config keys not matching ENV vars names | [ipfs-cluster/ipfs-cluster#837](https://github.com/ipfs-cluster/ipfs-cluster/issues/837) | [ipfs-cluster/ipfs-cluster#778](https://github.com/ipfs-cluster/ipfs-cluster/issues/778)
* cluster: peers forgotten after being down | [ipfs-cluster/ipfs-cluster#648](https://github.com/ipfs-cluster/ipfs-cluster/issues/648) | [ipfs-cluster/ipfs-cluster#860](https://github.com/ipfs-cluster/ipfs-cluster/issues/860)
* cluster: State sync should not keep tracking when queue is full | [ipfs-cluster/ipfs-cluster#377](https://github.com/ipfs-cluster/ipfs-cluster/issues/377) | [ipfs-cluster/ipfs-cluster#901](https://github.com/ipfs-cluster/ipfs-cluster/issues/901)
* cluster: avoid random order on peer lists and listen multiaddresses | [ipfs-cluster/ipfs-cluster#327](https://github.com/ipfs-cluster/ipfs-cluster/issues/327) | [ipfs-cluster/ipfs-cluster#878](https://github.com/ipfs-cluster/ipfs-cluster/issues/878)
* cluster: fix recover and allocation re-assignment to existing pins | [ipfs-cluster/ipfs-cluster#912](https://github.com/ipfs-cluster/ipfs-cluster/issues/912) | [ipfs-cluster/ipfs-cluster#888](https://github.com/ipfs-cluster/ipfs-cluster/issues/888)
* cluster: Use `/p2p/` instead of `/ipfs/` in multiaddresses | [ipfs-cluster/ipfs-cluster#431](https://github.com/ipfs-cluster/ipfs-cluster/issues/431) | [ipfs-cluster/ipfs-cluster#877](https://github.com/ipfs-cluster/ipfs-cluster/issues/877)
* cluster: consolidate parsing of pin options | [ipfs-cluster/ipfs-cluster#913](https://github.com/ipfs-cluster/ipfs-cluster/issues/913)
* docker: use GOPROXY to build containers | [ipfs-cluster/ipfs-cluster#872](https://github.com/ipfs-cluster/ipfs-cluster/issues/872)
* docker: support `IPFS_CLUSTER_CONSENSUS` flag and other improvements | [ipfs-cluster/ipfs-cluster#882](https://github.com/ipfs-cluster/ipfs-cluster/issues/882)
* ctl: increase space for peernames | [ipfs-cluster/ipfs-cluster#887](https://github.com/ipfs-cluster/ipfs-cluster/issues/887)
* Add full support for environment variables in configurations and initialization | [ipfs-cluster/ipfs-cluster#656](https://github.com/ipfs-cluster/ipfs-cluster/issues/656) | [ipfs-cluster/ipfs-cluster#663](https://github.com/ipfs-cluster/ipfs-cluster/issues/663) | [ipfs-cluster/ipfs-cluster#667](https://github.com/ipfs-cluster/ipfs-cluster/issues/667)
* Switch to codecov | [ipfs-cluster/ipfs-cluster#683](https://github.com/ipfs-cluster/ipfs-cluster/issues/683)
* Support user-defined allocations | [ipfs-cluster/ipfs-cluster#646](https://github.com/ipfs-cluster/ipfs-cluster/issues/646) | [ipfs-cluster/ipfs-cluster#647](https://github.com/ipfs-cluster/ipfs-cluster/issues/647)
* Support user-defined metadata in pin objects | [ipfs-cluster/ipfs-cluster#681](https://github.com/ipfs-cluster/ipfs-cluster/issues/681)
* Make normal types serializable and remove `*Serial` types | [ipfs-cluster/ipfs-cluster#654](https://github.com/ipfs-cluster/ipfs-cluster/issues/654) | [ipfs-cluster/ipfs-cluster#688](https://github.com/ipfs-cluster/ipfs-cluster/issues/688) | [ipfs-cluster/ipfs-cluster#700](https://github.com/ipfs-cluster/ipfs-cluster/issues/700)
* Support IPFS paths in the IPFS proxy | [ipfs-cluster/ipfs-cluster#480](https://github.com/ipfs-cluster/ipfs-cluster/issues/480) | [ipfs-cluster/ipfs-cluster#690](https://github.com/ipfs-cluster/ipfs-cluster/issues/690)
* Use go-datastore as backend for the cluster state | [ipfs-cluster/ipfs-cluster#655](https://github.com/ipfs-cluster/ipfs-cluster/issues/655)
* Hide `ipfsproxy.extract_headers_ttl` and `ipfsproxy.extract_headers_path` options by default | [ipfs-cluster/ipfs-cluster#699](https://github.com/ipfs-cluster/ipfs-cluster/issues/699)
This release needs an state upgrade before starting the Cluster daemon. Run `ipfs-cluster-service state upgrade` or run it as `ipfs-cluster-service daemon --upgrade`. We recommend backing up the `~/.ipfs-cluster` folder or exporting the pinset with `ipfs-cluster-service state export`.
##### Configuration changes
Configurations now respects environment variables for all sections. They are
IPFS Cluster version 0.9.0 comes with one big new feature, [OpenCensus](https://opencensus.io) support! This allows for the collection of distributed traces and metrics from the IPFS Cluster application as well as supporting libraries. Currently, we support the use of [Jaeger](https://jaegertracing.io) as the tracing backend and [Prometheus](https://prometheus.io) as the metrics backend. Support for other [OpenCensus backends](https://opencensus.io/exporters/) will be added as requested by the community.
* The snap distribution of IPFS Cluster has been removed | [ipfs-cluster/ipfs-cluster#593](https://github.com/ipfs-cluster/ipfs-cluster/issues/593) | [ipfs-cluster/ipfs-cluster#649](https://github.com/ipfs-cluster/ipfs-cluster/issues/649).
* IPFS Proxy extraction to its own `API` component: `ipfsproxy` | [ipfs-cluster/ipfs-cluster#453](https://github.com/ipfs-cluster/ipfs-cluster/issues/453) | [ipfs-cluster/ipfs-cluster#576](https://github.com/ipfs-cluster/ipfs-cluster/issues/576) | [ipfs-cluster/ipfs-cluster#616](https://github.com/ipfs-cluster/ipfs-cluster/issues/616) | [ipfs-cluster/ipfs-cluster#617](https://github.com/ipfs-cluster/ipfs-cluster/issues/617)
* Add full CORS handling to `restapi` | [ipfs-cluster/ipfs-cluster#639](https://github.com/ipfs-cluster/ipfs-cluster/issues/639) | [ipfs-cluster/ipfs-cluster#640](https://github.com/ipfs-cluster/ipfs-cluster/issues/640)
*`restapi` configuration section entries can be overridden from environment variables | [ipfs-cluster/ipfs-cluster#609](https://github.com/ipfs-cluster/ipfs-cluster/issues/609)
* Update to `go-ipfs-files` 2.0 | [ipfs-cluster/ipfs-cluster#613](https://github.com/ipfs-cluster/ipfs-cluster/issues/613)
* Tests for the `/monitor/metrics` endpoint | [ipfs-cluster/ipfs-cluster#587](https://github.com/ipfs-cluster/ipfs-cluster/issues/587) | [ipfs-cluster/ipfs-cluster#622](https://github.com/ipfs-cluster/ipfs-cluster/issues/622)
* Support `stream-channels=fase` query parameter in `/add` | [ipfs-cluster/ipfs-cluster#632](https://github.com/ipfs-cluster/ipfs-cluster/issues/632) | [ipfs-cluster/ipfs-cluster#633](https://github.com/ipfs-cluster/ipfs-cluster/issues/633)
* Support server side `/pins` filtering | [ipfs-cluster/ipfs-cluster#445](https://github.com/ipfs-cluster/ipfs-cluster/issues/445) | [ipfs-cluster/ipfs-cluster#478](https://github.com/ipfs-cluster/ipfs-cluster/issues/478) | [ipfs-cluster/ipfs-cluster#627](https://github.com/ipfs-cluster/ipfs-cluster/issues/627)
* Fix `ipfs-cluster-ctl` usage with HTTPs and `/dns*` hostnames | [ipfs-cluster/ipfs-cluster#626](https://github.com/ipfs-cluster/ipfs-cluster/issues/626)
* Minor fixes in sharness | [ipfs-cluster/ipfs-cluster#641](https://github.com/ipfs-cluster/ipfs-cluster/issues/641) | [ipfs-cluster/ipfs-cluster#643](https://github.com/ipfs-cluster/ipfs-cluster/issues/643)
* Fix error handling when parsing the configuration | [ipfs-cluster/ipfs-cluster#642](https://github.com/ipfs-cluster/ipfs-cluster/issues/642)
Note that the REST API response format for the `/add` endpoint has changed. Thus all clients need to be upgraded to deal with the new format. The `rest/api/client` has been accordingly updated.
* Clean (rotate) the state when running `init` | [ipfs-cluster/ipfs-cluster#532](https://github.com/ipfs-cluster/ipfs-cluster/issues/532) | [ipfs-cluster/ipfs-cluster#553](https://github.com/ipfs-cluster/ipfs-cluster/issues/553)
* Configurable REST API headers and CORS defaults | [ipfs-cluster/ipfs-cluster#578](https://github.com/ipfs-cluster/ipfs-cluster/issues/578)
* Upgrade libp2p and other deps | [ipfs-cluster/ipfs-cluster#580](https://github.com/ipfs-cluster/ipfs-cluster/issues/580) | [ipfs-cluster/ipfs-cluster#590](https://github.com/ipfs-cluster/ipfs-cluster/issues/590) | [ipfs-cluster/ipfs-cluster#592](https://github.com/ipfs-cluster/ipfs-cluster/issues/592) | [ipfs-cluster/ipfs-cluster#598](https://github.com/ipfs-cluster/ipfs-cluster/issues/598) | [ipfs-cluster/ipfs-cluster#599](https://github.com/ipfs-cluster/ipfs-cluster/issues/599)
* Use `gossipsub` to broadcast metrics | [ipfs-cluster/ipfs-cluster#573](https://github.com/ipfs-cluster/ipfs-cluster/issues/573)
* Download gx and gx-go from IPFS preferentially | [ipfs-cluster/ipfs-cluster#577](https://github.com/ipfs-cluster/ipfs-cluster/issues/577) | [ipfs-cluster/ipfs-cluster#581](https://github.com/ipfs-cluster/ipfs-cluster/issues/581)
* Expose peer metrics in the API + ctl commands | [ipfs-cluster/ipfs-cluster#449](https://github.com/ipfs-cluster/ipfs-cluster/issues/449) | [ipfs-cluster/ipfs-cluster#572](https://github.com/ipfs-cluster/ipfs-cluster/issues/572) | [ipfs-cluster/ipfs-cluster#589](https://github.com/ipfs-cluster/ipfs-cluster/issues/589) | [ipfs-cluster/ipfs-cluster#587](https://github.com/ipfs-cluster/ipfs-cluster/issues/587)
* Add a `docker-compose.yml` template, which creates a two peer cluster | [ipfs-cluster/ipfs-cluster#585](https://github.com/ipfs-cluster/ipfs-cluster/issues/585) | [ipfs-cluster/ipfs-cluster#588](https://github.com/ipfs-cluster/ipfs-cluster/issues/588)
* Support overwriting configuration values in the `cluster` section with environmental values | [ipfs-cluster/ipfs-cluster#575](https://github.com/ipfs-cluster/ipfs-cluster/issues/575) | [ipfs-cluster/ipfs-cluster#596](https://github.com/ipfs-cluster/ipfs-cluster/issues/596)
* Set snaps to `classic` confinement mode and revert it since approval never arrived | [ipfs-cluster/ipfs-cluster#579](https://github.com/ipfs-cluster/ipfs-cluster/issues/579) | [ipfs-cluster/ipfs-cluster#594](https://github.com/ipfs-cluster/ipfs-cluster/issues/594)
* Use Go's reverse proxy library in the proxy endpoint | [ipfs-cluster/ipfs-cluster#570](https://github.com/ipfs-cluster/ipfs-cluster/issues/570) | [ipfs-cluster/ipfs-cluster#605](https://github.com/ipfs-cluster/ipfs-cluster/issues/605)
* Do not error remote pins | [ipfs-cluster/ipfs-cluster#600](https://github.com/ipfs-cluster/ipfs-cluster/issues/600) | [ipfs-cluster/ipfs-cluster#603](https://github.com/ipfs-cluster/ipfs-cluster/issues/603)
* Clean up testing folders in `rest` and `rest/client` after tests | [ipfs-cluster/ipfs-cluster#607](https://github.com/ipfs-cluster/ipfs-cluster/issues/607)
The configurations from previous versions are compatible, but a new `headers` key has been added to the `restapi` section. By default it gets CORS headers which will allow read-only interaction from any origin.
Additionally, all fields from the main `cluster` configuration section can now be overwrriten with environment variables. i.e. `CLUSTER_SECRET`, or `CLUSTER_DISABLEREPINNING`.
##### REST API
The `/add` endpoint stream now returns different objects, in line with the rest of the API types.
Before:
```
type AddedOutput struct {
Error
Name string
Hash string `json:",omitempty"`
Bytes int64 `json:",omitempty"`
Size string `json:",omitempty"`
}
```
Now:
```
type AddedOutput struct {
Name string `json:"name"`
Cid string `json:"cid,omitempty"`
Bytes uint64 `json:"bytes,omitempty"`
Size uint64 `json:"size,omitempty"`
}
```
The `/add` endpoint no longer reports errors as part of an AddedOutput object, but instead it uses trailer headers (same as `go-ipfs`). They are handled in the `client`.
##### Go APIs
The `AddedOutput` object has changed, thus the `api/rest/client` from older versions will not work with this one.
IPFS version 0.6.0 is a new minor release of IPFS Cluster.
We have increased the minor release number to signal changes to the Go APIs after upgrading to the new `cid` package, but, other than that, this release does not include any major changes.
It brings a number of small fixes and features of which we can highlight two useful ones:
* the first is the support for multiple cluster daemon versions in the same cluster, as long as they share the same major/minor release. That means, all releases in the `0.6` series (`0.6.0`, `0.6.1` and so on...) will be able to speak among each others, allowing partial cluster upgrades.
* the second is the inclusion of a `PeerName` key in the status (`PinInfo`) objects. `ipfs-cluster-status` will now show peer names instead of peer IDs, making it easy to identify the status for each peer.
Many thanks to all the contributors to this release: @lanzafame, @meiqimichelle, @kishansagathiya, @cannium, @jglukasik and @mike-ngu.
* Build with go-1.11 | [ipfs-cluster/ipfs-cluster#558](https://github.com/ipfs-cluster/ipfs-cluster/issues/558)
* Peer names in `PinInfo` | [ipfs-cluster/ipfs-cluster#446](https://github.com/ipfs-cluster/ipfs-cluster/issues/446) | [ipfs-cluster/ipfs-cluster#531](https://github.com/ipfs-cluster/ipfs-cluster/issues/531)
* Wrap API client in an interface | [ipfs-cluster/ipfs-cluster#447](https://github.com/ipfs-cluster/ipfs-cluster/issues/447) | [ipfs-cluster/ipfs-cluster#523](https://github.com/ipfs-cluster/ipfs-cluster/issues/523) | [ipfs-cluster/ipfs-cluster#564](https://github.com/ipfs-cluster/ipfs-cluster/issues/564)
* Docs: Typos in Readme and some improvements | [ipfs-cluster/ipfs-cluster#547](https://github.com/ipfs-cluster/ipfs-cluster/issues/547) | [ipfs-cluster/ipfs-cluster#567](https://github.com/ipfs-cluster/ipfs-cluster/issues/567)
* Fix tests in `stateless` PinTracker | [ipfs-cluster/ipfs-cluster#552](https://github.com/ipfs-cluster/ipfs-cluster/issues/552) | [ipfs-cluster/ipfs-cluster#563](https://github.com/ipfs-cluster/ipfs-cluster/issues/563)
This functionality is provided by `ipfs-cluster-ctl add` and by the API endpoint `/add`. The upload format (multipart) is similar to the IPFS `/add` endpoint, as well as the options (chunker, layout...). Cluster `add` generates the same DAG as `ipfs add` would, but it sends the added blocks directly to their allocations, pinning them on completion. The pin happens very quickly, as content is already locally available in the allocated peers.
The release also includes most of the needed code for the [Sharding feature](https://ipfscluster.io/developer/rfcs/dag-sharding-rfc/), but it is not yet usable/enabled, pending features from go-ipfs.
The 0.5.0 release additionally includes a new experimental PinTracker implementation: the `stateless` pin tracker. The stateless pin tracker relies on the IPFS pinset and the cluster state to keep track of pins, rather than keeping an in-memory copy of the cluster pinset, thus reducing the memory usage when having huge pinsets. It can be enabled with `ipfs-cluster-service daemon --pintracker stateless`.
The last major feature is the use of a DHT as routing layer for cluster peers. This means that peers should be able to discover each others as long as they are connected to one cluster peer. This simplifies the setup requirements for starting a cluster and helps avoiding situations which make the cluster unhealthy.
This release requires a state upgrade migration. It can be performed with `ipfs-cluster-service state upgrade` or simply launching the daemon with `ipfs-cluster-service daemon --upgrade`.
* Libp2p upgrades (up to v6) | [ipfs-cluster/ipfs-cluster#456](https://github.com/ipfs-cluster/ipfs-cluster/issues/456) | [ipfs-cluster/ipfs-cluster#482](https://github.com/ipfs-cluster/ipfs-cluster/issues/482)
* Support `/dns` multiaddresses for `node_multiaddress` | [ipfs-cluster/ipfs-cluster#462](https://github.com/ipfs-cluster/ipfs-cluster/issues/462) | [ipfs-cluster/ipfs-cluster#463](https://github.com/ipfs-cluster/ipfs-cluster/issues/463)
* Increase `state_sync_interval` to 10 minutes | [ipfs-cluster/ipfs-cluster#468](https://github.com/ipfs-cluster/ipfs-cluster/issues/468) | [ipfs-cluster/ipfs-cluster#469](https://github.com/ipfs-cluster/ipfs-cluster/issues/469)
**IMPORTANT**: `0s` is the new default for the `read_timeout` and `write_timeout` values in the `restapi` configuration section, as well as `proxy_read_timeout` and `proxy_write_timeout` options in the `ipfshttp` section. Adding files to cluster (via the REST api or the proxy) is likely to timeout otherwise.
The `peerstore` file (in the configuration folder), no longer requires listing the multiaddresses for all cluster peers when initializing the cluster with a fixed peerset. It only requires the multiaddresses for one other cluster peer. The rest will be inferred using the DHT. The peerstore file is updated only on clean shutdown, and will store all known multiaddresses, even if not pertaining to cluster peers.
The new `stateless` PinTracker implementation uses a new configuration subsection in the `pin_tracker` key. This is only generated with `ipfs-cluster-service init`. When not present, a default configuration will be used (and a warning printed).
The `state_sync_interval` default has been increased to 10 minutes, as frequent syncing is not needed with the improvements in the PinTracker. Users are welcome to update this setting.
##### REST API
The `/add` endpoint has been added. The `replication_factor_min` and `replication_factor_max` options (in `POST allocations/<cid>`) have been deprecated and subsititued for `replication-min` and `replication-max`, although backwards comaptibility is kept.
Keep Alive has been disabled for the HTTP servers, as a bug in Go's HTTP client implementation may result adding corrupted content (and getting corrupted DAGs). However, while the libp2p API endpoint also suffers this, it will only close libp2p streams. Thus the performance impact on the libp2p-http endpoint should be minimal.
##### Go APIs
The `Config.PeerAddr` key in the `rest/client` module is deprecated. `APIAddr` should be used for both HTTP and LibP2P API endpoints. The type of address is automatically detected.
The IPFSConnector `Pin` call now receives an integer instead of a `Recursive` flag. It indicates the maximum depth to which something should be pinned. The only supported value is `-1` (meaning recursive). `BlockGet` and `BlockPut` calls have been added to the IPFSConnector component.
##### Other
As noted above, upgrade to `state` format version 5 is needed before starting the cluster service.
The IPFS Cluster version 0.4.0 includes breaking changes and a considerable number of new features causing them. The documentation (particularly that affecting the configuration and startup of peers) has been updated accordingly in https://ipfscluster.io . Be sure to also read it if you are upgrading.
* Refactorings around the `consensus` component, removing dependencies to the main component and allowing separate initialization: this has prompted to re-approach how we handle the peerset, the peer addresses and the peer's startup when using bootstrap. We have gained finer control of Raft, which has allowed us to provide a clearer configuration and a better start up procedure, specially when bootstrapping. The configuration file no longer mutates while cluster is running.
* Improvements to the `pintracker`: our pin tracker is now able to cancel ongoing pins when receiving an unpin request for the same CID, and vice-versa. It will also optimize multiple pin requests (by only queuing and triggering them once) and can now report
whether an item is pinning (a request to ipfs is ongoing) vs. pin-queued (waiting for a worker to perform the request to ipfs).
* Broadcasting of monitoring metrics using PubSub: we have added a new `monitor` implementation that uses PubSub (rather than RPC broadcasting). With the upcoming improvements to PubSub this means that we can do efficient broadcasting of metrics while at the same time not requiring peers to have RPC permissions, which is preparing the ground for collaborative clusters.
* Documentation moved to website and fixes | [ipfs-cluster/ipfs-cluster#390](https://github.com/ipfs-cluster/ipfs-cluster/issues/390) | [ipfs-cluster/ipfs-cluster#391](https://github.com/ipfs-cluster/ipfs-cluster/issues/391) | [ipfs-cluster/ipfs-cluster#393](https://github.com/ipfs-cluster/ipfs-cluster/issues/393) | [ipfs-cluster/ipfs-cluster#347](https://github.com/ipfs-cluster/ipfs-cluster/issues/347)
* Run Docker container with `daemon --upgrade` by default | [ipfs-cluster/ipfs-cluster#394](https://github.com/ipfs-cluster/ipfs-cluster/issues/394)
* Remove the `ipfs-cluster-ctl peers add` command (bootstrap should be used to add peers) | [ipfs-cluster/ipfs-cluster#397](https://github.com/ipfs-cluster/ipfs-cluster/issues/397)
* Add tests using HTTPs endpoints | [ipfs-cluster/ipfs-cluster#191](https://github.com/ipfs-cluster/ipfs-cluster/issues/191) | [ipfs-cluster/ipfs-cluster#403](https://github.com/ipfs-cluster/ipfs-cluster/issues/403)
* Set `refs` as default `pinning_method` and `10` as default `concurrent_pins` | [ipfs-cluster/ipfs-cluster#420](https://github.com/ipfs-cluster/ipfs-cluster/issues/420)
* Use latest `gx` and `gx-go`. Be more verbose when installing | [ipfs-cluster/ipfs-cluster#418](https://github.com/ipfs-cluster/ipfs-cluster/issues/418)
* Makefile: Properly retrigger builds on source change | [ipfs-cluster/ipfs-cluster#426](https://github.com/ipfs-cluster/ipfs-cluster/issues/426)
* Improvements to StateSync() | [ipfs-cluster/ipfs-cluster#429](https://github.com/ipfs-cluster/ipfs-cluster/issues/429)
* Rename `ipfs-cluster-data` folder to `raft` | [ipfs-cluster/ipfs-cluster#430](https://github.com/ipfs-cluster/ipfs-cluster/issues/430)
* Officially support go 1.10 | [ipfs-cluster/ipfs-cluster#439](https://github.com/ipfs-cluster/ipfs-cluster/issues/439)
* Update to libp2p 5.0.17 | [ipfs-cluster/ipfs-cluster#440](https://github.com/ipfs-cluster/ipfs-cluster/issues/440)
* Don't keep peers /ip*/ addresses if we know DNS addresses for them | [ipfs-cluster/ipfs-cluster#381](https://github.com/ipfs-cluster/ipfs-cluster/issues/381)
* Do not fail when running with `daemon --upgrade` and no state is present | [ipfs-cluster/ipfs-cluster#395](https://github.com/ipfs-cluster/ipfs-cluster/issues/395)
* IPFS Proxy: handle arguments passed as part of the url | [ipfs-cluster/ipfs-cluster#380](https://github.com/ipfs-cluster/ipfs-cluster/issues/380) | [ipfs-cluster/ipfs-cluster#392](https://github.com/ipfs-cluster/ipfs-cluster/issues/392)
* WaitForUpdates() may return before state is fully synced | [ipfs-cluster/ipfs-cluster#378](https://github.com/ipfs-cluster/ipfs-cluster/issues/378)
* Configuration mutates no more and shadowing is no longer necessary | [ipfs-cluster/ipfs-cluster#235](https://github.com/ipfs-cluster/ipfs-cluster/issues/235)
* Fix lock file not being removed on cluster force-kill | [ipfs-cluster/ipfs-cluster#423](https://github.com/ipfs-cluster/ipfs-cluster/issues/423) | [ipfs-cluster/ipfs-cluster#437](https://github.com/ipfs-cluster/ipfs-cluster/issues/437)
* Fix indirect pins not being correctly parsed | [ipfs-cluster/ipfs-cluster#428](https://github.com/ipfs-cluster/ipfs-cluster/issues/428) | [ipfs-cluster/ipfs-cluster#436](https://github.com/ipfs-cluster/ipfs-cluster/issues/436)
* Enable NAT support in libp2p host | [ipfs-cluster/ipfs-cluster#346](https://github.com/ipfs-cluster/ipfs-cluster/issues/346) | [ipfs-cluster/ipfs-cluster#441](https://github.com/ipfs-cluster/ipfs-cluster/issues/441)
* Fix pubsub monitor not working on ARM | [ipfs-cluster/ipfs-cluster#433](https://github.com/ipfs-cluster/ipfs-cluster/issues/433) | [ipfs-cluster/ipfs-cluster#443](https://github.com/ipfs-cluster/ipfs-cluster/issues/443)
This release introduces **breaking changes to the configuration file**. An error will be displayed if `ipfs-cluster-service` is started with an old configuration file. We recommend re-initing the configuration file altogether.
* The `peers` and `bootstrap` keys have been removed from the main section of the configuration
* You might need to provide Peer multiaddresses in a text file named `peerstore`, in your `~/.ipfs-cluster` folder (one per line). This allows your peers how to contact other peers.
* A `disable_repinning` option has been added to the main configuration section. Defaults to `false`.
* A `init_peerset` has been added to the `raft` configuration section. It should be used to define the starting set of peers when a cluster starts for the first time and is not bootstrapping to an existing running peer (otherwise it is ignored). The value is an array of peer IDs.
* A `backups_rotate` option has been added to the `raft` section and specifies how many copies of the Raft state to keep as backups when the state is cleaned up.
* An `ipfs_request_timeout` option has been introduced to the `ipfshttp` configuration section, and controls the timeout of general requests to the ipfs daemon. Defaults to 5 minutes.
* A `pin_timeout` option has been introduced to the `ipfshttp` section, it controls the timeout for Pin requests to ipfs. Defaults to 24 hours.
* An `unpin_timeout` option has been introduced to the `ipfshttp` section. it controls the timeout for Unpin requests to ipfs. Defaults to 3h.
* Both `pinning_timeout` and `unpinning_timeout` options have been removed from the `maptracker` section.
* A `monitor/pubsubmon` section configures the new PubSub monitoring component. The section is identical to the existing `monbasic`, its only option being `check_interval` (defaults to 15 seconds).
The `ipfs-cluster-data` folder has been renamed to `raft`. Upon `ipfs-cluster-service daemon` start, the renaming will happen automatically if it exists. Otherwise it will be created with the new name.
##### REST API
There are no changes to REST APIs in this release.
##### Go APIs
Several component APIs have changed: `Consensus`, `PeerMonitor` and `IPFSConnector` have added new methods or changed methods signatures.
##### Other
Calling `ipfs-cluster-service` without subcommands no longer runs the peer. It is necessary to call `ipfs-cluster-service daemon`. Several daemon-specific flags have been made subcommand flags: `--bootstrap` and `--alloc`.
The `--bootstrap` flag can now take a list of comma-separated multiaddresses. Using `--bootstrap` will automatically run `state clean`.
The `ipfs-cluster-ctl` no longer has a `peers add` subcommand. Peers should not be added this way, but rather bootstrapped to an existing running peer.
This release comes full with new features. The biggest ones are the support for parallel pinning (using `refs -r` rather than `pin add` to pin things in IPFS), and the exposing of the http endpoints through libp2p. This allows users to securely interact with the HTTP API without having to setup SSL certificates.
*`--no-status` for `ipfs-cluster-ctl pin add/rm` allows to speed up adding and removing by not fetching the status one second afterwards. Useful for ingesting pinsets to cluster | [ipfs-cluster/ipfs-cluster#286](https://github.com/ipfs-cluster/ipfs-cluster/issues/286) | [ipfs-cluster/ipfs-cluster#329](https://github.com/ipfs-cluster/ipfs-cluster/issues/329)
*`--wait` flag for `ipfs-cluster-ctl pin add/rm` allows to wait until a CID is fully pinned or unpinned [ipfs-cluster/ipfs-cluster#338](https://github.com/ipfs-cluster/ipfs-cluster/issues/338) | [ipfs-cluster/ipfs-cluster#348](https://github.com/ipfs-cluster/ipfs-cluster/issues/348) | [ipfs-cluster/ipfs-cluster#363](https://github.com/ipfs-cluster/ipfs-cluster/issues/363)
* REST API client now provides an `IPFS()` method which returns a `go-ipfs-api` shell instance pointing to the proxy endpoint | [ipfs-cluster/ipfs-cluster#269](https://github.com/ipfs-cluster/ipfs-cluster/issues/269) | [ipfs-cluster/ipfs-cluster#356](https://github.com/ipfs-cluster/ipfs-cluster/issues/356)
* Added support for priority pins and non-recursive pins (sharding-related) | [ipfs-cluster/ipfs-cluster#341](https://github.com/ipfs-cluster/ipfs-cluster/issues/341) | [ipfs-cluster/ipfs-cluster#342](https://github.com/ipfs-cluster/ipfs-cluster/issues/342)
There are no breaking API changes and all configurations should be backwards compatible. The `api/rest/client` provides a new `IPFS()` method.
We recommend updating the `service.json` configurations to include all the new configuration options:
* The `pin_method` option has been added to the `ipfshttp` section. It supports `refs` and `pin` (default) values. Use `refs` for parallel pinning, but only if you don't run automatic GC on your ipfs nodes.
* The `concurrent_pins` option has been added to the `maptracker` section. Only useful with `refs` option in `pin_method`.
* The `listen_multiaddress` option in the `restapi` section should be renamed to `http_listen_multiaddress`.
This release will require a **state upgrade**. Run `ipfs-cluster-service state upgrade` in all your peers, or start cluster with `ipfs-cluster-service daemon --upgrade`.
*`ipfs-cluster-service daemon --upgrade` allows to automatically run migrations before starting | [ipfs-cluster/ipfs-cluster#300](https://github.com/ipfs-cluster/ipfs-cluster/issues/300) | [ipfs-cluster/ipfs-cluster#307](https://github.com/ipfs-cluster/ipfs-cluster/issues/307)
*`ipfs-cluster-service state version` reports the shared state format version | [ipfs-cluster/ipfs-cluster#298](https://github.com/ipfs-cluster/ipfs-cluster/issues/298) | [ipfs-cluster/ipfs-cluster#307](https://github.com/ipfs-cluster/ipfs-cluster/issues/307)
*`ipfs-cluster-service health graph` generates a .dot graph file of cluster connectivity | [ipfs-cluster/ipfs-cluster#17](https://github.com/ipfs-cluster/ipfs-cluster/issues/17) | [ipfs-cluster/ipfs-cluster#291](https://github.com/ipfs-cluster/ipfs-cluster/issues/291) | [ipfs-cluster/ipfs-cluster#311](https://github.com/ipfs-cluster/ipfs-cluster/issues/311)
* Do not upgrade state if already up to date | [ipfs-cluster/ipfs-cluster#296](https://github.com/ipfs-cluster/ipfs-cluster/issues/296) | [ipfs-cluster/ipfs-cluster#307](https://github.com/ipfs-cluster/ipfs-cluster/issues/307)
* Commands to export and import the internal cluster state: these allow to perform easy and human-readable dumps of the shared cluster state while offline, and eventually restore it in a different peer or cluster.
* The introduction of `replication_factor_min` and `replication_factor_max` parameters for every Pin (along with the deprecation of `replication_factor`). The defaults are specified in the configuration. For more information on the usage and behavior of these new options, check the IPFS cluster guide.
* New `ipfs-cluster-service state export/import/cleanup` commands | [ipfs-cluster/ipfs-cluster#240](https://github.com/ipfs-cluster/ipfs-cluster/issues/240) | [ipfs-cluster/ipfs-cluster#290](https://github.com/ipfs-cluster/ipfs-cluster/issues/290)
* New min/max replication factor control | [ipfs-cluster/ipfs-cluster#277](https://github.com/ipfs-cluster/ipfs-cluster/issues/277) | [ipfs-cluster/ipfs-cluster#292](https://github.com/ipfs-cluster/ipfs-cluster/issues/292)
* Added tests for Dockerfiles | [ipfs-cluster/ipfs-cluster#200](https://github.com/ipfs-cluster/ipfs-cluster/issues/200) | [ipfs-cluster/ipfs-cluster#282](https://github.com/ipfs-cluster/ipfs-cluster/issues/282)
* Enabled Jenkins testing and made it work | [ipfs-cluster/ipfs-cluster#256](https://github.com/ipfs-cluster/ipfs-cluster/issues/256) | [ipfs-cluster/ipfs-cluster#294](https://github.com/ipfs-cluster/ipfs-cluster/issues/294)
* Force cleanup after sharness tests | [ipfs-cluster/ipfs-cluster#181](https://github.com/ipfs-cluster/ipfs-cluster/issues/181) | [ipfs-cluster/ipfs-cluster#288](https://github.com/ipfs-cluster/ipfs-cluster/issues/288)
* Fix state version validation on start | [ipfs-cluster/ipfs-cluster#293](https://github.com/ipfs-cluster/ipfs-cluster/issues/293)
* Wait until last index is applied before attempting snapshot on shutdown | [ipfs-cluster/ipfs-cluster#275](https://github.com/ipfs-cluster/ipfs-cluster/issues/275)
These release is compatible with previous versions of ipfs-cluster on the API level, with the exception of the `ipfs-cluster-service version` command, which returns `x.x.x-shortcommit` rather than `ipfs-cluster-service version 0.3.1`. The former output is still available as `ipfs-cluster-service --version`.
The `replication_factor` option is deprecated, but still supported and will serve as a shortcut to set both `replication_factor_min` and `replication_factor_max` to the same value. This affects the configuration file, the REST API and the `ipfs-cluster-ctl pin add` command.
This release includes changes around the consensus state management, so that upgrades can be performed when the internal format changes. It also comes with several features and changes to support a live deployment and integration with IPFS pin-bot, including a REST API client for Go.
*`ipfs-cluster-service state upgrade` | [ipfs-cluster/ipfs-cluster#194](https://github.com/ipfs-cluster/ipfs-cluster/issues/194)
*`ipfs-cluster-test` Docker image runs with `ipfs:master` | [ipfs-cluster/ipfs-cluster#155](https://github.com/ipfs-cluster/ipfs-cluster/issues/155) | [ipfs-cluster/ipfs-cluster#259](https://github.com/ipfs-cluster/ipfs-cluster/issues/259)
*`ipfs-cluster` Docker image only runs `ipfs-cluster-service` (and not the ipfs daemon anymore) | [ipfs-cluster/ipfs-cluster#197](https://github.com/ipfs-cluster/ipfs-cluster/issues/197) | [ipfs-cluster/ipfs-cluster#155](https://github.com/ipfs-cluster/ipfs-cluster/issues/155) | [ipfs-cluster/ipfs-cluster#259](https://github.com/ipfs-cluster/ipfs-cluster/issues/259)
* Support for DNS multiaddresses for cluster peers | [ipfs-cluster/ipfs-cluster#155](https://github.com/ipfs-cluster/ipfs-cluster/issues/155) | [ipfs-cluster/ipfs-cluster#259](https://github.com/ipfs-cluster/ipfs-cluster/issues/259)
* Add configuration section and options for `pin_tracker` | [ipfs-cluster/ipfs-cluster#155](https://github.com/ipfs-cluster/ipfs-cluster/issues/155) | [ipfs-cluster/ipfs-cluster#259](https://github.com/ipfs-cluster/ipfs-cluster/issues/259)
* Add `local` flag to Status, Sync, Recover endpoints which allows to run this operations only in the peer receiving the request | [ipfs-cluster/ipfs-cluster#155](https://github.com/ipfs-cluster/ipfs-cluster/issues/155) | [ipfs-cluster/ipfs-cluster#259](https://github.com/ipfs-cluster/ipfs-cluster/issues/259)
* New REST API Client module `github.com/ipfs-cluster/ipfs-cluster/api/rest/client` allows to integrate against cluster | [ipfs-cluster/ipfs-cluster#260](https://github.com/ipfs-cluster/ipfs-cluster/issues/260) | [ipfs-cluster/ipfs-cluster#263](https://github.com/ipfs-cluster/ipfs-cluster/issues/263) | [ipfs-cluster/ipfs-cluster#266](https://github.com/ipfs-cluster/ipfs-cluster/issues/266)
* A few rounds addressing code quality issues | [ipfs-cluster/ipfs-cluster#264](https://github.com/ipfs-cluster/ipfs-cluster/issues/264)
This release should stay backwards compatible with the previous one. Nevertheless, some REST API endpoints take the `local` flag, and matching new Go public functions have been added (`RecoverAllLocal`, `SyncAllLocal`...).
* Fix configuration save | [ipfs-cluster/ipfs-cluster#213](https://github.com/ipfs-cluster/ipfs-cluster/issues/213) | [ipfs-cluster/ipfs-cluster#214](https://github.com/ipfs-cluster/ipfs-cluster/issues/214)
* Forward progress updates with IPFS-Proxy | [ipfs-cluster/ipfs-cluster#224](https://github.com/ipfs-cluster/ipfs-cluster/issues/224) | [ipfs-cluster/ipfs-cluster#231](https://github.com/ipfs-cluster/ipfs-cluster/issues/231)
* Delay ipfs connect swarms on boot and safeguard against panic condition | [ipfs-cluster/ipfs-cluster#238](https://github.com/ipfs-cluster/ipfs-cluster/issues/238)
* Multiple minor fixes | [ipfs-cluster/ipfs-cluster#236](https://github.com/ipfs-cluster/ipfs-cluster/issues/236)
* Avoid shutting down consensus in the middle of a commit
* Return an ID containing current peers in PeerAdd
* Do not shut down libp2p host in the middle of peer removal
* Send cluster addresses to the new peer before adding it
* Wait for configuration save on init
* Fix error message when not enough allocations exist for a pin
This releases introduces some changes affecting the configuration file and some breaking changes affecting `go` and the REST APIs:
* The `consensus.raft` section of the configuration has new options but should be backwards compatible.
* The `Consensus` component interface has changed, `LogAddPeer` and `LogRmPeer` have been replaced by `AddPeer` and `RmPeer`. It additionally provides `Clean` and `Peers` methods. The `consensus/raft` implementation has been updated accordingly.
* The `api.ID` (used in REST API among others) object key `ClusterPeers` key is now a list of peer IDs, and not a list of multiaddresses as before. The object includes a new key `ClusterPeersAddresses` which includes the multiaddresses.
* Note that `--bootstrap` and `--leave` flags when calling `ipfs-cluster-service` will be stored permanently in the configuration (see [ipfs-cluster/ipfs-cluster#235](https://github.com/ipfs-cluster/ipfs-cluster/issues/235)).
* Peers and bootstrap entries in the configuration are ignored | [ipfs-cluster/ipfs-cluster#203](https://github.com/ipfs-cluster/ipfs-cluster/issues/203) | [ipfs-cluster/ipfs-cluster#204](https://github.com/ipfs-cluster/ipfs-cluster/issues/204)
* Informers do not work on 32-bit architectures | [ipfs-cluster/ipfs-cluster#202](https://github.com/ipfs-cluster/ipfs-cluster/issues/202) | [ipfs-cluster/ipfs-cluster#205](https://github.com/ipfs-cluster/ipfs-cluster/issues/205)
* Replication factor entry in the configuration is ignored | [ipfs-cluster/ipfs-cluster#208](https://github.com/ipfs-cluster/ipfs-cluster/issues/208) | [ipfs-cluster/ipfs-cluster#209](https://github.com/ipfs-cluster/ipfs-cluster/issues/209)
The fix for 32-bit architectures has required a change in the `IPFSConnector` interface (`FreeSpace()` and `Reposize()` return `uint64` now). The current implementation by the `ipfshttp` module has changed accordingly.
* Basic authentication support added to API component | [ipfs-cluster/ipfs-cluster#121](https://github.com/ipfs-cluster/ipfs-cluster/issues/121) | [ipfs-cluster/ipfs-cluster#147](https://github.com/ipfs-cluster/ipfs-cluster/issues/147) | [ipfs-cluster/ipfs-cluster#179](https://github.com/ipfs-cluster/ipfs-cluster/issues/179)
* Copy peers to bootstrap when leaving a cluster | [ipfs-cluster/ipfs-cluster#170](https://github.com/ipfs-cluster/ipfs-cluster/issues/170) | [ipfs-cluster/ipfs-cluster#112](https://github.com/ipfs-cluster/ipfs-cluster/issues/112)
* New configuration format | [ipfs-cluster/ipfs-cluster#162](https://github.com/ipfs-cluster/ipfs-cluster/issues/162) | [ipfs-cluster/ipfs-cluster#177](https://github.com/ipfs-cluster/ipfs-cluster/issues/177)
* Freespace disk metric implementation. It's now the default. | [ipfs-cluster/ipfs-cluster#142](https://github.com/ipfs-cluster/ipfs-cluster/issues/142) | [ipfs-cluster/ipfs-cluster#99](https://github.com/ipfs-cluster/ipfs-cluster/issues/99)
* IPFS Connector should use only POST | [ipfs-cluster/ipfs-cluster#176](https://github.com/ipfs-cluster/ipfs-cluster/issues/176) | [ipfs-cluster/ipfs-cluster#161](https://github.com/ipfs-cluster/ipfs-cluster/issues/161)
*`ipfs-cluster-ctl` exit status with error responses | [ipfs-cluster/ipfs-cluster#174](https://github.com/ipfs-cluster/ipfs-cluster/issues/174)
* Sharness tests and update testing container | [ipfs-cluster/ipfs-cluster#171](https://github.com/ipfs-cluster/ipfs-cluster/issues/171)
*`ipfs-cluster-service`: Do not run service with unknown subcommands | [ipfs-cluster/ipfs-cluster#186](https://github.com/ipfs-cluster/ipfs-cluster/issues/186)
This release introduces some breaking changes affecting configuration files and `go` integrations:
* Config: The old configuration format is no longer valid and cluster will fail to start from it. Configuration file needs to be re-initialized with `ipfs-cluster-service init`.
* Go: The `restapi` component has been renamed to `rest` and some of its public methods have been renamed.
* Go: Initializers (`New<Component>(...)`) for most components have changed to accept a `Config` object. Some initializers have been removed.
