cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,935 Commits 3 Branches 70 Tags 15 MiB
master
Commit Graph

116 Commits

Author SHA1 Message Date
Hector Sanjuan
19d1ab3d3f
Dependency upgrades (#2026) 
* Update dependencies

* Update dependencies

* Update tests. Remove Pretty()

* Fix problems with new behaviour of Paths

* Typo
 2024-01-30 00:58:28 +01:00
Hector Sanjuan
fa966102b6 Enable pebble in all architectures 
Upstream was fixed to build on all platforms.
 2023-12-07 11:06:27 +01:00
Max
ecb70f1573 Send readiness notification to systemd 2023-04-14 15:40:50 +02:00
Hector Sanjuan
b6e8a6efd2 Pebble: make default 
* Make pebble default (badger3 as fallback for unsupported platforms)
* Better configurations defaults for Pebble
 2023-03-06 18:41:51 +01:00
Hector Sanjuan
7c32a117f1 service: do not show pebble as an option on unsupported architectures 2023-03-03 18:19:02 +01:00
Hector Sanjuan
7cc91cae4f
Dependency upgrades (#1840) 
* Bump github.com/libp2p/go-libp2p-kad-dht from 0.18.0 to 0.20.0 (#1821)

Bumps [github.com/libp2p/go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) from 0.18.0 to 0.20.0.
- [Release notes](https://github.com/libp2p/go-libp2p-kad-dht/releases)
- [Commits](https://github.com/libp2p/go-libp2p-kad-dht/compare/v0.18.0...v0.20.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-kad-dht
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/dgraph-io/badger/v3 from 3.2103.4 to 3.2103.5 (#1822)

Bumps [github.com/dgraph-io/badger/v3](https://github.com/dgraph-io/badger) from 3.2103.4 to 3.2103.5.
- [Release notes](https://github.com/dgraph-io/badger/releases)
- [Changelog](https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dgraph-io/badger/compare/v3.2103.4...v3.2103.5)

---
updated-dependencies:
- dependency-name: github.com/dgraph-io/badger/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump go.uber.org/multierr from 1.8.0 to 1.9.0 (#1823)

Bumps [go.uber.org/multierr](https://github.com/uber-go/multierr) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/uber-go/multierr/releases)
- [Changelog](https://github.com/uber-go/multierr/blob/master/CHANGELOG.md)
- [Commits](https://github.com/uber-go/multierr/compare/v1.8.0...v1.9.0)

---
updated-dependencies:
- dependency-name: go.uber.org/multierr
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/rs/cors from 1.8.2 to 1.8.3 (#1826)

Bumps [github.com/rs/cors](https://github.com/rs/cors) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/rs/cors/releases)
- [Commits](https://github.com/rs/cors/compare/v1.8.2...v1.8.3)

---
updated-dependencies:
- dependency-name: github.com/rs/cors
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/ipfs/go-merkledag from 0.8.1 to 0.9.0 (#1830)

Bumps [github.com/ipfs/go-merkledag](https://github.com/ipfs/go-merkledag) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/ipfs/go-merkledag/releases)
- [Commits](https://github.com/ipfs/go-merkledag/compare/v0.8.1...v0.9.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-merkledag
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 (#1831)

Bumps [github.com/dustin/go-humanize](https://github.com/dustin/go-humanize) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/dustin/go-humanize/releases)
- [Commits](https://github.com/dustin/go-humanize/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/dustin/go-humanize
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/urfave/cli from 1.22.10 to 1.22.12 (#1834)

Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.10 to 1.22.12.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.10...v1.22.12)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/ipfs/go-unixfs from 0.4.1 to 0.4.2 (#1829)

Bumps [github.com/ipfs/go-unixfs](https://github.com/ipfs/go-unixfs) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/ipfs/go-unixfs/releases)
- [Commits](https://github.com/ipfs/go-unixfs/compare/v0.4.1...v0.4.2)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-unixfs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump go.opencensus.io from 0.23.0 to 0.24.0 (#1794)

Dependabot couldn't find the original pull request head commit, 9dedea6eeef2c8b5f7358cc29b5b088ac6cf7aba.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/ugorji/go/codec from 1.2.7 to 1.2.8 (#1827)

Bumps [github.com/ugorji/go/codec](https://github.com/ugorji/go) from 1.2.7 to 1.2.8.
- [Release notes](https://github.com/ugorji/go/releases)
- [Commits](https://github.com/ugorji/go/compare/v1.2.7...v1.2.8)

---
updated-dependencies:
- dependency-name: github.com/ugorji/go/codec
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/libp2p/go-libp2p-pubsub from 0.8.2 to 0.8.3 (#1838)

Bumps [github.com/libp2p/go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/libp2p/go-libp2p-pubsub/releases)
- [Commits](https://github.com/libp2p/go-libp2p-pubsub/compare/v0.8.2...v0.8.3)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-pubsub
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* update go-libp2p

* Bump github.com/ipfs/go-block-format from 0.0.3 to 0.1.1 (#1837)

Bumps [github.com/ipfs/go-block-format](https://github.com/ipfs/go-block-format) from 0.0.3 to 0.1.1.
- [Release notes](https://github.com/ipfs/go-block-format/releases)
- [Commits](https://github.com/ipfs/go-block-format/compare/v0.0.3...v0.1.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-block-format
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update go-mfs (#1818)

* Fix docker build failure (git safe directory) (#1836)

Git refuses to run `git rev-parse HEAD` now, in docker.

* Remove go-block-format and use go-libipfs (deprecation)

* Update ipfs-lite

* Bump golang.org/x/crypto from 0.3.0 to 0.5.0 (#1839)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.3.0 to 0.5.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/compare/v0.3.0...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-27 14:34:53 +01:00
Hector Sanjuan
f496e522cd Add Pebble support as datastore backend 
Pebble is a modern datastore based on RocksDB. This commit adds support to use
it as datastore backend for IPFS-Cluster.
 2022-12-02 11:29:33 +01:00
Hector Sanjuan
970bdc8cb0 Add Badger v3 support 
This commit adds badger v3 support as datastore backend for IPFS Cluster.

Badgerv3 has a number of improvements over v1, particularly around
compaction. It is significantly more modern and should behave better in
general (some testing is pending).
 2022-12-02 11:29:33 +01:00
Hector Sanjuan
c10c37f25b Adjust CRDT consensus timeout to be infinite (1 year) 
This allows for crdt migrations to finish correctly.
 2022-11-25 12:56:01 +01:00
Hector Sanjuan
ffef2cea0a service: add crdt info, mark-dirty, mark-clean commands 
Provide access to some crdt-internal operations
 2022-09-26 19:39:04 +02:00
Hector Sanjuan
29d6f69819 Add "service state crdt dot" command 
This subcommand allows to export the peer's CRDT DAG as dot file.

It also opens the door to have more crdt-specific subcommands, i.e. CAR export
etc.
 2022-09-13 16:36:10 +02:00
Hector Sanjuan
5452b59a2e
Dependency upgrades (#1755) 
* Update go-libp2p to v0.22.0

* Testing with go1.19

* build(deps): bump github.com/multiformats/go-multicodec

Bumps [github.com/multiformats/go-multicodec](https://github.com/multiformats/go-multicodec) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/multiformats/go-multicodec/releases)
- [Commits](https://github.com/multiformats/go-multicodec/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multicodec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/ipld/go-car from 0.4.0 to 0.5.0

Bumps [github.com/ipld/go-car](https://github.com/ipld/go-car) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/ipld/go-car/releases)
- [Commits](https://github.com/ipld/go-car/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/ipld/go-car
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.12.2 to 1.13.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.2...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/hashicorp/go-hclog from 1.2.1 to 1.3.0

Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/hashicorp/go-hclog/releases)
- [Commits](https://github.com/hashicorp/go-hclog/compare/v1.2.1...v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/ipfs/go-ds-crdt from 0.3.6 to 0.3.7

Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.3.6...v0.3.7)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.10.2 to 2.14.1

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.2 to 2.14.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.2...v2.14.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-http from 0.3.0 to 0.4.0

Bumps [github.com/libp2p/go-libp2p-http](https://github.com/libp2p/go-libp2p-http) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/libp2p/go-libp2p-http/releases)
- [Commits](https://github.com/libp2p/go-libp2p-http/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-http
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-gorpc from 0.4.0 to 0.5.0

Bumps [github.com/libp2p/go-libp2p-gorpc](https://github.com/libp2p/go-libp2p-gorpc) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/libp2p/go-libp2p-gorpc/releases)
- [Commits](https://github.com/libp2p/go-libp2p-gorpc/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-gorpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump contrib.go.opencensus.io/exporter/prometheus

Bumps [contrib.go.opencensus.io/exporter/prometheus](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/releases)
- [Commits](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/compare/v0.4.1...v0.4.2)

---
updated-dependencies:
- dependency-name: contrib.go.opencensus.io/exporter/prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-raft from 0.1.8 to 0.2.0

Bumps [github.com/libp2p/go-libp2p-raft](https://github.com/libp2p/go-libp2p-raft) from 0.1.8 to 0.2.0.
- [Release notes](https://github.com/libp2p/go-libp2p-raft/releases)
- [Commits](https://github.com/libp2p/go-libp2p-raft/compare/v0.1.8...v0.2.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-raft
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/urfave/cli from 1.22.9 to 1.22.10

Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.9 to 1.22.10.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.9...v1.22.10)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix checker/linter/staticcheck warnings

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-06 16:57:17 +02:00
Hector Sanjuan
d7107791ed Rename links to use the ipfscluster.io domain 2022-06-16 17:43:30 +02:00
Hector Sanjuan
508791b547 Migrate from ipfs/ipfs-cluster to ipfs-cluster/ipfs-cluster 
This performs the necessary renamings.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
4daece2b98 Feat: add a new "pinqueue" informer component 
This new component broadcasts metrics about the current size of the pinqueue,
which can in turn be used to inform allocations.

It has a weight_bucket_size option that serves to divide the actual size by a
given factor. This allows considering peers with similar queue sizes to have
the same weight.

Additionally, some changes have been made to the balanced allocator so that a
combination of tags, pinqueue sizes and free-spaces can be used. When
allocating by [<tag>, pinqueue, freespace], the allocator will prioritize
choosing peers with the smallest pin queue weight first, and of those with the
same weight, it will allocate based on freespace.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
8dc18cd3d7 Metrics: add additional metrics 
* Gauge for total number of ipfs pins
* Counter for pin/add
* Counter for pin/add errors
* Counter for Block/Puts
* Counter for blocks added
* Counter for block/added size
* Counted for block/added errors
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
333b933cd3 service: load pinsvc api first (just order consistency) 2022-03-11 00:45:44 +01:00
Hector Sanjuan
fd57a032e3 service: randomize pinsvcapi ports with --randomports too 2022-03-11 00:07:26 +01:00
Hector Sanjuan
524e2ecbc0 Wire in pinsvcapi to ipfs-cluster-service 2022-02-02 00:52:39 +01:00
Hector Sanjuan
e7ae13e6cf service: add buffered io for import and export 2022-01-04 18:15:06 +01:00
Hector Sanjuan
f4c3fc4ce5 State import: allow replication factor and allocations overwrite on import 2021-12-14 11:21:19 +01:00
Hector Sanjuan
3f895df8cf Fix: backwards compatibility for allocator default settings 
When the allocator is not defined in the configuration, it will take defaults
and assume there is a "tags" informer. That is not the case. When not defined,
we assume it should allocate only by "freespace".
 2021-10-20 16:55:35 +02:00
Hector Sanjuan
af9313d4fe Remove test files that should have not been committed 2021-10-19 13:51:17 +02:00
Hector Sanjuan
26e229df94 Rename allocator/metrics to allocator/balanced 2021-10-06 11:26:38 +02:00
Hector Sanjuan
cf4fb74993 service/follow: Enable new metrics allocator 2021-09-15 21:49:26 +02:00
Hector Sanjuan
2f21724d1d logging: correctly use defaults for LoggingFacilitiesExtra 2021-08-06 11:31:05 +02:00
Hector Sanjuan
46a14f9602 Set badger as default datastore for ipfs-cluster-service 
This goes back to badger as the default. Testing performed on 0.14.0-rc2
showed very successful GC behaviour on large clusters.

Being that we run with badger at scale and that the main problem seems
resolved, it seems appropiate to default to badger.
 2021-07-08 00:37:03 +02:00
Hector Sanjuan
099e23cbd1 Run tests also using leveldb backend 2021-06-11 18:43:54 +02:00
Hector Sanjuan
0eef0ede89 Support a levelDB backend for cluster 
Badger can take 1000x the amount of needed space if not GC'ed or compacted
(#1320), even for non heavy usage. Cluster has no provisions to run datastore
GC operations and while they could be added, they are not ensured to
help. Improvements on Badger v3 might help but would still need to GC
explicitally.

Cluster was however designed to support any go-datastore as backend.

This commit adds LevelDB support. LevelDB go-datastore wrapper is mature, does
not need GC and should work well for most cluster usecases, which are not
overly demanding.

A new `--datastore` flag has been added on init. The store backend is selected
based on the value in the configuration, similar to how raft/crdt is. The
default is set to leveldb. From now on it should be easier to add additional
backends, i.e. badgerv3.
 2021-06-09 19:40:36 +02:00
Hector Sanjuan
a56ce73f92 Switch to building with -trimpath instead of asm and gc trimpath flags 
Kudos to https://github.com/ipfs/distributions/pull/314
 2020-10-20 14:32:20 +02:00
Kishan Mohanbhai Sagathiya
ae8e74453b Fix #937: Print full working configuration at startup 
Only when using debug mode

Co-authored-by: Hector Sanjuan <code@hector.link>
 2020-05-15 01:33:04 +02:00
Hector Sanjuan
7ffd18e41b Feat: upgrade to dual DHT 2020-04-14 22:03:24 +02:00
Hector Sanjuan
65ad4bd632 cluster/daemons: Close the datastore AFTER the DHT. 
Avoids panics.

This also removes the abnormality of cluster closing a datastore
that it did not create.
 2020-04-02 16:29:41 +02:00
Hector Sanjuan
ab74987bc8 clusterhost: place dht records in persistened DHT 
With this cluster peers will start remembering provider records accross
re-starts.
 2020-04-01 20:15:48 +02:00
Hector Sanjuan
b3853caf36 Dependency ugprade: changes needed 
* Libp2p protectors no longer needed, use PSK directly
* Generate cluster 32-byte secret here (helper gone from pnet)
* Switch to go-log/v2 in all places
* DHT bootstrapping not needed. Adjust DHT options for tests.
* Do not rely on dissappeared CidToDsKey and DsKeyToCid functions fro dshelp.
* Disable QUIC (does not support private networks)
* Fix tests: autodiscovery started working properly
 2020-03-22 14:50:25 +01:00
Hector Sanjuan
c80a207e41 Fix #1012: Trim paths on Makefile builds 
This results in more reproduceable builds.

Additionally, sources command to list all go-files and not just those
one directory below.
 2020-03-06 14:49:19 +01:00
Hector Sanjuan
5d44275d0e
Chore: Add license file with URLS (#1014) 
* Chore: Add license file with URLS

Source: 59df5e58a2

* Chore: license copyright updates
 2020-03-06 13:52:19 +01:00
Hector Sanjuan
531379b1d9
Feature: Support multiple listeners in configuration 
* add ipv6 listening addresses to the default config

* ipfsproxy: support multiple listeners. Add default ipv6.

* mm

* restapi: support multiple listen addresses. enable ipv6

* cluster_config: format default listen addresses

* commands: update for multiple listeners. Fix randomports for udp and ipv6.

* ipfs-cluster-service: fix randomports test

* multiple listeners: fix remaining tests

* golint

* Disable ipv6 in defaults

It is not supported by docker by default. It is not supported in travis-CI
build environments. User can enable it now manually.

* proxy: disable ipv6 in test

* ipfshttp: fix test

Co-authored-by: @RubenKelevra <cyrond@gmail.com>
 2020-02-28 11:16:16 -05:00
Yang Hau
7986d94242
fix: Fix typos (#1001) 
Fix typos in files
 2020-02-03 10:30:04 +01:00
Hector Sanjuan
60ce5b3228 Update copyrights to 2020x 2019-12-20 14:40:34 +01:00
Hector Sanjuan
0fa01d5435 Distribute the correct dual license with ctl and service 2019-12-20 14:04:48 +01:00
Hector Sanjuan
2391d2d246
Merge pull request #960 from ipfs/feat/log-env-flags 
Environment variable to set loglevels for individual identifiers
 2019-12-16 17:20:17 +01:00
Kishan Sagathiya
5258a4d428 Remove map pintracker (#944) 
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.

Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
 2019-12-12 21:22:54 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils 
Service: extract some methods to cmdutils
 2019-12-06 11:08:58 +01:00
Kishan Mohanbhai Sagathiya
a136509522 No need to verify log levels 2019-12-06 11:22:08 +05:30
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt 
Fix: disable default libp2p API endpoint with CRDT consensus
 2019-12-05 16:14:35 +01:00
Kishan Sagathiya
30fd5ee6a0 Feat: Allow and run multiple informer components (#962) 
This introduces the possiblity of running Cluster with multiple informer components. The first one in the list is the used for "allocations". This is just groundwork for working with several informers in the future.
 2019-12-05 15:08:43 +01:00
Kishan Mohanbhai Sagathiya
df36995213 detail description and example for how to customize logging 2019-12-04 23:22:38 +05:30
Kishan Mohanbhai Sagathiya
794646ff12 Addressed reviews 2019-12-04 23:10:35 +05:30
Hector Sanjuan
f6ea95bccf Service/cmdutils: fix small typos (review) 2019-12-02 16:08:47 +01:00