cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,729 Commits 3 Branches 70 Tags 15 MiB
66bf7d44bf
Commit Graph

349 Commits

Author SHA1 Message Date
Hector Sanjuan
66bf7d44bf update to libp2p v0.19.1 and go1.18 2022-06-16 17:43:29 +02:00
dependabot[bot]
b38ea6b4a0 build(deps): bump github.com/urfave/cli from 1.22.8 to 1.22.9 
Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.8 to 1.22.9.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.8...v1.22.9)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
dependabot[bot]
74b7215482 build(deps): bump github.com/urfave/cli from 1.22.5 to 1.22.8 
Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.5 to 1.22.8.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.5...v1.22.8)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
dependabot[bot]
2d8d477278 build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.5.1 
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.0 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
95521b06a0
Merge pull request #1627 from ipfs/dependency-upgrades 
Dependency upgrades
 2022-04-07 14:48:02 +02:00
Hector Sanjuan
c48f48d5a1
Merge pull request #1625 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipld-format-0.4.0 
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.1 to 0.4.0
 2022-04-07 14:30:02 +02:00
dependabot[bot]
4984e446d7
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.1 to 0.4.0 
Bumps [github.com/ipfs/go-ipld-format](https://github.com/ipfs/go-ipld-format) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/ipfs/go-ipld-format/releases)
- [Commits](https://github.com/ipfs/go-ipld-format/compare/v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipld-format
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 11:08:14 +00:00
dependabot[bot]
68e1037f0f
build(deps): bump github.com/ipfs/go-log/v2 from 2.5.0 to 2.5.1 
Bumps [github.com/ipfs/go-log/v2](https://github.com/ipfs/go-log) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/ipfs/go-log/releases)
- [Commits](https://github.com/ipfs/go-log/compare/v2.5.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-log/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 11:07:34 +00:00
Hector Sanjuan
b7769493f4
Merge pull request #1618 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-path-0.3.0 
build(deps): bump github.com/ipfs/go-path from 0.2.2 to 0.3.0
 2022-03-28 20:22:54 +02:00
dependabot[bot]
e41cd74e61
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.0 to 0.3.1 
Bumps [github.com/ipfs/go-ipld-format](https://github.com/ipfs/go-ipld-format) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/ipfs/go-ipld-format/releases)
- [Commits](https://github.com/ipfs/go-ipld-format/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipld-format
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-28 11:06:14 +00:00
dependabot[bot]
befec2cc82
build(deps): bump github.com/ipfs/go-path from 0.2.2 to 0.3.0 
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path) from 0.2.2 to 0.3.0.
- [Release notes](https://github.com/ipfs/go-path/releases)
- [Commits](https://github.com/ipfs/go-path/compare/v0.2.2...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-25 12:40:27 +00:00
Hector Sanjuan
267d8c2a17 Fix builds on darwin, openbsd... 
go-libp2p-connmgr does not build on some architectures due to some
dependencies (https://github.com/libp2p/go-libp2p-connmgr/pull/103). By
extension, go-libp2p does not build when using the connmgr.

The fix is in master.
 2022-03-25 13:38:05 +01:00
Hector Sanjuan
58aea6c392
Merge pull request #1606 from ipfs/dependabot/go_modules/dependency-upgrades/contrib.go.opencensus.io/exporter/prometheus-0.4.1 
build(deps): bump contrib.go.opencensus.io/exporter/prometheus from 0.4.0 to 0.4.1
 2022-03-23 14:13:10 +01:00
Hector Sanjuan
10e93a7c46
Merge pull request #1608 from ipfs/dependabot/go_modules/dependency-upgrades/google.golang.org/protobuf-1.28.0 
build(deps): bump google.golang.org/protobuf from 1.27.1 to 1.28.0
 2022-03-23 14:13:02 +01:00
Hector Sanjuan
2dd8d5b1ec Dependency: update go-codec-dagpb 2022-03-23 02:05:38 +01:00
Hector Sanjuan
eee53bfa4f Streaming Peers(): make Peers() a streaming call 
This commit makes all the changes to make Peers() a streaming call.

While Peers is usually a non problematic call, for consistency, all calls
returning collections assembled through broadcast to cluster peers are now
streaming calls.
 2022-03-23 01:27:57 +01:00
dependabot[bot]
7aa1f3344d
build(deps): bump google.golang.org/protobuf from 1.27.1 to 1.28.0 
Bumps [google.golang.org/protobuf](https://github.com/protocolbuffers/protobuf-go) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/protocolbuffers/protobuf-go/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf-go/blob/master/release.bash)
- [Commits](https://github.com/protocolbuffers/protobuf-go/compare/v1.27.1...v1.28.0)

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-22 14:53:37 +00:00
Hector Sanjuan
0d73d33ef5 Pintracker: streaming methods 
This commit continues the work of taking advantage of the streaming
capabilities in go-libp2p-gorpc by improving the ipfsconnector and pintracker
components.

StatusAll and RecoverAll methods are now streaming methods, with the REST API
output changing accordingly to produce a stream of GlobalPinInfos rather than
a json array.

pin/ls request to the ipfs daemon now use ?stream=true and avoid having to
load the full pinset map on memory. StatusAllLocal and RecoverAllLocal
requests to the pin tracker stream all the way and no longer store the full
pinset, and the full PinInfo status slice before sending it out.

We have additionally switched to a pattern where streaming methods receive the
channel as an argument, allowing the caller to decide on whether to launch a
goroutine, do buffering etc.
 2022-03-22 15:38:01 +01:00
dependabot[bot]
d2fc658e59
build(deps): bump contrib.go.opencensus.io/exporter/prometheus 
Bumps [contrib.go.opencensus.io/exporter/prometheus](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus) from 0.4.0 to 0.4.1.
- [Release notes](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/releases)
- [Commits](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/compare/v0.4.0...v0.4.1)

---
updated-dependencies:
- dependency-name: contrib.go.opencensus.io/exporter/prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-19 11:40:12 +00:00
Hector Sanjuan
f5d78811c1 update go-libp2p-gorpc to 0.3.0 2022-03-19 12:09:23 +01:00
Hector Sanjuan
9b9d76f92d Pinset streaming and method type revamp 
This commit introduces the new go-libp2p-gorpc streaming capabilities for
Cluster. The main aim is to work towards heavily reducing memory usage when
working with very large pinsets.

As a side-effect, it takes the chance to revampt all types for all public
methods so that pointers to static what should be static objects are not used
anymore. This should heavily reduce heap allocations and GC activity.

The main change is that state.List now returns a channel from which to read
the pins, rather than pins being all loaded into a huge slice.

Things reading pins have been all updated to iterate on the channel rather
than on the slice. The full pinset is no longer fully loaded onto memory for
things that run regularly like StateSync().

Additionally, the /allocations endpoint of the rest API no longer returns an
array of pins, but rather streams json-encoded pin objects directly. This
change has extended to the restapi client (which puts pins into a channel as
they arrive) and to ipfs-cluster-ctl.

There are still pending improvements like StatusAll() calls which should also
stream responses, and specially BlockPut calls which should stream blocks
directly into IPFS on a single call.

These are coming up in future commits.
 2022-03-19 03:02:55 +01:00
Hector Sanjuan
6b0b0afdb0
Dependency upgrades (#1600) 
Dependency upgrades
 2022-03-15 11:50:44 +01:00
Hector Sanjuan
8583953357 Update go-ds-crdt with panic fix 2022-02-16 14:20:46 +01:00
Hector Sanjuan
3ffaf0fcc7
Merge pull request #1579 from ipfs/gorpc-upgrade 
go-libp2p-gorpc upgrade
 2022-02-15 19:33:43 +01:00
Hector Sanjuan
fa36958221 Merge branch 'master' into dependency-upgrades 2022-02-15 19:14:50 +01:00
Hector Sanjuan
9a0bca222b go-libp2p-gorpc upgrade 2022-02-15 19:12:12 +01:00
Hector Sanjuan
dc2b82e772
Merge pull request #1565 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-files-0.1.1 
Bump github.com/ipfs/go-ipfs-files from 0.1.0 to 0.1.1
 2022-02-15 19:11:58 +01:00
Hector Sanjuan
5fb851babf
Merge pull request #1577 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ds-crdt-0.3.1 
Bump github.com/ipfs/go-ds-crdt from 0.3.0 to 0.3.1
 2022-02-15 19:11:47 +01:00
dependabot[bot]
a1841a3162
Bump github.com/ipfs/go-ds-crdt from 0.3.0 to 0.3.1 
Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-15 16:41:12 +00:00
Hector Sanjuan
975814229c crdt: increase number of workers from 5 to 50. Upgrade. 
The go-ds-crdt upgrade disables multi-head-processing by default again. We see
this causes a lot of branching.

We however increase the number of workers. With large deltas, it may be
possible that all the 5 workers are busy downloading a delta or processing
them, while we potentially have hundreds of children in the DAG. Thus it is
not bad to attempt to do more things in parallel.
 2022-02-15 16:54:42 +01:00
dependabot[bot]
a9250966a2
Bump github.com/ipfs/go-path from 0.2.1 to 0.2.2 
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/ipfs/go-path/releases)
- [Commits](https://github.com/ipfs/go-path/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 11:09:39 +00:00
dependabot[bot]
55e9ab622c
Bump github.com/ipfs/go-ipfs-files from 0.1.0 to 0.1.1 
Bumps [github.com/ipfs/go-ipfs-files](https://github.com/ipfs/go-ipfs-files) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/ipfs/go-ipfs-files/releases)
- [Commits](https://github.com/ipfs/go-ipfs-files/compare/v0.1.0...v0.1.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-07 11:13:37 +00:00
Hector Sanjuan
670f9ac7d9
Merge pull request #1550 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-files-0.1.0 
Bump github.com/ipfs/go-ipfs-files from 0.0.9 to 0.1.0
 2022-02-02 01:29:29 +01:00
Hector Sanjuan
92e5eaa58e
Merge pull request #1557 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/prometheus/client_golang-1.12.1 
Bump github.com/prometheus/client_golang from 1.11.0 to 1.12.1
 2022-02-02 01:07:43 +01:00
Hector Sanjuan
fbc45b17e4
Merge pull request #1558 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-config-0.19.0 
Bump github.com/ipfs/go-ipfs-config from 0.18.0 to 0.19.0
 2022-02-02 01:07:25 +01:00
dependabot[bot]
8786588ac1
Bump github.com/ipfs/go-ds-crdt from 0.2.4 to 0.3.0 
Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.2.4 to 0.3.0.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.2.4...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-01 23:41:52 +00:00
Hector Sanjuan
881266a1a0 Update go-ds-crdt 
The new version supports auto-repair of the datastore.

Fixes #1007.
 2022-02-01 23:16:50 +01:00
dependabot[bot]
c331026ca5
Bump github.com/ipfs/go-ipfs-config from 0.18.0 to 0.19.0 
Bumps [github.com/ipfs/go-ipfs-config](https://github.com/ipfs/go-ipfs-config) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/ipfs/go-ipfs-config/releases)
- [Commits](https://github.com/ipfs/go-ipfs-config/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-config
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-31 11:07:15 +00:00
dependabot[bot]
36f85a96f8
Bump github.com/prometheus/client_golang from 1.11.0 to 1.12.1 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.11.0 to 1.12.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.11.0...v1.12.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-31 11:06:44 +00:00
dependabot[bot]
fa7d7ded12
Bump github.com/ipfs/go-ipfs-files from 0.0.9 to 0.1.0 
Bumps [github.com/ipfs/go-ipfs-files](https://github.com/ipfs/go-ipfs-files) from 0.0.9 to 0.1.0.
- [Release notes](https://github.com/ipfs/go-ipfs-files/releases)
- [Commits](https://github.com/ipfs/go-ipfs-files/compare/v0.0.9...v0.1.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-24 11:06:22 +00:00
Hector Sanjuan
f2ccf7a772 Update go-ds-crdt 2022-01-11 18:08:20 +01:00
Hector Sanjuan
a93886aeee
Merge pull request #1540 from ipfs/dependency-upgrades 
Dependency upgrades
 2022-01-11 17:11:13 +01:00
Hector Sanjuan
acbe12b034
Merge pull request #1539 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/hashicorp/go-hclog-1.1.0 
Bump github.com/hashicorp/go-hclog from 1.0.0 to 1.1.0
 2022-01-11 17:09:49 +01:00
Hector Sanjuan
862f4b515f Update protobuf compiler and generated files 2022-01-11 16:47:45 +01:00
dependabot[bot]
aa50177cc4
Bump github.com/hashicorp/go-hclog from 1.0.0 to 1.1.0 
Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/hashicorp/go-hclog/releases)
- [Commits](https://github.com/hashicorp/go-hclog/compare/v1.0...v1.1.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-11 15:33:10 +00:00
Hector Sanjuan
5abb804362 Update go-libp2p 2022-01-11 16:31:55 +01:00
Hector Sanjuan
dbb68c2a80
Merge pull request #1535 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/rs/cors-1.8.2 
Bump github.com/rs/cors from 1.8.0 to 1.8.2
 2022-01-10 23:04:10 +01:00
Hector Sanjuan
3537c8ebd5
Merge pull request #1527 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-path-0.2.1 
Bump github.com/ipfs/go-path from 0.1.2 to 0.2.1
 2022-01-10 23:03:52 +01:00
dependabot[bot]
261a87e205
Bump github.com/ipfs/go-path from 0.1.2 to 0.2.1 
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path) from 0.1.2 to 0.2.1.
- [Release notes](https://github.com/ipfs/go-path/releases)
- [Commits](https://github.com/ipfs/go-path/compare/v0.1.2...v0.2.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-10 14:52:07 +00:00
Hector Sanjuan
97a58adc2b
Merge pull request #1526 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-api-0.3.0 
Bump github.com/ipfs/go-ipfs-api from 0.2.0 to 0.3.0
 2022-01-10 15:51:42 +01:00