Hector Sanjuan
5452b59a2e
Dependency upgrades ( #1755 )
...
* Update go-libp2p to v0.22.0
* Testing with go1.19
* build(deps): bump github.com/multiformats/go-multicodec
Bumps [github.com/multiformats/go-multicodec](https://github.com/multiformats/go-multicodec ) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/multiformats/go-multicodec/releases )
- [Commits](https://github.com/multiformats/go-multicodec/compare/v0.5.0...v0.6.0 )
---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multicodec
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/ipld/go-car from 0.4.0 to 0.5.0
Bumps [github.com/ipld/go-car](https://github.com/ipld/go-car ) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/ipld/go-car/releases )
- [Commits](https://github.com/ipld/go-car/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: github.com/ipld/go-car
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/prometheus/client_golang
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang ) from 1.12.2 to 1.13.0.
- [Release notes](https://github.com/prometheus/client_golang/releases )
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md )
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.2...v1.13.0 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/hashicorp/go-hclog from 1.2.1 to 1.3.0
Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog ) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/hashicorp/go-hclog/releases )
- [Commits](https://github.com/hashicorp/go-hclog/compare/v1.2.1...v1.3.0 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/ipfs/go-ds-crdt from 0.3.6 to 0.3.7
Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt ) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases )
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.3.6...v0.3.7 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/urfave/cli/v2 from 2.10.2 to 2.14.1
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli ) from 2.10.2 to 2.14.1.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v2.10.2...v2.14.1 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/libp2p/go-libp2p-http from 0.3.0 to 0.4.0
Bumps [github.com/libp2p/go-libp2p-http](https://github.com/libp2p/go-libp2p-http ) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/libp2p/go-libp2p-http/releases )
- [Commits](https://github.com/libp2p/go-libp2p-http/compare/v0.3.0...v0.4.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-http
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/libp2p/go-libp2p-gorpc from 0.4.0 to 0.5.0
Bumps [github.com/libp2p/go-libp2p-gorpc](https://github.com/libp2p/go-libp2p-gorpc ) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/libp2p/go-libp2p-gorpc/releases )
- [Commits](https://github.com/libp2p/go-libp2p-gorpc/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-gorpc
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump contrib.go.opencensus.io/exporter/prometheus
Bumps [contrib.go.opencensus.io/exporter/prometheus](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus ) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/releases )
- [Commits](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/compare/v0.4.1...v0.4.2 )
---
updated-dependencies:
- dependency-name: contrib.go.opencensus.io/exporter/prometheus
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/libp2p/go-libp2p-raft from 0.1.8 to 0.2.0
Bumps [github.com/libp2p/go-libp2p-raft](https://github.com/libp2p/go-libp2p-raft ) from 0.1.8 to 0.2.0.
- [Release notes](https://github.com/libp2p/go-libp2p-raft/releases )
- [Commits](https://github.com/libp2p/go-libp2p-raft/compare/v0.1.8...v0.2.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-raft
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/urfave/cli from 1.22.9 to 1.22.10
Bumps [github.com/urfave/cli](https://github.com/urfave/cli ) from 1.22.9 to 1.22.10.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v1.22.9...v1.22.10 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fix checker/linter/staticcheck warnings
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-06 16:57:17 +02:00
Hector Sanjuan
d19c7facff
Fix: leaking goroutines on aborted /add requests
...
It has been observed that some peers have a growing number of goroutines,
usually stuck in go-libp2p-gorpc.MultiStream() function, which is waiting to
read items from the arguments channel.
We suspect this is due to aborted /add requests. In situations when the add
request is aborted or fails, Finalize() is never called and the blocks channel
stays open, so MultiStream() can never exit, and the BlockStreamer can never
stop streaming etc.
As a fix, we added the requirement to call Close() when we stop using a
ClusterDAGService (error or not). This should ensure that the blocks channel
is always closed and not just on Finalize().
2022-07-08 17:39:59 +02:00
Hector Sanjuan
455aa6def1
Fix #1691 : Error when adding directories with many files
...
Fixes #1691 by updating to the latest go-unixfs and adding a test.
The test is verified to fail on the previous go-unixfs version.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
755cebbe0d
Enable spell checking and fix spelling errors (using US locale)
2022-06-16 17:43:30 +02:00
Hector Sanjuan
508791b547
Migrate from ipfs/ipfs-cluster to ipfs-cluster/ipfs-cluster
...
This performs the necessary renamings.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
a97ed10d0b
Adopt api.Cid type - replaces cid.Cid everwhere.
...
This commit introduces an api.Cid type and replaces the usage of cid.Cid
everywhere.
The main motivation here is to override MarshalJSON so that Cids are
JSON-ified as '"Qm...."' instead of '{ "/": "Qm....." }', as this "ipld"
representation of IDs is horrible to work with, and our APIs are not issuing
IPLD objects to start with.
Unfortunately, there is no way to do this cleanly, and the best way is to just
switch everything to our own type.
2022-04-07 14:27:39 +02:00
Hector Sanjuan
759dd4802d
Fix 1598: Include allocations in responses from /add
...
This adds a new allocations field to add response objects which
provides the cluster peers to which the content has been allocated.
In the case of sharded dags, it provides peers for the current shard.
2022-03-15 11:03:11 +01:00
Hector Sanjuan
26007114b4
Adder: try to make test less flaky
2021-08-29 01:16:28 +02:00
Hector Sanjuan
072f0bc722
Feat: support adding CAR files
...
This commit adds a new add option: "format".
This option specifies how IPFS Cluster is expected to build the DAG when
adding content. By default, it takes a "unixfs", which chunks and DAG-ifies as
it did before, resulting in a UnixFSv1 DAG.
Alternatively, it can be set to "car". In this case, Cluster will directly
read blocks from the CAR file and add them.
Adding CAR files or doing normal processing is independent from letting
cluster do sharding or not. If sharding is ever enabled, Cluster could
potentially shard a large CAR file among peers.
Currently, importing CAR files is limited to a single CAR file with a single
root (the one that is pinned). Future iterations may support multiple CARs
and/or multiple roots by transparently wrapping them.
2021-04-21 13:55:06 +02:00
Kishan Mohanbhai Sagathiya
70e429f925
Fix #852 : Improve error handling on add
...
Keep sending blocks while adding as long as one destination works.
2019-08-13 16:06:20 +02:00
Hector Sanjuan
3720d288a0
Adder: fix old mock rpc in tests
2019-08-09 17:13:01 +02:00
Hector Sanjuan
ced9f2f7f1
Update to go-ipfs-files 2.0.3
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:29:44 +01:00
Hector Sanjuan
15413edbc6
Unrewrite deps
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
406d2b53f8
files2.0: Close subfiles in adder
...
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
9b0ff603b8
files2.0: use type helpers
...
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
c6944e4376
files2.0: update for no errors from NewMultiFileReader
...
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
d0678aad6e
files2.0: fix ContextCancelled test
...
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Adrian Lanzafame
31474f6490
update go-cid and go-libp2p
...
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-24 11:35:38 +10:00
Hector Sanjuan
7b826a5335
Dependency upgrades
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:34 +02:00
Hector Sanjuan
f707621a39
Allow to use adder only once
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 11:51:27 +02:00
Hector Sanjuan
50fc3c4e95
Address comments from review
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05
Fix: close files. Close test files. Clean properly. Fix windows tests.
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
26cbecca4c
Fix tests
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 03:42:39 +02:00
Hector Sanjuan
78b8f47c14
Fix: Wrap-in-directory
...
Usually we had wrap-in-directory enabled by default because otherwise
we had an error when adding single, non-directory files.
wrap-in-directory happens automatically when adding more than one file
so that was no problem. Thigns also worked when adding a folder and Wrap was
disabled. The only case was adding a single with wrap disabled (a default option).
This patches the ipfsadd/add.go file to remember the last added file so that
we can use it's Cid as the resulting root of the adding process without
having to fetch it from our dummy dagservice.
We have to pass this CID to our Finalize() functions, because it turns out that
in this case (single file without wrap-in-directory), the last block added to the DAG
is not the IPFS root (the ipfsadd/Adder adds the mfs root folder last always).
This was the case when wrap-in-directory was enabled by default.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 01:16:30 +02:00
Hector Sanjuan
0ed14a4b39
Adder: add context to add functions. Fix allocation everywhere. Helpers.
...
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-08 21:10:42 +02:00
Hector Sanjuan
87f4fcf958
Make the adder modules ipld.DAGService modules.
...
This removes a bunch of the channel dance and block forwarding
by having the adder submodules be DAGServices themselves and take
Add() directly from the ipfsAdder.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00