cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,924 Commits 3 Branches 70 Tags 15 MiB
af89f73c2e
Commit Graph

2924 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot]
2d8d477278 build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.5.1 
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.0 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
ec35841918
Merge pull request #1656 from ipfs/fix/0-space 
Do not issue freespace metric when free-space is 0.
 2022-05-03 19:09:31 +02:00
Hector Sanjuan
060ef812b2 Do not issue freespace metric when free-space is 0. 2022-05-03 18:39:44 +02:00
Hector Sanjuan
0b25f9ce60
Merge pull request #1651 from ipfs/fix/tracker-metrics 
Fix: tracker metrics not updated on phase change
 2022-05-03 18:23:53 +02:00
Hector Sanjuan
8252944aa8 Fix: tracker metrics not updated on phase change 2022-04-30 20:57:57 +02:00
Hector Sanjuan
d89e62117b
Merge pull request #1646 from ipfs/fix/leaks-on-ipfs-restart 
Fix: bad behaviour when adding and ipfs is down
 2022-04-29 22:57:26 +02:00
Hector Sanjuan
585eb1e8e9
Merge pull request #1647 from ipfs/fix/tracker-metrics-gauge 
pintracker: metrics: convert pinning/queued/error metrics to gauges
 2022-04-29 22:57:04 +02:00
Hector Sanjuan
bcecb8e1b4 ipfshttp: do not return io.EOF errors to the caller 2022-04-26 15:20:01 +02:00
Hector Sanjuan
de40b2cd23 pintracker: metrics: convert pinning/queued/error metrics to gauges 
We were currently tracking this metrics as a counter (SUM). The number is
good, but conceptually this is more a gauge (LastValue), given it can go down.

Thus we switch it by tracking the aggregation numbers directy in the operation
tracker.
 2022-04-26 15:13:35 +02:00
Hector Sanjuan
8626fbc166 Fix: bad behaviour when adding and ipfs is down 
Adding keeps retrying indefinitely to send blocks to ipfs. When ipfs is down this:

* Stores all errors
* Keeps retrying
* Additionally, forgot to close bodies

This resulted in memory leaks. The new behaviour does not keep retrying. And
response bodies are closed after reading.
 2022-04-26 14:54:04 +02:00
Hector Sanjuan
09814e5176
Merge pull request #1645 from ipfs/fix/metrics-batching-state 
Fix: wrong pins metric when batching enabled
 2022-04-26 14:53:13 +02:00
Hector Sanjuan
cfae287550 ipfshttp: do not touch the pins metric here 
The connector was issuing pins metrics and it is all wrong.
 2022-04-26 00:36:41 +02:00
Hector Sanjuan
6716f5471a Fix: wrong pins metric when batching enabled 
When batching is enabled, the "batchingstate" is used to add/remove pins, but the
non-batching state is used as read-only state for doing List().

This means that both states will be writing pins metrics but the batching
state is never used for List(), so it never has the right total number.

This fixes that.
 2022-04-25 23:45:34 +02:00
Hector Sanjuan
cc40a37daa Fix date in changelog 2022-04-22 16:46:56 +02:00
Hector Sanjuan
6f01277bd1
Release v1.0.0 2022-04-22 16:42:17 +02:00
Hector Sanjuan
4075789167
Merge pull request #1619 from ipfs/v1.0.0/changelog 
Changelog for v1.0.0
 2022-04-22 16:39:23 +02:00
Hector Sanjuan
4d66390283 Changelog for v1.0.0 2022-04-22 16:35:17 +02:00
Hector Sanjuan
e69e690194
Merge pull request #1637 from ipfs/pintracker-metrics 
metrics: track total pins, queued, pinning, pin error.
 2022-04-22 16:15:13 +02:00
Hector Sanjuan
3169fba9d1 metrics: track total pins, queued, pinning, pin error. 
This fixes #1470 and #1187.
 2022-04-22 15:57:48 +02:00
Hector Sanjuan
4b351cad47
Merge pull request #1634 from ipfs/fix/1633-ctl-error-code 
ctl: Set exit status to 1 when app.Run() fails
 2022-04-12 12:04:15 +02:00
Hector Sanjuan
288e7f9e3d ctl: Set exit status to 1 when app.Run() fails 
1 corresponds to arguments errors and anything that prevents the app from
making a request.
 2022-04-12 11:49:26 +02:00
Hector Sanjuan
e4b91af901
Release candidate v1.0.0-rc5 2022-04-11 21:29:23 +02:00
Hector Sanjuan
df47b38b2d
Merge pull request #1632 from ipfs/fix-block-cid-mismatch 
Fix warnings on block-cid mismatch when adding
 2022-04-11 20:59:11 +02:00
Hector Sanjuan
cd2fe8f655 Fix warnings on block-cid mismatch when adding 
Because we are adding blocks on a single call, and we choose the format
parameter based on the prefix of the first block, IPFS will return block CIDs
based on that option.

This caused warnings when adding content that has multiple CID prefixes: for
example, any cid-version=1 file will include both dag-pb CIDs and raw
CIDs. Since the first block is usually a leave, IPFS will only return
raw-cids, and cause a warning because of the CID-mistmatch.

This fixes things by comparing multihashes only.

But! We might be writing blocks with the wrong CID and then the good CID won't
work!

Correct, we might, in some corner cases.

In go-ipfs >= 0.12.0, all blocks are addressed by multihash so CID prefixes
are irrelevant. This problem does not exist in that case.

In go-ipfs < 0.12.0, if a read for a CIDv1 DAG-PB fails, it is retried as it
it was raw. This means that if we wrote something with cidv1/format=raw, that
should have been a cidv1/format=dag-pb, the read will still work. That covers
some common cases (i.e. adding with cid-version=1) because the first block
should be a raw-leaf. Default-params (cidv0) is not affected since everything
is raw multihashes. However, there are still possible CAR layouts etc.  where
cluster will write blocks wrongly to older IPFS versions.
 2022-04-11 20:44:40 +02:00
Hector Sanjuan
2827427be8
Merge pull request #1631 from ipfs/fix/recover-all 
Bug fixing for v1.0.0-rc4
 2022-04-11 19:25:11 +02:00
Hector Sanjuan
fc93e23272 ipfshttp: make blockPut warning a debug message 
Something is wrong with it, when adding CAR files.
 2022-04-11 15:07:13 +02:00
Hector Sanjuan
3f07b20f3b pintracker: Deadlock on RecoverAll and StatusAll methods 2022-04-11 15:03:33 +02:00
Hector Sanjuan
81fa2c5665
Release candidate v1.0.0-rc4 2022-04-07 15:06:04 +02:00
Hector Sanjuan
e2403c682f
Merge pull request #1626 from ipfs/feat/cid-strings 
Adopt api.Cid type - replaces cid.Cid everwhere.
 2022-04-07 15:05:00 +02:00
Hector Sanjuan
6706cb9e51
Merge pull request #1628 from ipfs/feat/pin-type-json 
types: serialize pin types as string and not as a number
 2022-04-07 15:01:41 +02:00
Hector Sanjuan
fbeb7de9f1 Sharness: update state export file 2022-04-07 14:54:16 +02:00
Hector Sanjuan
95521b06a0
Merge pull request #1627 from ipfs/dependency-upgrades 
Dependency upgrades
 2022-04-07 14:48:02 +02:00
Hector Sanjuan
f01612ca63 types: serialize pin types as string and not as a number 
The pin type in json objects will not be understandable by humans
 2022-04-07 14:44:04 +02:00
Hector Sanjuan
c48f48d5a1
Merge pull request #1625 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipld-format-0.4.0 
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.1 to 0.4.0
 2022-04-07 14:30:02 +02:00
Hector Sanjuan
595d8c8aeb
Merge pull request #1624 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-log/v2-2.5.1 
build(deps): bump github.com/ipfs/go-log/v2 from 2.5.0 to 2.5.1
 2022-04-07 14:29:53 +02:00
Hector Sanjuan
153eeb2107
Merge pull request #1623 from ipfs/feat/always-include-error-trailer 
Always include the X-Stream-Error trailer (even without errors)
 2022-04-07 14:29:38 +02:00
Hector Sanjuan
a97ed10d0b Adopt api.Cid type - replaces cid.Cid everwhere. 
This commit introduces an api.Cid type and replaces the usage of cid.Cid
everywhere.

The main motivation here is to override MarshalJSON so that Cids are
JSON-ified as '"Qm...."' instead of '{ "/": "Qm....." }', as this "ipld"
representation of IDs is horrible to work with, and our APIs are not issuing
IPLD objects to start with.

Unfortunately, there is no way to do this cleanly, and the best way is to just
switch everything to our own type.
 2022-04-07 14:27:39 +02:00
dependabot[bot]
4984e446d7
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.1 to 0.4.0 
Bumps [github.com/ipfs/go-ipld-format](https://github.com/ipfs/go-ipld-format) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/ipfs/go-ipld-format/releases)
- [Commits](https://github.com/ipfs/go-ipld-format/compare/v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipld-format
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 11:08:14 +00:00
dependabot[bot]
68e1037f0f
build(deps): bump github.com/ipfs/go-log/v2 from 2.5.0 to 2.5.1 
Bumps [github.com/ipfs/go-log/v2](https://github.com/ipfs/go-log) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/ipfs/go-log/releases)
- [Commits](https://github.com/ipfs/go-log/compare/v2.5.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-log/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 11:07:34 +00:00
Hector Sanjuan
d1ab0bd0ee Adjust test checking for trailer errors 2022-04-04 12:45:34 +02:00
Hector Sanjuan
787aea5f31 Always include the X-Stream-Error trailer (even without errors) 
This is to potentially address things like this:

https://github.com/nodejs/undici/issues/432#issuecomment-1047931107
 2022-04-04 12:09:54 +02:00
Hector Sanjuan
a2b89f9b3b
Release candidate v1.0.0-rc3 2022-03-28 20:44:41 +02:00
Hector Sanjuan
e19809af9d
Merge pull request #1621 from ipfs/rc-fixes 
Fixes to BlockStream and adder
 2022-03-28 20:44:13 +02:00
Hector Sanjuan
b7769493f4
Merge pull request #1618 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-path-0.3.0 
build(deps): bump github.com/ipfs/go-path from 0.2.2 to 0.3.0
 2022-03-28 20:22:54 +02:00
Hector Sanjuan
c5a887f579
Merge pull request #1620 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipld-format-0.3.1 
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.0 to 0.3.1
 2022-03-28 20:22:38 +02:00
Hector Sanjuan
e287bd1189 adder/single: do not re-add the same node twice 
The ipfsadder actually ends up DAG-putting some nodes several times
(i.e. non-leafs, root)... but usually one after the other. This prevents that
and avoids sending the same data multiple times over the wire (not a good
thing to 3x a small payload because of this).
 2022-03-28 20:22:03 +02:00
Hector Sanjuan
f07c1e6552 fix: BlockStream rpc: only cancel response context at the end 2022-03-28 17:55:31 +02:00
Hector Sanjuan
8908da32be ipfshttp: abort adding when context cancelled 2022-03-28 17:54:55 +02:00
Hector Sanjuan
e06c1eb8e2 adder: Do not warn about errors when there were none 2022-03-28 13:59:09 +02:00
dependabot[bot]
e41cd74e61
build(deps): bump github.com/ipfs/go-ipld-format from 0.3.0 to 0.3.1 
Bumps [github.com/ipfs/go-ipld-format](https://github.com/ipfs/go-ipld-format) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/ipfs/go-ipld-format/releases)
- [Commits](https://github.com/ipfs/go-ipld-format/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipld-format
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-28 11:06:14 +00:00