cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,756 Commits 3 Branches 70 Tags 15 MiB
d23240f6c8
Commit Graph

2756 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hector Sanjuan
d23240f6c8 CodeQL analysis: fix some security warnings 
They are related to logging user-input.
 2022-06-16 17:44:34 +02:00
Hector Sanjuan
b9a4578f0b Create codeql-analysis.yml 2022-06-16 17:44:34 +02:00
Hector Sanjuan
71bbf4ab74 Adder: update error message when doing Gets on the DAGService. 
Indicate that this is not expected and is likely a bug.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
455aa6def1 Fix #1691: Error when adding directories with many files 
Fixes #1691 by updating to the latest go-unixfs and adding a test.

The test is verified to fail on the previous go-unixfs version.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
144c66359e Test module: allow the MockDAGService to be write-only. 
The adder needs a write-only DAGService.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
e0247e4105 README: more improvements to badges 2022-06-16 17:43:30 +02:00
Hector Sanjuan
b705212ac0 Improve badges in README 2022-06-16 17:43:30 +02:00
Hector Sanjuan
bc8b65a099 Rename Cancelled to Canceled in operationtracker 
Part of fixing misspellings.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
9bd611e7c3 Error on mispellings 2022-06-16 17:43:30 +02:00
Hector Sanjuan
755cebbe0d Enable spell checking and fix spelling errors (using US locale) 2022-06-16 17:43:30 +02:00
Hector Sanjuan
6260b11e8c Use Codecov@v2 github action 2022-06-16 17:43:30 +02:00
Hector Sanjuan
6723acd333 Fix badge image link in readme 2022-06-16 17:43:30 +02:00
Hector Sanjuan
d7107791ed Rename links to use the ipfscluster.io domain 2022-06-16 17:43:30 +02:00
Hector Sanjuan
508791b547 Migrate from ipfs/ipfs-cluster to ipfs-cluster/ipfs-cluster 
This performs the necessary renamings.
 2022-06-16 17:43:30 +02:00
ipfs-mgmt-read-write[bot]
34396ddb2c Add .github/workflows/stale.yml 2022-06-16 17:43:30 +02:00
Sungyub NA
fbdeebc71e Update comment at docker-compose 
License: MIT
Signed-off-by: Sungyub NA <darkerkorean@gmail.com>
 2022-06-16 17:43:30 +02:00
@RubenKelevra
3b51cc39f4 improve language; fix typos; move breaking change under own headline 2022-06-16 17:43:30 +02:00
Hector Sanjuan
3911f8483a Release v1.0.1 2022-06-16 17:43:30 +02:00
Hector Sanjuan
5a3d36b41a Changelog for v1.0.1 2022-06-16 17:43:30 +02:00
Hector Sanjuan
90df0ba35e GetMetadata: ignore linter check 2022-06-16 17:43:29 +02:00
Hector Sanjuan
7a05eeeb60 Fix #1547: Store pin metadata in a sorted array 
This deprecates the Metadata protobuf map and starts serializing metadata as an
array that is always sorted by the metadata key.

This should resolve the issue that an state export file can be imported
resulting in two different CRDT dags because the binary representation of the
pin can arbitrarily change depending on how the keys in the map are listed,
and thus the CID of a delta is impacted.

So with this, a state export should result in exactly the same DAG regardless
of where it is imported.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
03740e5bf0 ctl: fix "pin ls" hangs for pinsets > 1024 items. 
Seems we forgot to adapt "pin ls" and run it in a background goroutine, so it
just hangs.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
447b79093c Fix #1655: rest api lbclients panic on error with streaming methods 
I think this should fix the issue. As solution we make every retry with a
temporary channel and copy results to the final channel which is only closed
by us. This only affects streaming methods.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
bc6fe29d8b Update to go1.18 2022-06-16 17:43:29 +02:00
Hector Sanjuan
72050c1e59 Stay in go1.17 2022-06-16 17:43:29 +02:00
Hector Sanjuan
cef51c2129 Revert "Test with go 1.18" 
This reverts commit 9e6428cb1fec27e56f88dbc6f1cca219b035431a.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
a559b137c1 Test with go 1.18 2022-06-16 17:43:29 +02:00
Hector Sanjuan
66bf7d44bf update to libp2p v0.19.1 and go1.18 2022-06-16 17:43:29 +02:00
Hector Sanjuan
4daece2b98 Feat: add a new "pinqueue" informer component 
This new component broadcasts metrics about the current size of the pinqueue,
which can in turn be used to inform allocations.

It has a weight_bucket_size option that serves to divide the actual size by a
given factor. This allows considering peers with similar queue sizes to have
the same weight.

Additionally, some changes have been made to the balanced allocator so that a
combination of tags, pinqueue sizes and free-spaces can be used. When
allocating by [<tag>, pinqueue, freespace], the allocator will prioritize
choosing peers with the smallest pin queue weight first, and of those with the
same weight, it will allocate based on freespace.
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
8dc18cd3d7 Metrics: add additional metrics 
* Gauge for total number of ipfs pins
* Counter for pin/add
* Counter for pin/add errors
* Counter for Block/Puts
* Counter for blocks added
* Counter for block/added size
* Counted for block/added errors
 2022-06-16 17:43:29 +02:00
dependabot[bot]
b38ea6b4a0 build(deps): bump github.com/urfave/cli from 1.22.8 to 1.22.9 
Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.8 to 1.22.9.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.8...v1.22.9)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
dependabot[bot]
74b7215482 build(deps): bump github.com/urfave/cli from 1.22.5 to 1.22.8 
Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.5 to 1.22.8.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.5...v1.22.8)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
dependabot[bot]
2d8d477278 build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.5.1 
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.0 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-16 17:43:29 +02:00
Hector Sanjuan
ec35841918
Merge pull request #1656 from ipfs/fix/0-space 
Do not issue freespace metric when free-space is 0.
 2022-05-03 19:09:31 +02:00
Hector Sanjuan
060ef812b2 Do not issue freespace metric when free-space is 0. 2022-05-03 18:39:44 +02:00
Hector Sanjuan
0b25f9ce60
Merge pull request #1651 from ipfs/fix/tracker-metrics 
Fix: tracker metrics not updated on phase change
 2022-05-03 18:23:53 +02:00
Hector Sanjuan
8252944aa8 Fix: tracker metrics not updated on phase change 2022-04-30 20:57:57 +02:00
Hector Sanjuan
d89e62117b
Merge pull request #1646 from ipfs/fix/leaks-on-ipfs-restart 
Fix: bad behaviour when adding and ipfs is down
 2022-04-29 22:57:26 +02:00
Hector Sanjuan
585eb1e8e9
Merge pull request #1647 from ipfs/fix/tracker-metrics-gauge 
pintracker: metrics: convert pinning/queued/error metrics to gauges
 2022-04-29 22:57:04 +02:00
Hector Sanjuan
bcecb8e1b4 ipfshttp: do not return io.EOF errors to the caller 2022-04-26 15:20:01 +02:00
Hector Sanjuan
de40b2cd23 pintracker: metrics: convert pinning/queued/error metrics to gauges 
We were currently tracking this metrics as a counter (SUM). The number is
good, but conceptually this is more a gauge (LastValue), given it can go down.

Thus we switch it by tracking the aggregation numbers directy in the operation
tracker.
 2022-04-26 15:13:35 +02:00
Hector Sanjuan
8626fbc166 Fix: bad behaviour when adding and ipfs is down 
Adding keeps retrying indefinitely to send blocks to ipfs. When ipfs is down this:

* Stores all errors
* Keeps retrying
* Additionally, forgot to close bodies

This resulted in memory leaks. The new behaviour does not keep retrying. And
response bodies are closed after reading.
 2022-04-26 14:54:04 +02:00
Hector Sanjuan
09814e5176
Merge pull request #1645 from ipfs/fix/metrics-batching-state 
Fix: wrong pins metric when batching enabled
 2022-04-26 14:53:13 +02:00
Hector Sanjuan
cfae287550 ipfshttp: do not touch the pins metric here 
The connector was issuing pins metrics and it is all wrong.
 2022-04-26 00:36:41 +02:00
Hector Sanjuan
6716f5471a Fix: wrong pins metric when batching enabled 
When batching is enabled, the "batchingstate" is used to add/remove pins, but the
non-batching state is used as read-only state for doing List().

This means that both states will be writing pins metrics but the batching
state is never used for List(), so it never has the right total number.

This fixes that.
 2022-04-25 23:45:34 +02:00
Hector Sanjuan
cc40a37daa Fix date in changelog 2022-04-22 16:46:56 +02:00
Hector Sanjuan
6f01277bd1
Release v1.0.0 2022-04-22 16:42:17 +02:00
Hector Sanjuan
4075789167
Merge pull request #1619 from ipfs/v1.0.0/changelog 
Changelog for v1.0.0
 2022-04-22 16:39:23 +02:00
Hector Sanjuan
4d66390283 Changelog for v1.0.0 2022-04-22 16:35:17 +02:00
Hector Sanjuan
e69e690194
Merge pull request #1637 from ipfs/pintracker-metrics 
metrics: track total pins, queued, pinning, pin error.
 2022-04-22 16:15:13 +02:00