cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,597 Commits 3 Branches 70 Tags 15 MiB
d0e905babe
Commit Graph

2597 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hector Sanjuan
d0e905babe Fix critical log level 
The CRITICAL log level no longer exists and we were logging more than we
wanted to.
 2022-02-15 19:38:33 +01:00
Hector Sanjuan
3ffaf0fcc7
Merge pull request #1579 from ipfs/gorpc-upgrade 
go-libp2p-gorpc upgrade
 2022-02-15 19:33:43 +01:00
Hector Sanjuan
4c40ab5547
Merge pull request #1580 from ipfs/dependency-upgrades 
Dependency upgrades
 2022-02-15 19:33:14 +01:00
Hector Sanjuan
fa36958221 Merge branch 'master' into dependency-upgrades 2022-02-15 19:14:50 +01:00
Hector Sanjuan
9a0bca222b go-libp2p-gorpc upgrade 2022-02-15 19:12:12 +01:00
Hector Sanjuan
dc2b82e772
Merge pull request #1565 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-files-0.1.1 
Bump github.com/ipfs/go-ipfs-files from 0.1.0 to 0.1.1
 2022-02-15 19:11:58 +01:00
Hector Sanjuan
5fb851babf
Merge pull request #1577 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ds-crdt-0.3.1 
Bump github.com/ipfs/go-ds-crdt from 0.3.0 to 0.3.1
 2022-02-15 19:11:47 +01:00
Hector Sanjuan
90e67eb80d
Merge pull request #1573 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-path-0.2.2 
Bump github.com/ipfs/go-path from 0.2.1 to 0.2.2
 2022-02-15 19:11:37 +01:00
dependabot[bot]
a1841a3162
Bump github.com/ipfs/go-ds-crdt from 0.3.0 to 0.3.1 
Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-15 16:41:12 +00:00
Hector Sanjuan
913339e7d5
Merge pull request #1576 from ipfs/update-go-ds-crdt-3 
crdt: increase number of workers from 5 to 50. Upgrade.
 2022-02-15 17:39:04 +01:00
Hector Sanjuan
975814229c crdt: increase number of workers from 5 to 50. Upgrade. 
The go-ds-crdt upgrade disables multi-head-processing by default again. We see
this causes a lot of branching.

We however increase the number of workers. With large deltas, it may be
possible that all the 5 workers are busy downloading a delta or processing
them, while we potentially have hundreds of children in the DAG. Thus it is
not bad to attempt to do more things in parallel.
 2022-02-15 16:54:42 +01:00
Hector Sanjuan
168b0caa86
Merge pull request #1575 from ipfs/fix/1569-leave-peername-empty 
Fix #1569: Leave peername empty when unknown
 2022-02-14 20:00:28 +01:00
Hector Sanjuan
05163a5706 Fix #1569: Leave peername empty when unknown 2022-02-14 18:37:58 +01:00
Hector Sanjuan
6edc04336a
Merge pull request #1574 from ipfs/fix/1570-bad-graph-comment 
Fix #1570: fix comment description about graph nodes' color
 2022-02-14 14:56:28 +01:00
Hector Sanjuan
2794bd89e8 Fix #1570: fix comment description about graph nodes' color 2022-02-14 14:54:24 +01:00
dependabot[bot]
a9250966a2
Bump github.com/ipfs/go-path from 0.2.1 to 0.2.2 
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/ipfs/go-path/releases)
- [Commits](https://github.com/ipfs/go-path/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 11:09:39 +00:00
dependabot[bot]
55e9ab622c
Bump github.com/ipfs/go-ipfs-files from 0.1.0 to 0.1.1 
Bumps [github.com/ipfs/go-ipfs-files](https://github.com/ipfs/go-ipfs-files) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/ipfs/go-ipfs-files/releases)
- [Commits](https://github.com/ipfs/go-ipfs-files/compare/v0.1.0...v0.1.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-07 11:13:37 +00:00
Hector Sanjuan
670f9ac7d9
Merge pull request #1550 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-files-0.1.0 
Bump github.com/ipfs/go-ipfs-files from 0.0.9 to 0.1.0
 2022-02-02 01:29:29 +01:00
Hector Sanjuan
5f746f870b
Release candidate v0.14.5-rc1 2022-02-02 01:08:22 +01:00
Hector Sanjuan
92e5eaa58e
Merge pull request #1557 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/prometheus/client_golang-1.12.1 
Bump github.com/prometheus/client_golang from 1.11.0 to 1.12.1
 2022-02-02 01:07:43 +01:00
Hector Sanjuan
fbc45b17e4
Merge pull request #1558 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ipfs-config-0.19.0 
Bump github.com/ipfs/go-ipfs-config from 0.18.0 to 0.19.0
 2022-02-02 01:07:25 +01:00
Hector Sanjuan
18ea2d82c9
Merge pull request #1563 from ipfs/dependabot/go_modules/dependency-upgrades/github.com/ipfs/go-ds-crdt-0.3.0 
Bump github.com/ipfs/go-ds-crdt from 0.2.4 to 0.3.0
 2022-02-02 01:07:15 +01:00
Hector Sanjuan
5221afb9b2
Merge pull request #1562 from ipfs/feat/filter-cids 
Restapi: add "cids" query param to /pins
 2022-02-02 00:57:43 +01:00
dependabot[bot]
8786588ac1
Bump github.com/ipfs/go-ds-crdt from 0.2.4 to 0.3.0 
Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.2.4 to 0.3.0.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.2.4...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-01 23:41:52 +00:00
Hector Sanjuan
e7813e249c
Merge pull request #1561 from ipfs/update-go-ds-crdt-2 
Update go-ds-crdt and add RepairInterval option
 2022-02-02 00:41:01 +01:00
Hector Sanjuan
223b54cab6 Restapi: add "cids" query param to /pins 
This allows to specifically request status for several CIDs as
provided in the "cids" query parameter, instead of request status for
all CIDs.

In this case, the filter is ignored.
 2022-02-02 00:39:09 +01:00
Hector Sanjuan
a64a1c2b6e crdt: add RepairInterval option 2022-02-01 23:29:40 +01:00
Hector Sanjuan
881266a1a0 Update go-ds-crdt 
The new version supports auto-repair of the datastore.

Fixes #1007.
 2022-02-01 23:16:50 +01:00
Hector Sanjuan
67ab6aa0f8
Merge pull request #1560 from ipfs/feat/local-allocate 
Adder: adding with --local forcefully allocates the local peer to the pin
 2022-02-01 22:43:02 +01:00
Hector Sanjuan
7e389d551c Add test about local allocation 2022-02-01 22:29:54 +01:00
Hector Sanjuan
07b4971277 Adder: adding with --local forcefully allocates the local peer to the pin 
It is not good to add something locally only to pin it somewhere else:
  * The locally used space is not GCed automatically or anything and is lost
  * Pay the penalty of having to copy things somewhere else
 2022-02-01 20:38:41 +01:00
Hector Sanjuan
7071f6777b
Merge pull request #1543 from ipfs/fix/warning-msg 
Remove warning message about informer metrics
 2022-02-01 14:37:00 +01:00
Hector Sanjuan
2c204968b8
Merge pull request #1559 from ipfs/fix/repo-stat-hammering 
Fix: repo/stat gets hammered on busy cluster peers
 2022-02-01 14:36:27 +01:00
Hector Sanjuan
acde3f16d0 Fix: repo/stat gets hammered on busy cluster peers 
Given that every pin and block/put writes something to IPFS and thus increases
the repo size, a while ago we added a check to let the IPFS connector directly
trigger the sending of metrics every 10 of such requests. This was meant to
update the metrics more often so that balancing happened more granularly
(particularly the freespace one).

In practice, on a cluster that receives several hundreds of pin/adds
operations in a few seconds, this is just bad.

So:

* We disable by default the whole thing.
* We add a new InformerTriggerInterval configuration option to enable the thing.
* Fix a bug that made this always call the first informer, which may not
  have been the freespace one).
 2022-02-01 01:34:17 +01:00
Hector Sanjuan
20f49c1b51
Merge pull request #1556 from ipfs/fix/1554-tracker-status-ipfs-node-id 
Pintracker improvements
 2022-02-01 01:31:08 +01:00
Hector Sanjuan
60c6b16ac6 pintracker: Remove unnecessary locking 2022-01-31 21:04:11 +01:00
Hector Sanjuan
5e89c0ba41 Pintracker: set Name in operation tracker. Fixes #1212. 2022-01-31 21:04:11 +01:00
Hector Sanjuan
809b7fbda5 Pintracker: add IPFS ID to Pin Information 
Fixes #1554
Fixes: peer names unset for remote peers

This adds an IPFS field to pin status information (PinInfoShort).

It has not been easy to add this, given that the IPFS ID is something that
comes from outside of cluster (unlike the peer name). After several tries I
have settled in the following things:

- Use the ping metric to send out peer names and IPFS IDs to the peers in the
  cluster.
- Cache the latest known IPFS ID (if IPFS dies we should still be setting
  the ID).
- Provide an RPC method for the Pintracker to obtain IPFS ID from the cache.
- Given we now know information for peernames and IPFS IDs from other peers,
  we can use that information even if the requests to them error or we are not
  contacting (i.e. peers allocated as remote are not queried for status). We can
  use the information from the last received ping metric.
- This means we should keep metrics around even if peers go away, at least for
  a while rather than deleting them as soon as we detect that they have expired.

Puting it all together we now have a system to gossip peer information around on top
of the ping metrics.
 2022-01-31 17:53:09 +01:00
Hector Sanjuan
000dccc1cc Monitor: do not clean up metrics immediately after an alert 2022-01-31 17:53:09 +01:00
Hector Sanjuan
d4591b8442 Monitor: remove accrual detection. Add LatestForPeer method. 
Fixes #939
 2022-01-31 17:53:09 +01:00
dependabot[bot]
c331026ca5
Bump github.com/ipfs/go-ipfs-config from 0.18.0 to 0.19.0 
Bumps [github.com/ipfs/go-ipfs-config](https://github.com/ipfs/go-ipfs-config) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/ipfs/go-ipfs-config/releases)
- [Commits](https://github.com/ipfs/go-ipfs-config/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-config
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-31 11:07:15 +00:00
dependabot[bot]
36f85a96f8
Bump github.com/prometheus/client_golang from 1.11.0 to 1.12.1 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.11.0 to 1.12.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.11.0...v1.12.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-31 11:06:44 +00:00
Hector Sanjuan
ed348f29c1
state: abort list operation on context cancellation (#1555) 
Stopping cluster while listing pins seems to hang until the operation has finished.
 2022-01-29 00:06:08 +01:00
dependabot[bot]
fa7d7ded12
Bump github.com/ipfs/go-ipfs-files from 0.0.9 to 0.1.0 
Bumps [github.com/ipfs/go-ipfs-files](https://github.com/ipfs/go-ipfs-files) from 0.0.9 to 0.1.0.
- [Release notes](https://github.com/ipfs/go-ipfs-files/releases)
- [Commits](https://github.com/ipfs/go-ipfs-files/compare/v0.0.9...v0.1.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-24 11:06:22 +00:00
Hector Sanjuan
4b0983b75c Remove warning message about informer metrics 2022-01-15 14:26:39 +01:00
Hector Sanjuan
dca8923aaa
Set development version v0.14.4-next 2022-01-11 19:13:39 +01:00
Hector Sanjuan
d80a8b42c8
Release v0.14.4 2022-01-11 19:13:28 +01:00
Hector Sanjuan
4be8c55875
Changelog for v0.14.4 (#1542) 
* Changelog for v0.14.4
 2022-01-11 19:13:12 +01:00
Hector Sanjuan
00fd9066b2
Merge pull request #1541 from ipfs/update-go-ds-crdt 
Update go-ds-crdt
 2022-01-11 18:19:59 +01:00
Hector Sanjuan
f2ccf7a772 Update go-ds-crdt 2022-01-11 18:08:20 +01:00