cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,935 Commits 3 Branches 70 Tags 15 MiB
master
Commit Graph

58 Commits

Author SHA1 Message Date
Hector Sanjuan
5452b59a2e
Dependency upgrades (#1755) 
* Update go-libp2p to v0.22.0

* Testing with go1.19

* build(deps): bump github.com/multiformats/go-multicodec

Bumps [github.com/multiformats/go-multicodec](https://github.com/multiformats/go-multicodec) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/multiformats/go-multicodec/releases)
- [Commits](https://github.com/multiformats/go-multicodec/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multicodec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/ipld/go-car from 0.4.0 to 0.5.0

Bumps [github.com/ipld/go-car](https://github.com/ipld/go-car) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/ipld/go-car/releases)
- [Commits](https://github.com/ipld/go-car/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/ipld/go-car
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.12.2 to 1.13.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.2...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/hashicorp/go-hclog from 1.2.1 to 1.3.0

Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/hashicorp/go-hclog/releases)
- [Commits](https://github.com/hashicorp/go-hclog/compare/v1.2.1...v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/ipfs/go-ds-crdt from 0.3.6 to 0.3.7

Bumps [github.com/ipfs/go-ds-crdt](https://github.com/ipfs/go-ds-crdt) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/ipfs/go-ds-crdt/releases)
- [Commits](https://github.com/ipfs/go-ds-crdt/compare/v0.3.6...v0.3.7)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ds-crdt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.10.2 to 2.14.1

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.2 to 2.14.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.2...v2.14.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-http from 0.3.0 to 0.4.0

Bumps [github.com/libp2p/go-libp2p-http](https://github.com/libp2p/go-libp2p-http) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/libp2p/go-libp2p-http/releases)
- [Commits](https://github.com/libp2p/go-libp2p-http/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-http
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-gorpc from 0.4.0 to 0.5.0

Bumps [github.com/libp2p/go-libp2p-gorpc](https://github.com/libp2p/go-libp2p-gorpc) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/libp2p/go-libp2p-gorpc/releases)
- [Commits](https://github.com/libp2p/go-libp2p-gorpc/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-gorpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump contrib.go.opencensus.io/exporter/prometheus

Bumps [contrib.go.opencensus.io/exporter/prometheus](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/releases)
- [Commits](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/compare/v0.4.1...v0.4.2)

---
updated-dependencies:
- dependency-name: contrib.go.opencensus.io/exporter/prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/libp2p/go-libp2p-raft from 0.1.8 to 0.2.0

Bumps [github.com/libp2p/go-libp2p-raft](https://github.com/libp2p/go-libp2p-raft) from 0.1.8 to 0.2.0.
- [Release notes](https://github.com/libp2p/go-libp2p-raft/releases)
- [Commits](https://github.com/libp2p/go-libp2p-raft/compare/v0.1.8...v0.2.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-raft
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump github.com/urfave/cli from 1.22.9 to 1.22.10

Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.9 to 1.22.10.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.9...v1.22.10)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix checker/linter/staticcheck warnings

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-06 16:57:17 +02:00
Hector Sanjuan
755cebbe0d Enable spell checking and fix spelling errors (using US locale) 2022-06-16 17:43:30 +02:00
Hector Sanjuan
508791b547 Migrate from ipfs/ipfs-cluster to ipfs-cluster/ipfs-cluster 
This performs the necessary renamings.
 2022-06-16 17:43:30 +02:00
Hector Sanjuan
3169fba9d1 metrics: track total pins, queued, pinning, pin error. 
This fixes #1470 and #1187.
 2022-04-22 15:57:48 +02:00
Hector Sanjuan
9b9d76f92d Pinset streaming and method type revamp 
This commit introduces the new go-libp2p-gorpc streaming capabilities for
Cluster. The main aim is to work towards heavily reducing memory usage when
working with very large pinsets.

As a side-effect, it takes the chance to revampt all types for all public
methods so that pointers to static what should be static objects are not used
anymore. This should heavily reduce heap allocations and GC activity.

The main change is that state.List now returns a channel from which to read
the pins, rather than pins being all loaded into a huge slice.

Things reading pins have been all updated to iterate on the channel rather
than on the slice. The full pinset is no longer fully loaded onto memory for
things that run regularly like StateSync().

Additionally, the /allocations endpoint of the rest API no longer returns an
array of pins, but rather streams json-encoded pin objects directly. This
change has extended to the restapi client (which puts pins into a channel as
they arrive) and to ipfs-cluster-ctl.

There are still pending improvements like StatusAll() calls which should also
stream responses, and specially BlockPut calls which should stream blocks
directly into IPFS on a single call.

These are coming up in future commits.
 2022-03-19 03:02:55 +01:00
Hector Sanjuan
49825d1df7 monitor: Remove Distribution() method and test. 
Follow up to #939. Fixes #1586.
 2022-02-28 17:47:34 +01:00
Hector Sanjuan
000dccc1cc Monitor: do not clean up metrics immediately after an alert 2022-01-31 17:53:09 +01:00
Hector Sanjuan
d4591b8442 Monitor: remove accrual detection. Add LatestForPeer method. 
Fixes #939
 2022-01-31 17:53:09 +01:00
Hector Sanjuan
12756e0220 monitor: fix panic in checker 
Alert might be launched when no metrics for peer are received at all.
 2021-01-14 00:04:40 +01:00
Hector Sanjuan
90208b45f9 health/alerts endpoint: brush up old PR 2021-01-13 22:09:21 +01:00
Hector Sanjuan
4bcb91ee2b Merge branch 'master' into feat/alerts 2021-01-13 21:08:49 +01:00
Hector Sanjuan
0dfa9ca185
(chore) Upgrade dependencies 
Upgrade dependencies and bump to go1.15.
 2020-08-27 14:10:58 +02:00
Hector Sanjuan
b513ec194d Fix some mispellings 2020-04-14 23:47:09 +02:00
Hector Sanjuan
f83ff9b655 staticcheck: fix all staticcheck warnings in the project 2020-04-14 20:16:10 +02:00
Hector Sanjuan
b3853caf36 Dependency ugprade: changes needed 
* Libp2p protectors no longer needed, use PSK directly
* Generate cluster 32-byte secret here (helper gone from pnet)
* Switch to go-log/v2 in all places
* DHT bootstrapping not needed. Adjust DHT options for tests.
* Do not rely on dissappeared CidToDsKey and DsKeyToCid functions fro dshelp.
* Disable QUIC (does not support private networks)
* Fix tests: autodiscovery started working properly
 2020-03-22 14:50:25 +01:00
Kishan Mohanbhai Sagathiya
618ebd23f4 Check expiry in alert 2019-12-13 12:25:28 +05:30
Kishan Sagathiya
31534a429b Fix #374: health metrics improvements 
- Human-sizes for freespace metrics. Display whether if metric is
expires in something like "expires in 3m".
- When not passing metric name `ipfs-cluster-ctl health metrics` hits
the the metrics endpoint which returns a list of available metrics and
displays to user
- Humanize metrics output
- Sort metrics output
 2019-10-24 16:37:26 +02:00
Kishan Mohanbhai Sagathiya
76857112b2 Test that expired PeerMetrics gets deleted 2019-09-13 08:01:15 +07:00
Hector Sanjuan
e240c2a19f Simplify failed peer detection 2019-06-27 16:55:51 +01:00
Adrian Lanzafame
2255ba737b
fix ttl expiration check 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-25 12:54:41 +02:00
Hector Sanjuan
563a0da9ae Do alert for all metric types 2019-06-23 10:14:29 +01:00
Adrian Lanzafame
27295c10ac fix check failed 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-23 10:14:29 +01:00
Adrian Lanzafame
5e09da9d63 address pr feedback 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-23 10:14:29 +01:00
Adrian Lanzafame
e1b40d49c1 fix how accrual fd treats ttls 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-23 10:14:29 +01:00
Hector Sanjuan
b804e61ef0 Update deps along with go-libp2p-core refactor 
Lots of rewrites in imports...
 2019-06-14 13:10:45 +02:00
Hector Sanjuan
27368ab077 Fix: alert at most once PER METRIC 
Before it would alert at most once per peer, which prevented some metrics
from alerting at all.
 2019-06-11 11:44:12 +02:00
Hector Sanjuan
a0d93fc62c Change MaxAlertThreshold to 1 2019-06-11 10:54:12 +02:00
Adrian Lanzafame
14841e4e24 address pr feedback 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-11 10:54:12 +02:00
Adrian Lanzafame
7459917275 alerting for peers stops after one alert 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-06-11 10:54:12 +02:00
Adrian Lanzafame
a763560e0c
extend the initial size of metrics distribution to 5 
This prevents accrual failure detection from kicking in too
soon after a cluster has started.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-05-07 19:07:11 +10:00
Adrian Lanzafame
9464759ae6
remove hard timeout limits and use only accrual failure detection 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-30 12:06:01 +10:00
Adrian Lanzafame
42693eb06d
fix passing ctx from daemon to pubsub 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-29 17:58:28 +10:00
Adrian Lanzafame
32ca9167d6
use accrual instead of metric expiration 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-26 17:58:29 +10:00
Adrian Lanzafame
3c09ebcc71
add Alerts measure 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-26 17:56:44 +10:00
Adrian Lanzafame
b0dbcbaa8d
add reference to original prob.go 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-26 12:20:31 +10:00
Adrian Lanzafame
d5ecd9ef6a
WIP 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-23 20:30:26 +10:00
Adrian Lanzafame
eae4329cb3
address pr feedback 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:18:19 +10:00
Adrian Lanzafame
31af640e33
use allocations list to choose peer to repin 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:16:40 +10:00
Adrian Lanzafame
1349e99c1e
fix time taken by tests 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:16:39 +10:00
Adrian Lanzafame
4338ea6905
refactor prob to use gonum and pass []float64 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:16:39 +10:00
Adrian Lanzafame
bcbe7b453f
refactor from big.Float to float64 and add prob tests 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:14:13 +10:00
Adrian Lanzafame
e187b800cf
rename TS to ReceivedAt 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:14:13 +10:00
Adrian Lanzafame
3d6eb64db6
Add accrual failure detection method 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:14:13 +10:00
Adrian Lanzafame
13ed78786c
fix distribution test and general clean up 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:09:19 +10:00
Hector Sanjuan
4e61935379
Use defer for locks. Move to Prev() in All() 
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
 2019-04-18 16:09:19 +10:00
Hector Sanjuan
da3c543ce2
Revert "attempt copying slice" 
This reverts commit 0d4d40513fccd31b9cdc4db369aa87e87c529be4.
 2019-04-18 16:09:19 +10:00
Adrian Lanzafame
46d6cb155d
attempt copying slice 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:09:19 +10:00
Adrian Lanzafame
2b1b8a4389
remove use of last 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:09:18 +10:00
Adrian Lanzafame
ebcf40cf7d
rename TS to ReceivedAt 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:09:18 +10:00
Hector Sanjuan
7711ab8cfd
Replace underlying slice with ring.Ring in metrics window 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-04-18 16:09:18 +10:00