Hector Sanjuan
29c277b67f
Pintracker: add and track retry counts in the operation manager.
...
Report retry count in the PinStatus
2021-11-30 04:20:35 +01:00
Hector Sanjuan
be18c645fa
Merge pull request #1491 from ipfs/dependency-upgrades
...
Dependency upgrades
2021-10-27 18:51:04 +02:00
Hector Sanjuan
555b624beb
Dependency upgrades
2021-10-27 16:03:59 +02:00
Hector Sanjuan
3ddda1fb59
Merge branch 'master' into dependency-upgrades
2021-10-27 15:55:34 +02:00
Hector Sanjuan
a74b6faf96
Merge pull request #1484 from ipfs/feat/pin-time
...
Add: include a timestamp that tracks when a Pin was added to the state
2021-10-27 10:42:59 +02:00
Hector Sanjuan
f04afda51f
Merge pull request #1485 from ipfs/fix/default-allocator-compat
...
Fix: backwards compatibility of the new allocator
2021-10-27 10:38:05 +02:00
Hector Sanjuan
24d8b59609
Fix config test
2021-10-20 17:12:21 +02:00
Hector Sanjuan
53342d3c55
Fix go-multiaddr-net
2021-10-20 16:56:24 +02:00
Hector Sanjuan
32386d853a
Dependency upgrades
2021-10-20 16:56:24 +02:00
dependabot[bot]
067db00d7a
Bump github.com/multiformats/go-multihash from 0.0.15 to 0.0.16
...
Bumps [github.com/multiformats/go-multihash](https://github.com/multiformats/go-multihash ) from 0.0.15 to 0.0.16.
- [Release notes](https://github.com/multiformats/go-multihash/releases )
- [Commits](https://github.com/multiformats/go-multihash/compare/v0.0.15...v0.0.16 )
---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multihash
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 16:56:24 +02:00
dependabot[bot]
8ce4b55607
Bump github.com/ipfs/go-path from 0.1.0 to 0.1.1
...
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path ) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/ipfs/go-path/releases )
- [Commits](https://github.com/ipfs/go-path/compare/v0.1.0...v0.1.1 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 16:56:24 +02:00
dependabot[bot]
7ece0473fa
Bump github.com/libp2p/go-libp2p-quic-transport from 0.11.2 to 0.12.0
...
Bumps [github.com/libp2p/go-libp2p-quic-transport](https://github.com/libp2p/go-libp2p-quic-transport ) from 0.11.2 to 0.12.0.
- [Release notes](https://github.com/libp2p/go-libp2p-quic-transport/releases )
- [Commits](https://github.com/libp2p/go-libp2p-quic-transport/compare/v0.11.2...v0.12.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-quic-transport
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 16:56:24 +02:00
dependabot[bot]
e4a197796c
Bump github.com/ipfs/go-ipfs-config from 0.15.0 to 0.16.0
...
Bumps [github.com/ipfs/go-ipfs-config](https://github.com/ipfs/go-ipfs-config ) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/ipfs/go-ipfs-config/releases )
- [Commits](https://github.com/ipfs/go-ipfs-config/compare/v0.15.0...v0.16.0 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-config
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 16:56:24 +02:00
dependabot[bot]
42c363bbc2
Bump github.com/ipfs/go-cid from 0.0.7 to 0.1.0
...
Bumps [github.com/ipfs/go-cid](https://github.com/ipfs/go-cid ) from 0.0.7 to 0.1.0.
- [Release notes](https://github.com/ipfs/go-cid/releases )
- [Commits](https://github.com/ipfs/go-cid/compare/v0.0.7...v0.1.0 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-cid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 16:56:24 +02:00
Hector Sanjuan
c776051397
ctl: Include "added" column in pin ls command
...
Also, change format from humanize to "yyyy-mm-dd HH:MM:SS" for times.
2021-10-20 16:55:57 +02:00
Hector Sanjuan
e9857652f2
Add a timestamp to Pins
...
This adds a Timestamp field to the pin objects. This allows to track when they were pinned.
This:
* Allows the pin-tracker to actually show accurate information on when the pin
entered the system for pins that are not part of ongoing operations
(currently it shows time.Now())
* Adds support for reporting timestamp on a pinning services api.
2021-10-20 16:55:57 +02:00
Hector Sanjuan
3f895df8cf
Fix: backwards compatibility for allocator default settings
...
When the allocator is not defined in the configuration, it will take defaults
and assume there is a "tags" informer. That is not the case. When not defined,
we assume it should allocate only by "freespace".
2021-10-20 16:55:35 +02:00
Hector Sanjuan
1dca6e0eca
allocator: balanced allocator name should be "balanced"
...
It was an oversight to not rename this.
2021-10-20 16:55:35 +02:00
Hector Sanjuan
a741e3067d
Merge pull request #1486 from ipfs/feat/github-actions
...
Move testing pipeline to github actions.
2021-10-20 16:54:08 +02:00
Hector Sanjuan
066b1f4514
Move testing pipeline to github actions.
...
Since travis cannot stop sucking.
2021-10-20 16:52:46 +02:00
Hector Sanjuan
af9313d4fe
Remove test files that should have not been committed
2021-10-19 13:51:17 +02:00
Hector Sanjuan
96db605c50
Merge pull request #1468 from ipfs/fix/159-improved-allocators
...
Add tags informer and enable partition-based peer allocations for intelligent distribution
2021-10-06 14:35:16 +02:00
Hector Sanjuan
db00d651bf
Balanced allocator: weight-based ordering of partitions
...
This fixes the issue about partitions not being picked based
on the amount of freespace available in them.
It additionally removes the metrics registry and carries information directly
in the metric.
Metrics have two additional fields: Weight and Partitionable.
Informers have been updated to make use of these fields. Partitions have
weights that equals to the weight of the metrics under them.
Older cluster versions will not set these fields. Partitionable is false by
default and weight has a GetWeight() function to convert value->weight when
unset. This provides backwards compatibility for the freespace metric.
2021-10-06 14:10:06 +02:00
Hector Sanjuan
26e229df94
Rename allocator/metrics to allocator/balanced
2021-10-06 11:26:38 +02:00
Hector Sanjuan
6b31f44351
Address most comments from PR review
2021-10-05 14:04:28 +02:00
Hector Sanjuan
0dfe1ae063
Merge pull request #1471 from ipfs/feat/refactor-api
...
API: Refactor REST API to enable re-use of functionality.
2021-09-16 16:10:20 +02:00
Hector Sanjuan
63972f2b2e
API: Refactor REST API. Extract all functionality.
...
This is a preparatory PR to add additional APIs (Pinning Service API) easily
to cluster.
Instead of copy-pasting most of what the REST API does, I have refactored so
that the whole configuration, routing and request-handling utilities can be
re-used.
The worst part has been to divide the test between tests that test core
(common.API) functionality and tests that test specific REST API endpoint
functionality. I could not get away without an additional common/test package
to provide functions that are used from both places. This is a side effect of
testing both http and libp2p endpoints for every request etc.
2021-09-16 15:52:25 +02:00
Hector Sanjuan
cf4fb74993
service/follow: Enable new metrics allocator
2021-09-15 21:49:26 +02:00
Hector Sanjuan
1c0abde8a5
Informer: add tags informer
...
The tags informer produces metrics in the form tags:name/value
with the name and values defined in its configuration.
2021-09-15 20:07:37 +02:00
Hector Sanjuan
ea5e18078c
Informers: GetMetric() -> GetMetrics()
...
Support returning multiple metrics per informer.
2021-09-15 20:07:37 +02:00
Hector Sanjuan
b6a46cd8a4
allocator: rework the whole allocator system
...
The new "metrics" allocator is about to partition metrics and distribe
allocations among the partitions.
For example: given a region, an availability zone and free space on disk, the
allocator would be able to choose allocations by distributing among regions
and availability zones as much as possible, and for those peers in the same
region/az, selecting those with most free space first.
This requires a major overhaul of the allocator component.
2021-09-13 12:24:00 +02:00
Hector Sanjuan
4060f4196b
Revert "Informer/disk: deprecate RepoSize metric"
...
This reverts commit 3e54c4c695
.
2021-09-10 18:53:59 +02:00
Hector Sanjuan
3e54c4c695
Informer/disk: deprecate RepoSize metric
...
The allocator is hardcoded to descendalloc for freespace so it is not even useful
as it would allocate to peers with largest reposize first no matter what.
We are, in any case, reworking allocators and informers etc.
2021-09-08 17:36:54 +02:00
Hector Sanjuan
396a348a65
Merge pull request #1462 from ipfs/fix/1461-flaky-test
...
Adder: try to make test less flaky
2021-08-30 12:04:43 +02:00
Hector Sanjuan
26007114b4
Adder: try to make test less flaky
2021-08-29 01:16:28 +02:00
Hector Sanjuan
f9b20caa2a
changelog: fix v0.14.1 release date
2021-08-17 10:22:36 +02:00
Hector Sanjuan
267deeff28
Set development version v0.14.1-next
2021-08-16 18:03:18 +02:00
Hector Sanjuan
0162df60eb
Release v0.14.1
2021-08-16 18:02:52 +02:00
Hector Sanjuan
506b1657b9
Merge pull request #1452 from ipfs/v0.14.1/changelog
...
Changelog for v0.14.1
2021-08-16 17:54:52 +02:00
Hector Sanjuan
3f5490424b
Dependency upgrades ( #1451 )
...
* Bump github.com/hashicorp/go-hclog from 0.16.1 to 0.16.2
Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog ) from 0.16.1 to 0.16.2.
- [Release notes](https://github.com/hashicorp/go-hclog/releases )
- [Commits](https://github.com/hashicorp/go-hclog/compare/v0.16.1...v0.16.2 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-log/v2 from 2.2.0 to 2.3.0
Bumps [github.com/ipfs/go-log/v2](https://github.com/ipfs/go-log ) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/ipfs/go-log/releases )
- [Commits](https://github.com/ipfs/go-log/compare/v2.2.0...v2.3.0 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-log/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p from 0.14.3 to 0.14.4
Bumps [github.com/libp2p/go-libp2p](https://github.com/libp2p/go-libp2p ) from 0.14.3 to 0.14.4.
- [Release notes](https://github.com/libp2p/go-libp2p/releases )
- [Commits](https://github.com/libp2p/go-libp2p/compare/v0.14.3...v0.14.4 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-quic-transport from 0.11.1 to 0.11.2
Bumps [github.com/libp2p/go-libp2p-quic-transport](https://github.com/libp2p/go-libp2p-quic-transport ) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/libp2p/go-libp2p-quic-transport/releases )
- [Commits](https://github.com/libp2p/go-libp2p-quic-transport/compare/v0.11.1...v0.11.2 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-quic-transport
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/google/uuid from 1.2.0 to 1.3.0
Bumps [github.com/google/uuid](https://github.com/google/uuid ) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/google/uuid/releases )
- [Commits](https://github.com/google/uuid/compare/v1.2.0...v1.3.0 )
---
updated-dependencies:
- dependency-name: github.com/google/uuid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-peerstore from 0.2.7 to 0.2.8
Bumps [github.com/libp2p/go-libp2p-peerstore](https://github.com/libp2p/go-libp2p-peerstore ) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/libp2p/go-libp2p-peerstore/releases )
- [Commits](https://github.com/libp2p/go-libp2p-peerstore/compare/v0.2.7...v0.2.8 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-peerstore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-datastore from 0.4.5 to 0.4.6
Bumps [github.com/ipfs/go-datastore](https://github.com/ipfs/go-datastore ) from 0.4.5 to 0.4.6.
- [Release notes](https://github.com/ipfs/go-datastore/releases )
- [Commits](https://github.com/ipfs/go-datastore/compare/v0.4.5...v0.4.6 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-datastore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-fs-lock from 0.0.6 to 0.0.7
Bumps [github.com/ipfs/go-fs-lock](https://github.com/ipfs/go-fs-lock ) from 0.0.6 to 0.0.7.
- [Release notes](https://github.com/ipfs/go-fs-lock/releases )
- [Commits](https://github.com/ipfs/go-fs-lock/compare/v0.0.6...v0.0.7 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-fs-lock
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-ipns from 0.1.0 to 0.1.2
Bumps [github.com/ipfs/go-ipns](https://github.com/ipfs/go-ipns ) from 0.1.0 to 0.1.2.
- [Release notes](https://github.com/ipfs/go-ipns/releases )
- [Commits](https://github.com/ipfs/go-ipns/compare/v0.1.0...v0.1.2 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipns
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-noise from 0.2.0 to 0.2.2
Bumps [github.com/libp2p/go-libp2p-noise](https://github.com/libp2p/go-libp2p-noise ) from 0.2.0 to 0.2.2.
- [Release notes](https://github.com/libp2p/go-libp2p-noise/releases )
- [Commits](https://github.com/libp2p/go-libp2p-noise/compare/v0.2.0...v0.2.2 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-noise
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-pubsub from 0.4.1 to 0.5.4
Bumps [github.com/libp2p/go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub ) from 0.4.1 to 0.5.4.
- [Release notes](https://github.com/libp2p/go-libp2p-pubsub/releases )
- [Commits](https://github.com/libp2p/go-libp2p-pubsub/compare/v0.4.1...v0.5.4 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-pubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update go-ipfs-pinner
* Bump github.com/libp2p/go-libp2p-kad-dht from 0.12.2 to 0.13.0
Bumps [github.com/libp2p/go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht ) from 0.12.2 to 0.13.0.
- [Release notes](https://github.com/libp2p/go-libp2p-kad-dht/releases )
- [Commits](https://github.com/libp2p/go-libp2p-kad-dht/compare/v0.12.2...v0.13.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-kad-dht
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-path from 0.0.9 to 0.1.0
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path ) from 0.0.9 to 0.1.0.
- [Release notes](https://github.com/ipfs/go-path/releases )
- [Commits](https://github.com/ipfs/go-path/compare/v0.0.9...v0.1.0 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-raft from 0.1.7 to 0.1.8
Bumps [github.com/libp2p/go-libp2p-raft](https://github.com/libp2p/go-libp2p-raft ) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/libp2p/go-libp2p-raft/releases )
- [Commits](https://github.com/libp2p/go-libp2p-raft/compare/v0.1.7...v0.1.8 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-raft
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/libp2p/go-libp2p-tls from 0.1.3 to 0.2.0
Bumps [github.com/libp2p/go-libp2p-tls](https://github.com/libp2p/go-libp2p-tls ) from 0.1.3 to 0.2.0.
- [Release notes](https://github.com/libp2p/go-libp2p-tls/releases )
- [Commits](https://github.com/libp2p/go-libp2p-tls/compare/v0.1.3...v0.2.0 )
---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-tls
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/ipfs/go-merkledag from 0.3.2 to 0.4.0
Bumps [github.com/ipfs/go-merkledag](https://github.com/ipfs/go-merkledag ) from 0.3.2 to 0.4.0.
- [Release notes](https://github.com/ipfs/go-merkledag/releases )
- [Commits](https://github.com/ipfs/go-merkledag/compare/v0.3.2...v0.4.0 )
---
updated-dependencies:
- dependency-name: github.com/ipfs/go-merkledag
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Chore: dependency upgrades
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-16 17:52:31 +02:00
Hector Sanjuan
263c17c383
Changelog for v0.14.1
2021-08-16 17:51:57 +02:00
Hector Sanjuan
9fb80b3acb
Merge pull request #1444 from ipfs/fix/1427-wait-pin
...
Change --wait behaviour to return on replication-factor-min
2021-08-16 16:38:17 +02:00
Hector Sanjuan
3826620b81
Merge pull request #1446 from ipfs/fix/1409-msgpack-decode-error
...
Fix #1409 : Better support of metrics from older peers
2021-08-16 16:36:20 +02:00
Hector Sanjuan
e2659af8f6
ctl: WaitFor replication factor min to be reached
...
Fixes #1427 . Currently, if --wait is used when pinning it will wait until all
statuses reported for a pin are either Pinned or Remote. If a peer was lagging
behind and not syncing the state properly (reporting "unpinned" for example),
that would be enough to block waiting.
This modifies the behaviour of wait to return when replication_factor_min is
reached, regardless of what other statuses are.
2021-08-13 11:35:28 +02:00
Hector Sanjuan
c7847e9677
rest/client: WaitFor(): support waiting for N peers to reach target.
...
This commit updates the WaitFor StatusFilterParams to add a Limit field, and
reworks the waiting logic to simplify it.
The Limit parameter causes WaitFor to return successfully when N number of
peers have reached the target status, regardless of what other statuses are
(as long as they are not errors)
Other than that, the function logic should remain the same, blocking until all
statuses are the target, with a special case about target-Pinned, which causes
Remote statuses to be ignored.
2021-08-13 11:24:54 +02:00
Hector Sanjuan
4e61722394
Merge pull request #1445 from ipfs/fix/1415-307-redirect
...
restapi: fix #1415 : paths ending in / get 307-redirect instead of 301
2021-08-12 17:02:31 +02:00
Hector Sanjuan
ecf287c8e6
Fix #1409 : Better support of metrics from older peers
...
Before: receiving a metric from a peer <= 0.13.3 causes decoding error on logs.
Now: metric is correctly parsed and a warning message is printed once.
2021-08-12 00:03:05 +02:00
Hector Sanjuan
70386ea36a
restapi: fix #1415 : paths ending in / get 307-redirect instead of 301
...
The Gorilla muxer StrictSlash option uses a 301 permanent redirect, which
results in POST requests becoming GET requests in most clients. Thus we use
our own middleware that performs a 307 redirect. See issue #1415 for more
details.
2021-08-11 23:20:45 +02:00
Hector Sanjuan
7e5eac3ce0
Merge pull request #1438 from ipfs/fix/1436-recover-boot-bis
...
Fix #1436 : Do not block peer startup waiting for RecoverAll
2021-08-06 14:46:30 +02:00
Hector Sanjuan
cd8938582c
Merge pull request #1439 from ipfs/fix/logging-facs-extra
...
Fix: improve log level handling
2021-08-06 14:46:19 +02:00