cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,676 Commits 3 Branches 70 Tags 15 MiB
a97ed10d0b
Commit Graph

2676 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hector Sanjuan
f05aa7e1a3 Ipfs-lite v1.2.0 2021-11-30 16:04:01 +01:00
Hector Sanjuan
db93c50c2d Fix rest client tests: do not use QUIC 2021-11-30 07:05:35 +01:00
Hector Sanjuan
5638ed4ebf Fix dependency error 2021-11-30 06:54:26 +01:00
Hector Sanjuan
23dd3ba41b Go1.17 2021-11-30 06:51:06 +01:00
Hector Sanjuan
28dcb132b0 Disable QUIC for cluster peers (no private network support) 2021-11-30 06:34:13 +01:00
Hector Sanjuan
4b2d3a0561 Correctly switch to new relay options 2021-11-30 06:33:41 +01:00
Hector Sanjuan
4739ed9210 Changes pertaining to go-libp2p v0.16.0 2021-11-30 06:25:15 +01:00
Hector Sanjuan
fdc3abcb23 api: use default transports for the libp2p host. 2021-11-30 06:06:10 +01:00
Hector Sanjuan
33343751b0 Stateless: add tests to make sure AttemptCounts are set correctly 2021-11-30 04:43:16 +01:00
Hector Sanjuan
c4ca5b7abe pintracker: carry over attempt account only for operations of the same type 2021-11-30 04:20:35 +01:00
Hector Sanjuan
7cf40de354 pintracker: Fix attempt count not increasing. Expose priority queueing. 
"RetryCount" has been renamed to "AttemptCount", because it counts attempts.
 2021-11-30 04:20:35 +01:00
Hector Sanjuan
be5c2d1569 pintracker: PriorityPinMaxAge and PriorityPinMaxRetries 
These new configuration settings control whether pins are enqueued in the
priority queue or not.
 2021-11-30 04:20:35 +01:00
Hector Sanjuan
0a146dae76 pintracker: support a priority channel for pinning 2021-11-30 04:20:35 +01:00
Hector Sanjuan
29c277b67f Pintracker: add and track retry counts in the operation manager. 
Report retry count in the PinStatus
 2021-11-30 04:20:35 +01:00
Hector Sanjuan
be18c645fa
Merge pull request #1491 from ipfs/dependency-upgrades 
Dependency upgrades
 2021-10-27 18:51:04 +02:00
Hector Sanjuan
555b624beb Dependency upgrades 2021-10-27 16:03:59 +02:00
Hector Sanjuan
3ddda1fb59 Merge branch 'master' into dependency-upgrades 2021-10-27 15:55:34 +02:00
Hector Sanjuan
a74b6faf96
Merge pull request #1484 from ipfs/feat/pin-time 
Add: include a timestamp that tracks when a Pin was added to the state
 2021-10-27 10:42:59 +02:00
Hector Sanjuan
f04afda51f
Merge pull request #1485 from ipfs/fix/default-allocator-compat 
Fix: backwards compatibility of the new allocator
 2021-10-27 10:38:05 +02:00
Hector Sanjuan
24d8b59609 Fix config test 2021-10-20 17:12:21 +02:00
Hector Sanjuan
53342d3c55 Fix go-multiaddr-net 2021-10-20 16:56:24 +02:00
Hector Sanjuan
32386d853a Dependency upgrades 2021-10-20 16:56:24 +02:00
dependabot[bot]
067db00d7a Bump github.com/multiformats/go-multihash from 0.0.15 to 0.0.16 
Bumps [github.com/multiformats/go-multihash](https://github.com/multiformats/go-multihash) from 0.0.15 to 0.0.16.
- [Release notes](https://github.com/multiformats/go-multihash/releases)
- [Commits](https://github.com/multiformats/go-multihash/compare/v0.0.15...v0.0.16)

---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multihash
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-20 16:56:24 +02:00
dependabot[bot]
8ce4b55607 Bump github.com/ipfs/go-path from 0.1.0 to 0.1.1 
Bumps [github.com/ipfs/go-path](https://github.com/ipfs/go-path) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/ipfs/go-path/releases)
- [Commits](https://github.com/ipfs/go-path/compare/v0.1.0...v0.1.1)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-path
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-20 16:56:24 +02:00
dependabot[bot]
7ece0473fa Bump github.com/libp2p/go-libp2p-quic-transport from 0.11.2 to 0.12.0 
Bumps [github.com/libp2p/go-libp2p-quic-transport](https://github.com/libp2p/go-libp2p-quic-transport) from 0.11.2 to 0.12.0.
- [Release notes](https://github.com/libp2p/go-libp2p-quic-transport/releases)
- [Commits](https://github.com/libp2p/go-libp2p-quic-transport/compare/v0.11.2...v0.12.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-libp2p-quic-transport
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-20 16:56:24 +02:00
dependabot[bot]
e4a197796c Bump github.com/ipfs/go-ipfs-config from 0.15.0 to 0.16.0 
Bumps [github.com/ipfs/go-ipfs-config](https://github.com/ipfs/go-ipfs-config) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/ipfs/go-ipfs-config/releases)
- [Commits](https://github.com/ipfs/go-ipfs-config/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-ipfs-config
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-20 16:56:24 +02:00
dependabot[bot]
42c363bbc2 Bump github.com/ipfs/go-cid from 0.0.7 to 0.1.0 
Bumps [github.com/ipfs/go-cid](https://github.com/ipfs/go-cid) from 0.0.7 to 0.1.0.
- [Release notes](https://github.com/ipfs/go-cid/releases)
- [Commits](https://github.com/ipfs/go-cid/compare/v0.0.7...v0.1.0)

---
updated-dependencies:
- dependency-name: github.com/ipfs/go-cid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-20 16:56:24 +02:00
Hector Sanjuan
c776051397 ctl: Include "added" column in pin ls command 
Also, change format from humanize to "yyyy-mm-dd HH:MM:SS" for times.
 2021-10-20 16:55:57 +02:00
Hector Sanjuan
e9857652f2 Add a timestamp to Pins 
This adds a Timestamp field to the pin objects. This allows to track when they were pinned.

This:

* Allows the pin-tracker to actually show accurate information on when the pin
  entered the system for pins that are not part of ongoing operations
  (currently it shows time.Now())
* Adds support for reporting timestamp on a pinning services api.
 2021-10-20 16:55:57 +02:00
Hector Sanjuan
3f895df8cf Fix: backwards compatibility for allocator default settings 
When the allocator is not defined in the configuration, it will take defaults
and assume there is a "tags" informer. That is not the case. When not defined,
we assume it should allocate only by "freespace".
 2021-10-20 16:55:35 +02:00
Hector Sanjuan
1dca6e0eca allocator: balanced allocator name should be "balanced" 
It was an oversight to not rename this.
 2021-10-20 16:55:35 +02:00
Hector Sanjuan
a741e3067d
Merge pull request #1486 from ipfs/feat/github-actions 
Move testing pipeline to github actions.
 2021-10-20 16:54:08 +02:00
Hector Sanjuan
066b1f4514 Move testing pipeline to github actions. 
Since travis cannot stop sucking.
 2021-10-20 16:52:46 +02:00
Hector Sanjuan
af9313d4fe Remove test files that should have not been committed 2021-10-19 13:51:17 +02:00
Hector Sanjuan
96db605c50
Merge pull request #1468 from ipfs/fix/159-improved-allocators 
Add tags informer and enable partition-based peer allocations for intelligent distribution
 2021-10-06 14:35:16 +02:00
Hector Sanjuan
db00d651bf Balanced allocator: weight-based ordering of partitions 
This fixes the issue about partitions not being picked based
on the amount of freespace available in them.

It additionally removes the metrics registry and carries information directly
in the metric.

Metrics have two additional fields: Weight and Partitionable.

Informers have been updated to make use of these fields. Partitions have
weights that equals to the weight of the metrics under them.

Older cluster versions will not set these fields. Partitionable is false by
default and weight has a GetWeight() function to convert value->weight when
unset. This provides backwards compatibility for the freespace metric.
 2021-10-06 14:10:06 +02:00
Hector Sanjuan
26e229df94 Rename allocator/metrics to allocator/balanced 2021-10-06 11:26:38 +02:00
Hector Sanjuan
6b31f44351 Address most comments from PR review 2021-10-05 14:04:28 +02:00
Hector Sanjuan
0dfe1ae063
Merge pull request #1471 from ipfs/feat/refactor-api 
API: Refactor REST API to enable re-use of functionality.
 2021-09-16 16:10:20 +02:00
Hector Sanjuan
63972f2b2e API: Refactor REST API. Extract all functionality. 
This is a preparatory PR to add additional APIs (Pinning Service API) easily
to cluster.

Instead of copy-pasting most of what the REST API does, I have refactored so
that the whole configuration, routing and request-handling utilities can be
re-used.

The worst part has been to divide the test between tests that test core
(common.API) functionality and tests that test specific REST API endpoint
functionality. I could not get away without an additional common/test package
to provide functions that are used from both places. This is a side effect of
testing both http and libp2p endpoints for every request etc.
 2021-09-16 15:52:25 +02:00
Hector Sanjuan
cf4fb74993 service/follow: Enable new metrics allocator 2021-09-15 21:49:26 +02:00
Hector Sanjuan
1c0abde8a5 Informer: add tags informer 
The tags informer produces metrics in the form tags:name/value
with the name and values defined in its configuration.
 2021-09-15 20:07:37 +02:00
Hector Sanjuan
ea5e18078c Informers: GetMetric() -> GetMetrics() 
Support returning multiple metrics per informer.
 2021-09-15 20:07:37 +02:00
Hector Sanjuan
b6a46cd8a4 allocator: rework the whole allocator system 
The new "metrics" allocator is about to partition metrics and distribe
allocations among the partitions.

For example: given a region, an availability zone and free space on disk, the
allocator would be able to choose allocations by distributing among regions
and availability zones as much as possible, and for those peers in the same
region/az, selecting those with most free space first.

This requires a major overhaul of the allocator component.
 2021-09-13 12:24:00 +02:00
Hector Sanjuan
4060f4196b Revert "Informer/disk: deprecate RepoSize metric" 
This reverts commit 3e54c4c695.
 2021-09-10 18:53:59 +02:00
Hector Sanjuan
3e54c4c695 Informer/disk: deprecate RepoSize metric 
The allocator is hardcoded to descendalloc for freespace so it is not even useful
as it would allocate to peers with largest reposize first no matter what.

We are, in any case, reworking allocators and informers etc.
 2021-09-08 17:36:54 +02:00
Hector Sanjuan
396a348a65
Merge pull request #1462 from ipfs/fix/1461-flaky-test 
Adder: try to make test less flaky
 2021-08-30 12:04:43 +02:00
Hector Sanjuan
26007114b4 Adder: try to make test less flaky 2021-08-29 01:16:28 +02:00
Hector Sanjuan
f9b20caa2a changelog: fix v0.14.1 release date 2021-08-17 10:22:36 +02:00
Hector Sanjuan
267deeff28
Set development version v0.14.1-next 2021-08-16 18:03:18 +02:00