Hector Sanjuan
9f57f7280f
api: improve tls setup debug logging
...
Currently the debug messages are a bit confusing.
2022-06-20 21:30:54 +02:00
Hector Sanjuan
57a49fc36f
tls has also been consolidated on go-libp2p ( #1712 )
...
* Depedencies: tls security has also been consolidated on go-libp2p
* build(deps): bump github.com/urfave/cli/v2 from 2.5.1 to 2.10.1
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli ) from 2.5.1 to 2.10.1.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v2.5.1...v2.10.1 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/multiformats/go-multicodec
Bumps [github.com/multiformats/go-multicodec](https://github.com/multiformats/go-multicodec ) from 0.4.1 to 0.5.0.
- [Release notes](https://github.com/multiformats/go-multicodec/releases )
- [Commits](https://github.com/multiformats/go-multicodec/compare/v0.4.1...v0.5.0 )
---
updated-dependencies:
- dependency-name: github.com/multiformats/go-multicodec
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/prometheus/client_golang
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang ) from 1.12.1 to 1.12.2.
- [Release notes](https://github.com/prometheus/client_golang/releases )
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md )
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.1...v1.12.2 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/hashicorp/go-hclog from 1.2.0 to 1.2.1
Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog ) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/hashicorp/go-hclog/releases )
- [Commits](https://github.com/hashicorp/go-hclog/compare/v1.2.0...v1.2.1 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-hclog
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump github.com/urfave/cli/v2 from 2.10.1 to 2.10.2
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli ) from 2.10.1 to 2.10.2.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v2.10.1...v2.10.2 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-20 21:14:45 +02:00
Hector Sanjuan
bc54fd60a6
Merge pull request #1705 from ipfs-cluster/pintracker-wrong-status
...
pintracker: fix status objects missing or having wrong fields
2022-06-20 21:12:20 +02:00
Hector Sanjuan
177af57361
Merge pull request #1703 from ipfs-cluster/pinning-api-auth
...
API: Support JWT bearer token authorization
2022-06-20 21:12:06 +02:00
Hector Sanjuan
d7da1b6044
API: Support JWT bearer token authorization
...
The Pinning Services API standard mandates Bearer token authentication.
This adds JWT bearer token authentication to the IPFS Cluster REST and PINSVC
APIs.
The basic_auth_credentials configuration option needs to be not null and have
at least one username/passwords entry.
A user authenticated via Basic Auth can then "POST /token" and obtain a json
object:
```json { "token" : "<JWTtoken>" } ```
The JWT token has the "iss" (issuer) field set to the Basic auth user that
authorized its creation and is HMAC-signed with its password.
When basic-auth-credentials are set, the APIs will verify that requests come
with either Basic Auth authorization header or with a Bearer token
authorization header.
Bearer tokens will be decoded and the signature will be verified against the
password of the issuer.
At the moment we provide no support to revoke tokens, set "expiration date",
"not before" etc, but this may come in the future.
2022-06-20 20:04:39 +02:00
Hector Sanjuan
f4a445e019
pintracker: fix status objects missing or having wrong fields
...
The operation tracker was not setting some fields correctly when producing
PinInfo objects. Additionally, recover operations were submitted with empty
pin objects, which resulted in the status for pins sent on recover operations
to be missing fields.
2022-06-20 20:03:34 +02:00
Hector Sanjuan
c9120aa672
Merge pull request #1704 from ipfs-cluster/pinsvc-compliance
...
PinSVC API: fix bugs and increase compliance
2022-06-20 18:50:16 +02:00
Hector Sanjuan
ff488e1fec
pinsvc: do not set error descriptions
...
It is duplicated with "reason".
2022-06-20 18:49:23 +02:00
Hector Sanjuan
49e770ce8c
Dependency upgrades ( #1711 )
...
Updates to libp2p and go-cid.
Cluster now conforms with go-ipfs 0.13.0 changes to block/put.
2022-06-20 18:23:50 +02:00
Hector Sanjuan
bb8e8725e8
PinSVC API: fix bugs and increase compliance
...
These changes fix a few bugs in the PinSVC API and have been discovered by
running the compliance tool at https://github.com/ipfs-shipyard/pinning-service-compliance .
In particular, the error objects did not respect the spec, filters and counts
were not done right. An additional PR will follow with fixes to the pintracker
because some pin status information was not correctly set.
2022-06-17 17:06:37 +02:00
Hector Sanjuan
57c3b180b5
Additional fixes for security warnings from CodeQL
2022-06-16 17:44:34 +02:00
Hector Sanjuan
d23240f6c8
CodeQL analysis: fix some security warnings
...
They are related to logging user-input.
2022-06-16 17:44:34 +02:00
Hector Sanjuan
b9a4578f0b
Create codeql-analysis.yml
2022-06-16 17:44:34 +02:00
Hector Sanjuan
71bbf4ab74
Adder: update error message when doing Gets on the DAGService.
...
Indicate that this is not expected and is likely a bug.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
455aa6def1
Fix #1691 : Error when adding directories with many files
...
Fixes #1691 by updating to the latest go-unixfs and adding a test.
The test is verified to fail on the previous go-unixfs version.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
144c66359e
Test module: allow the MockDAGService to be write-only.
...
The adder needs a write-only DAGService.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
e0247e4105
README: more improvements to badges
2022-06-16 17:43:30 +02:00
Hector Sanjuan
b705212ac0
Improve badges in README
2022-06-16 17:43:30 +02:00
Hector Sanjuan
bc8b65a099
Rename Cancelled to Canceled in operationtracker
...
Part of fixing misspellings.
2022-06-16 17:43:30 +02:00
Hector Sanjuan
9bd611e7c3
Error on mispellings
2022-06-16 17:43:30 +02:00
Hector Sanjuan
755cebbe0d
Enable spell checking and fix spelling errors (using US locale)
2022-06-16 17:43:30 +02:00
Hector Sanjuan
6260b11e8c
Use Codecov@v2 github action
2022-06-16 17:43:30 +02:00
Hector Sanjuan
6723acd333
Fix badge image link in readme
2022-06-16 17:43:30 +02:00
Hector Sanjuan
d7107791ed
Rename links to use the ipfscluster.io domain
2022-06-16 17:43:30 +02:00
Hector Sanjuan
508791b547
Migrate from ipfs/ipfs-cluster to ipfs-cluster/ipfs-cluster
...
This performs the necessary renamings.
2022-06-16 17:43:30 +02:00
ipfs-mgmt-read-write[bot]
34396ddb2c
Add .github/workflows/stale.yml
2022-06-16 17:43:30 +02:00
Sungyub NA
fbdeebc71e
Update comment at docker-compose
...
License: MIT
Signed-off-by: Sungyub NA <darkerkorean@gmail.com>
2022-06-16 17:43:30 +02:00
@RubenKelevra
3b51cc39f4
improve language; fix typos; move breaking change under own headline
2022-06-16 17:43:30 +02:00
Hector Sanjuan
3911f8483a
Release v1.0.1
2022-06-16 17:43:30 +02:00
Hector Sanjuan
5a3d36b41a
Changelog for v1.0.1
2022-06-16 17:43:30 +02:00
Hector Sanjuan
90df0ba35e
GetMetadata: ignore linter check
2022-06-16 17:43:29 +02:00
Hector Sanjuan
7a05eeeb60
Fix #1547 : Store pin metadata in a sorted array
...
This deprecates the Metadata protobuf map and starts serializing metadata as an
array that is always sorted by the metadata key.
This should resolve the issue that an state export file can be imported
resulting in two different CRDT dags because the binary representation of the
pin can arbitrarily change depending on how the keys in the map are listed,
and thus the CID of a delta is impacted.
So with this, a state export should result in exactly the same DAG regardless
of where it is imported.
2022-06-16 17:43:29 +02:00
Hector Sanjuan
03740e5bf0
ctl: fix "pin ls" hangs for pinsets > 1024 items.
...
Seems we forgot to adapt "pin ls" and run it in a background goroutine, so it
just hangs.
2022-06-16 17:43:29 +02:00
Hector Sanjuan
447b79093c
Fix #1655 : rest api lbclients panic on error with streaming methods
...
I think this should fix the issue. As solution we make every retry with a
temporary channel and copy results to the final channel which is only closed
by us. This only affects streaming methods.
2022-06-16 17:43:29 +02:00
Hector Sanjuan
bc6fe29d8b
Update to go1.18
2022-06-16 17:43:29 +02:00
Hector Sanjuan
72050c1e59
Stay in go1.17
2022-06-16 17:43:29 +02:00
Hector Sanjuan
cef51c2129
Revert "Test with go 1.18"
...
This reverts commit 9e6428cb1fec27e56f88dbc6f1cca219b035431a.
2022-06-16 17:43:29 +02:00
Hector Sanjuan
a559b137c1
Test with go 1.18
2022-06-16 17:43:29 +02:00
Hector Sanjuan
66bf7d44bf
update to libp2p v0.19.1 and go1.18
2022-06-16 17:43:29 +02:00
Hector Sanjuan
4daece2b98
Feat: add a new "pinqueue" informer component
...
This new component broadcasts metrics about the current size of the pinqueue,
which can in turn be used to inform allocations.
It has a weight_bucket_size option that serves to divide the actual size by a
given factor. This allows considering peers with similar queue sizes to have
the same weight.
Additionally, some changes have been made to the balanced allocator so that a
combination of tags, pinqueue sizes and free-spaces can be used. When
allocating by [<tag>, pinqueue, freespace], the allocator will prioritize
choosing peers with the smallest pin queue weight first, and of those with the
same weight, it will allocate based on freespace.
2022-06-16 17:43:29 +02:00
Hector Sanjuan
8dc18cd3d7
Metrics: add additional metrics
...
* Gauge for total number of ipfs pins
* Counter for pin/add
* Counter for pin/add errors
* Counter for Block/Puts
* Counter for blocks added
* Counter for block/added size
* Counted for block/added errors
2022-06-16 17:43:29 +02:00
dependabot[bot]
b38ea6b4a0
build(deps): bump github.com/urfave/cli from 1.22.8 to 1.22.9
...
Bumps [github.com/urfave/cli](https://github.com/urfave/cli ) from 1.22.8 to 1.22.9.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v1.22.8...v1.22.9 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-06-16 17:43:29 +02:00
dependabot[bot]
74b7215482
build(deps): bump github.com/urfave/cli from 1.22.5 to 1.22.8
...
Bumps [github.com/urfave/cli](https://github.com/urfave/cli ) from 1.22.5 to 1.22.8.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v1.22.5...v1.22.8 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-06-16 17:43:29 +02:00
dependabot[bot]
2d8d477278
build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.5.1
...
Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli ) from 2.4.0 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases )
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md )
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.5.1 )
---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-06-16 17:43:29 +02:00
Hector Sanjuan
ec35841918
Merge pull request #1656 from ipfs/fix/0-space
...
Do not issue freespace metric when free-space is 0.
2022-05-03 19:09:31 +02:00
Hector Sanjuan
060ef812b2
Do not issue freespace metric when free-space is 0.
2022-05-03 18:39:44 +02:00
Hector Sanjuan
0b25f9ce60
Merge pull request #1651 from ipfs/fix/tracker-metrics
...
Fix: tracker metrics not updated on phase change
2022-05-03 18:23:53 +02:00
Hector Sanjuan
8252944aa8
Fix: tracker metrics not updated on phase change
2022-04-30 20:57:57 +02:00
Hector Sanjuan
d89e62117b
Merge pull request #1646 from ipfs/fix/leaks-on-ipfs-restart
...
Fix: bad behaviour when adding and ipfs is down
2022-04-29 22:57:26 +02:00
Hector Sanjuan
585eb1e8e9
Merge pull request #1647 from ipfs/fix/tracker-metrics-gauge
...
pintracker: metrics: convert pinning/queued/error metrics to gauges
2022-04-29 22:57:04 +02:00