Commit Graph

96 Commits

Author SHA1 Message Date
Kishan Mohanbhai Sagathiya
2c364ddf5e Status filters for ipfs-cluster-ctl status
Improved matching of filters and tracker status

Fixes #445

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-01-05 14:44:37 +05:30
Kishan Mohanbhai Sagathiya
bed9672a74 Status filters for ipfs-cluster-ctl status
Added a fail case where an invalid filter is passed in.
Update `api.rpcClient.Call` to `api.rpcClient.CallContext` and pass
in the Request context r.Context() so that context can be cancelled
when the request is cancelled by caller

Fixes #445

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-01-05 14:44:37 +05:30
Kishan Mohanbhai Sagathiya
02e129fbfe Status filters for ipfs-cluster-ctl status
Added clients tests

Fixes #445

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-01-05 14:44:37 +05:30
Kishan Mohanbhai Sagathiya
f0321afd54 Status filters for ipfs-cluster-ctl status
Added filter option to `ipfs-cluster-ctl status`

When the --filter is passed, it will only fetch the peer information
where status of the pin matches with the filter value.
Valid filter values are tracker status types(i.e., "pinned",
"pin_error", "unpinning" etc), an alias of tracker status type (i.e.,
"queued" or "error"), comma separated list of tracker status type
and/or it aliases(i.e., "error,pinning")

On passing invalid filter value no status information will be shown

In particular, the filter would remove elements from []GlobalPinInfo
when none of the peers in GlobalPinInfo match the filter. If one peer
in the GlobalPinInfo matches the filter, the whole object is returned,
including the information for the other peers which may or not match it.

filter option works on statusAll("GET /pins"). For fetching pin status
for a CID("GET /pins/<cid>"), filter option would have no effect

Fixes #445

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-01-05 14:44:37 +05:30
paul
7c04dd1acc Fix #445: Implemented status filter for ipfs-cluster-ctl
License: MIT
Signed-off-by: Paul Jewell <sona1111@zoho.com>
2019-01-05 14:44:37 +05:30
Hector Sanjuan
b44c7fb990
Merge pull request #622 from ipfs/issue_587
Added tests for /monitor/metrics/{metrics_type}
2019-01-04 18:17:05 +01:00
Kishan Mohanbhai Sagathiya
fc678242fb Added tests for /monitor/metrics/{metrics_type}
Move ctl-health sharness tests to apprpriate file

Since the API is using the RPC mock to request metrics and it always
returns a mocked test metric we might just do c.Metrics("somemetricstype")
and check that there is no error. Here we just want to check that the
client is hitting an API endpoint (and understands the response).

Fixes #587

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-21 08:24:40 +05:30
Kishan Sagathiya
6049b216bb Sharness tests for ipfs-cluster-ctl health metrics
Added sharness tests for `ipfs-cluster-ctl health metrics <metricname>`

Fixes #587

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-21 08:24:40 +05:30
Kishan Sagathiya
e9cf8abb3a Added tests for /monitor/metrics/{metrics_type}
Added API and client tests for GET /monitor/metrics/{metrics_type}

Fixes #587

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-21 08:24:40 +05:30
Hector Sanjuan
b93b2fedd4 Fix interpreting Host parameter correctly.
We should deprecate passing in Host/Port in the config,
but in the meantime, it hardcoded /dns4/, meaning that if
someone placed an ipv6 address in there things would break badly
and weirdly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-19 21:19:47 +01:00
Hector Sanjuan
7c26f1ecb8 Fix HTTPs with DNS multiaddresses
Before we resolved all /dns*/ multiaddresses before we used them.

When using HTTPs, the Go HTTP Client only sees the resolved IP address
and it is unable to negotiate TLS with a cerficate because the request
is not going to the hostname the certificate is signed for, but to
the IP. This leverages a recent feature in go-multiaddr-net
and uses directly the user-provided hostname.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-19 21:15:37 +01:00
Hector Sanjuan
11cc0485c7 API/client: Do not clean folders mid-test.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:55:34 +01:00
Hector Sanjuan
ced9f2f7f1 Update to go-ipfs-files 2.0.3
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:29:44 +01:00
Hector Sanjuan
15413edbc6 Unrewrite deps
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
406d2b53f8 files2.0: Close subfiles in adder
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
c6944e4376 files2.0: update for no errors from NewMultiFileReader
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
00df384e1c go-ipfs-files 2.0 updates
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Hector Sanjuan
1d924bfc49 Remove shard testing folders
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-11-01 19:15:08 +01:00
Hector Sanjuan
19b1124999 Make metrics human
Issue #572 exposes metrics but they carry the peer ID in binary.

This was ok with our internal codecs but it doesn't seem to work
very well with json, and makes the output format unusable.

This makes the Metric.Peer field a string.

Additinoally, fixes calling the command without arguments and displaying
the date in the right format.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-26 14:11:30 +02:00
Hector Sanjuan
9330ac82e2 Fix tests with latest libp2p
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-25 14:55:01 +02:00
Hector Sanjuan
b86f3cedb4 Files package has been extracted from cmdkit
Related: https://github.com/ipfs/go-ipfs-cmdkit/pull/31

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-24 01:28:05 +02:00
Hector Sanjuan
79fe5919d2 Fix: escape filter parameter query
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-22 23:45:33 +02:00
Hector Sanjuan
48c89fbe4d
Merge pull request #572 from kishansagathiya/issue_449
Issue #449 API endpoint for Peer Monitor metrics
2018-10-22 23:32:50 +02:00
Kishan Sagathiya
ec4588a5ce Issue #449 API endpoint for Peer Monitor metrics
Rename method PeerMonitorLatestMetrics to Metrics
Addressing first round of comment as in
https://github.com/ipfs/ipfs-cluster/pull/572#pullrequestreview-165367171

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 13:36:20 +05:30
Kishan Sagathiya
3ac7d6c9cc Issue #449 API endpoint for Peer Monitor metrics
Support the new endpoint for later metrics in `rest/api/client`

Support the new method created in `rest/api/client` in
ipfs-cluster-ctl. i.e. `ipfs-cluster-ctl health metrics <name>` would
show the peers and the last list of metrics logged for each as returned
by the Peer Monitor, in a friendly way.

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Hector Sanjuan
4e3553fb1f Fix: escape the add query parameters correctly in the client
When adding something that has a name with spaces, things fail
very badly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-19 20:19:52 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
b6306a6e61 Add endpoints output: improvements and compliance
This straigthens some mistakes with the outputs of the /add endpoints.

Currently, we had exactly the same output format which:

* was not exactly the ipfs API output format but was sort of similar
* made some weird concessions to be compatible (like having a string-type "size")
* was not aligned with Cluster API conventions (lowercase keys)

This corrects all this:

* The Cluster API /add output format now uses the right types and lowercase keys.
* `Hash` is now `Cid`, because the field carries a Cid.
* We copy error handling with request trailers from IPFS, and avoid carrying the
  errors in the output objects.
* The proxy now returns exactly the types as ipfs would
* We add the X-Chunked-Output: 1 header, which is custom and redundant, but
otherwise breaks js-ipfs-api integrations with the /add endpoint.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-04 00:57:36 +02:00
Hector Sanjuan
dc0b211324 Docs: Fix comment for WaitFor function
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-03 11:04:24 +02:00
Hector Sanjuan
b5631aeb8a Fix IPFS() method missing in Client interface
Seems we forgot about this one.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 20:09:26 +02:00
Adrian Lanzafame
000f2b3f6e api/rest: change *cid.Cid to cid.Cid in client iface
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-01 12:51:57 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Adrian Lanzafame
31474f6490
update go-cid and go-libp2p
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-24 11:35:38 +10:00
Hector Sanjuan
0c5de3a849 Add/improve godoc descriptions for some modules.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-23 14:14:06 +02:00
Hector Sanjuan
9c3826fdc5 Use cmdkit WebFile.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 02:12:36 +02:00
Hector Sanjuan
2061029313 Fix hanging add in sharness.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 12:08:49 +02:00
Hector Sanjuan
b6182e0621 rest/client: Implement .Add(paths), which adds from local files and web files
.Add(paths) will interpret http* paths as WebFiles. These are read performing
a GET request to the location. Otherwise, the path is interpreted as a local
disk file/folder, and read from disk. ipfs-cluster-ctl has been updated
accordingly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 11:07:23 +02:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
2675bf8040
Merge pull request #489 from ipfs/feat/dht
Feat: Enable DHT-based peer discovery and routing for cluster peers
2018-08-14 11:34:35 +02:00
Hector Sanjuan
c1363f3ad9 rest/client: clarify comment about overriding ssl configs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 10:28:25 +02:00
Hector Sanjuan
522791df8d Feat/rest/client: auto-handling of libp2p endpoints
This removes PeerAddr and uses APIAddr directly, figuring out if it is
a Peer multiaddress or not.

PeerAddr is actually kept for compatiblity.

It also fixes a bad panic when resolving returned 0 results

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 16:29:41 +02:00
Hector Sanjuan
bbf707d61e Fix allocations filter option
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:44 +02:00
Hector Sanjuan
50fc3c4e95 Address comments from review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05 Fix: close files. Close test files. Clean properly. Fix windows tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
26cbecca4c Fix tests
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 03:42:39 +02:00
Hector Sanjuan
0151f5e312 Fixes for adding: set default timeouts to 0. Improve flags and param names.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-08 21:11:26 +02:00
Hector Sanjuan
585cc69eed Put add endpoint in /add
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
6ffc8b8347 Update metrics regularly after a few blockputs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
3575f05753 Support adding Output (api.AddedOutput)
This was a long FIXME/TODO. Handling adding output and
reporting to the client of the progress of the adding process.

This attempts to do it. It is not sure that it works correctly
(response body being written while the multipart request is still being read)

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
8f1a15b279 Move adder.Params to api.AddParams. Re-use in other modules
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00