Commit Graph

205 Commits

Author SHA1 Message Date
Hector Sanjuan
19b1124999 Make metrics human
Issue #572 exposes metrics but they carry the peer ID in binary.

This was ok with our internal codecs but it doesn't seem to work
very well with json, and makes the output format unusable.

This makes the Metric.Peer field a string.

Additinoally, fixes calling the command without arguments and displaying
the date in the right format.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-26 14:11:30 +02:00
Hector Sanjuan
9330ac82e2 Fix tests with latest libp2p
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-25 14:55:01 +02:00
Hector Sanjuan
b86f3cedb4 Files package has been extracted from cmdkit
Related: https://github.com/ipfs/go-ipfs-cmdkit/pull/31

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-24 01:28:05 +02:00
Hector Sanjuan
79fe5919d2 Fix: escape filter parameter query
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-22 23:45:33 +02:00
Hector Sanjuan
48c89fbe4d
Merge pull request #572 from kishansagathiya/issue_449
Issue #449 API endpoint for Peer Monitor metrics
2018-10-22 23:32:50 +02:00
Kishan Sagathiya
ec4588a5ce Issue #449 API endpoint for Peer Monitor metrics
Rename method PeerMonitorLatestMetrics to Metrics
Addressing first round of comment as in
https://github.com/ipfs/ipfs-cluster/pull/572#pullrequestreview-165367171

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 13:36:20 +05:30
Kishan Sagathiya
3ac7d6c9cc Issue #449 API endpoint for Peer Monitor metrics
Support the new endpoint for later metrics in `rest/api/client`

Support the new method created in `rest/api/client` in
ipfs-cluster-ctl. i.e. `ipfs-cluster-ctl health metrics <name>` would
show the peers and the last list of metrics logged for each as returned
by the Peer Monitor, in a friendly way.

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Kishan Sagathiya
fafbf7b929 Issue #449 API endpoint for Peer Monitor metrics
Opened new endpoint `GET /health/metrics/<name>` which would respond
with metrics of type <name>

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Hector Sanjuan
4e3553fb1f Fix: escape the add query parameters correctly in the client
When adding something that has a name with spaces, things fail
very badly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-19 20:19:52 +02:00
Hector Sanjuan
b0b826de39
Merge pull request #580 from ipfs/libp2p-6.0.19
Upgrade to libp2p-6.0.19. Update deps.
2018-10-18 12:07:46 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
86b74c059a Fix using new libp2p-http
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:47:23 +02:00
Hector Sanjuan
562ad713fc Update docs for sendResponse
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:43:57 +02:00
Hector Sanjuan
322e87dd59 Restapi: Add configurable response headers
By default, CORS headers allowing GET requests from everywhere are
set. This should facilitate the IPFS Web UI integration with the
Cluster API.

This commit refactors the sendResponse methods in the API, merging
them into one as it was difficult to follow the flows that actually
send something to the client. All tests now check the presence of
the configured headers too, to make sure no route was missed.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 11:42:25 +02:00
Hector Sanjuan
b6306a6e61 Add endpoints output: improvements and compliance
This straigthens some mistakes with the outputs of the /add endpoints.

Currently, we had exactly the same output format which:

* was not exactly the ipfs API output format but was sort of similar
* made some weird concessions to be compatible (like having a string-type "size")
* was not aligned with Cluster API conventions (lowercase keys)

This corrects all this:

* The Cluster API /add output format now uses the right types and lowercase keys.
* `Hash` is now `Cid`, because the field carries a Cid.
* We copy error handling with request trailers from IPFS, and avoid carrying the
  errors in the output objects.
* The proxy now returns exactly the types as ipfs would
* We add the X-Chunked-Output: 1 header, which is custom and redundant, but
otherwise breaks js-ipfs-api integrations with the /add endpoint.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-04 00:57:36 +02:00
Hector Sanjuan
dc0b211324 Docs: Fix comment for WaitFor function
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-03 11:04:24 +02:00
Hector Sanjuan
b5631aeb8a Fix IPFS() method missing in Client interface
Seems we forgot about this one.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 20:09:26 +02:00
Adrian Lanzafame
000f2b3f6e api/rest: change *cid.Cid to cid.Cid in client iface
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-01 12:51:57 +02:00
Hector Sanjuan
6e83ef5f24 Fix #548: Re-enable keep-alives. Set Connection: Close when adding.
This is a workaround to have clients behave properly with the /add
endpoint by asking them to close connections when done, effectively
disabling keep-alive for this.

This means we don't need to disable keep-alives fully on all servers,
since the rest of endpoints are not affected (they are not streaming
endpoints).

Reference https://github.com/ipfs/go-ipfs/issues/5168

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-09-29 02:07:57 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Kishan Sagathiya
2cd4420ee8 Issue #446 Adding peername to PinInfo
Fixing errors and better code placement

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:20:37 +05:30
Kishan Sagathiya
773b4de1f0 Issue #446 Adding peername to PinInfo
Removed comments and code used for debugging

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30
Kishan Sagathiya
ef85ba8780 Issue #446 Adding peername to PinInfo
This commit adds peername to PinInfo and GlobalPinInfo so that we have
a nicer and more meaningfull output for `ipfs-cluster-ctl` queries like
`status`, `sync` and `recover`

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30
Kishan Sagathiya
ff48fb319a Issue #446 Adding peername to PinInfo
This commit adds peername to PinInfo and GlobalPinInfo so that we have
a nicer and more meaningfull output for queries like
`ipfs-cluster-ctl status`

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30
Adrian Lanzafame
31474f6490
update go-cid and go-libp2p
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-24 11:35:38 +10:00
Hector Sanjuan
0c5de3a849 Add/improve godoc descriptions for some modules.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-23 14:14:06 +02:00
Hector Sanjuan
d6e516b84a
Merge pull request #513 from ipfs/remove-go-ipfs-dep
AddParams: support "cid-version" and "hash" function.
2018-08-21 20:49:02 +02:00
Hector Sanjuan
b266ff2fbf
Merge pull request #514 from ipfs/fix/466-proxy-repo-stat
Fix #466: Hijack repo/stat in the proxy and return aggregates from the cluster
2018-08-21 19:39:03 +02:00
Hector Sanjuan
05102c13a0 AddParams: support "cid-version" and "hash" function.
Additionally, remove dependency to go-ipfs after extraction from
"go-mfs".

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 19:38:41 +02:00
Hector Sanjuan
9c3826fdc5 Use cmdkit WebFile.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 02:12:36 +02:00
Hector Sanjuan
2ffa3d80de Fix 466: Hijack repo/stat in the proxy and return aggregates from the cluster.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 20:43:27 +02:00
Hector Sanjuan
2061029313 Fix hanging add in sharness.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 12:08:49 +02:00
Hector Sanjuan
b6182e0621 rest/client: Implement .Add(paths), which adds from local files and web files
.Add(paths) will interpret http* paths as WebFiles. These are read performing
a GET request to the location. Otherwise, the path is interpreted as a local
disk file/folder, and read from disk. ipfs-cluster-ctl has been updated
accordingly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 11:07:23 +02:00
Hector Sanjuan
be651da0d0 Fix: disable KeepAlives in server
See discussion in https://github.com/ipfs/go-ipfs/issues/5168

We cannot stream responses with keep-alives enabled.

I prefer this to not be a client feature, as otherwise users might end up
shooting themselves in the foot.

Note, the price is a corrupted request body which gets added
normally and gives wrong hashes!

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:36 +02:00
Hector Sanjuan
00a757fc0b Proxy/add: flush AddedOutput
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
b2914c2516 Fix #495: Hijack proxy /add correctly.
No more hacks around /add. This uses the local adder when hijacking /add.
It supports the parameters and works pretty well with the ipfs CLI, showing
progress and everything.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
584f532bb1 Fix tests after rebase to master
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 15:18:16 +02:00
Adrian Lanzafame
c6eada9db5 uses new gorpc method to distinguish err type
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:34:02 +02:00
Adrian Lanzafame
df2753dfc6 implements a stateless pintracker
Also updates to the optracker to make retrieving information easier.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:33:36 +02:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
2675bf8040
Merge pull request #489 from ipfs/feat/dht
Feat: Enable DHT-based peer discovery and routing for cluster peers
2018-08-14 11:34:35 +02:00
Hector Sanjuan
f4455d3733 Address comments
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 11:56:16 +02:00
Hector Sanjuan
c1363f3ad9 rest/client: clarify comment about overriding ssl configs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 10:28:25 +02:00
Hector Sanjuan
522791df8d Feat/rest/client: auto-handling of libp2p endpoints
This removes PeerAddr and uses APIAddr directly, figuring out if it is
a Peer multiaddress or not.

PeerAddr is actually kept for compatiblity.

It also fixes a bad panic when resolving returned 0 results

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 16:29:41 +02:00
Hector Sanjuan
f2dbd625a3 style
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:53:46 +02:00
Hector Sanjuan
bbf707d61e Fix allocations filter option
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:44 +02:00
Hector Sanjuan
7b826a5335 Dependency upgrades
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:34 +02:00
Hector Sanjuan
50fc3c4e95 Address comments from review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05 Fix: close files. Close test files. Clean properly. Fix windows tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
26cbecca4c Fix tests
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 03:42:39 +02:00