Commit Graph

144 Commits

Author SHA1 Message Date
Hector Sanjuan
11cc0485c7 API/client: Do not clean folders mid-test.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:55:34 +01:00
Hector Sanjuan
ced9f2f7f1 Update to go-ipfs-files 2.0.3
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:29:44 +01:00
Hector Sanjuan
15413edbc6 Unrewrite deps
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
406d2b53f8 files2.0: Close subfiles in adder
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
c6944e4376 files2.0: update for no errors from NewMultiFileReader
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
00df384e1c go-ipfs-files 2.0 updates
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Adrian Lanzafame
e115021fa7
add envcfg to rest api
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-09 13:20:25 +10:00
Hector Sanjuan
1d924bfc49 Remove shard testing folders
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-11-01 19:15:08 +01:00
Hector Sanjuan
19b1124999 Make metrics human
Issue #572 exposes metrics but they carry the peer ID in binary.

This was ok with our internal codecs but it doesn't seem to work
very well with json, and makes the output format unusable.

This makes the Metric.Peer field a string.

Additinoally, fixes calling the command without arguments and displaying
the date in the right format.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-26 14:11:30 +02:00
Hector Sanjuan
9330ac82e2 Fix tests with latest libp2p
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-25 14:55:01 +02:00
Hector Sanjuan
b86f3cedb4 Files package has been extracted from cmdkit
Related: https://github.com/ipfs/go-ipfs-cmdkit/pull/31

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-24 01:28:05 +02:00
Hector Sanjuan
79fe5919d2 Fix: escape filter parameter query
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-22 23:45:33 +02:00
Hector Sanjuan
48c89fbe4d
Merge pull request #572 from kishansagathiya/issue_449
Issue #449 API endpoint for Peer Monitor metrics
2018-10-22 23:32:50 +02:00
Kishan Sagathiya
ec4588a5ce Issue #449 API endpoint for Peer Monitor metrics
Rename method PeerMonitorLatestMetrics to Metrics
Addressing first round of comment as in
https://github.com/ipfs/ipfs-cluster/pull/572#pullrequestreview-165367171

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 13:36:20 +05:30
Kishan Sagathiya
3ac7d6c9cc Issue #449 API endpoint for Peer Monitor metrics
Support the new endpoint for later metrics in `rest/api/client`

Support the new method created in `rest/api/client` in
ipfs-cluster-ctl. i.e. `ipfs-cluster-ctl health metrics <name>` would
show the peers and the last list of metrics logged for each as returned
by the Peer Monitor, in a friendly way.

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Kishan Sagathiya
fafbf7b929 Issue #449 API endpoint for Peer Monitor metrics
Opened new endpoint `GET /health/metrics/<name>` which would respond
with metrics of type <name>

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Hector Sanjuan
4e3553fb1f Fix: escape the add query parameters correctly in the client
When adding something that has a name with spaces, things fail
very badly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-19 20:19:52 +02:00
Hector Sanjuan
b0b826de39
Merge pull request #580 from ipfs/libp2p-6.0.19
Upgrade to libp2p-6.0.19. Update deps.
2018-10-18 12:07:46 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
86b74c059a Fix using new libp2p-http
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:47:23 +02:00
Hector Sanjuan
562ad713fc Update docs for sendResponse
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:43:57 +02:00
Hector Sanjuan
322e87dd59 Restapi: Add configurable response headers
By default, CORS headers allowing GET requests from everywhere are
set. This should facilitate the IPFS Web UI integration with the
Cluster API.

This commit refactors the sendResponse methods in the API, merging
them into one as it was difficult to follow the flows that actually
send something to the client. All tests now check the presence of
the configured headers too, to make sure no route was missed.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 11:42:25 +02:00
Hector Sanjuan
b6306a6e61 Add endpoints output: improvements and compliance
This straigthens some mistakes with the outputs of the /add endpoints.

Currently, we had exactly the same output format which:

* was not exactly the ipfs API output format but was sort of similar
* made some weird concessions to be compatible (like having a string-type "size")
* was not aligned with Cluster API conventions (lowercase keys)

This corrects all this:

* The Cluster API /add output format now uses the right types and lowercase keys.
* `Hash` is now `Cid`, because the field carries a Cid.
* We copy error handling with request trailers from IPFS, and avoid carrying the
  errors in the output objects.
* The proxy now returns exactly the types as ipfs would
* We add the X-Chunked-Output: 1 header, which is custom and redundant, but
otherwise breaks js-ipfs-api integrations with the /add endpoint.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-04 00:57:36 +02:00
Hector Sanjuan
dc0b211324 Docs: Fix comment for WaitFor function
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-03 11:04:24 +02:00
Hector Sanjuan
b5631aeb8a Fix IPFS() method missing in Client interface
Seems we forgot about this one.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 20:09:26 +02:00
Adrian Lanzafame
000f2b3f6e api/rest: change *cid.Cid to cid.Cid in client iface
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-01 12:51:57 +02:00
Hector Sanjuan
6e83ef5f24 Fix #548: Re-enable keep-alives. Set Connection: Close when adding.
This is a workaround to have clients behave properly with the /add
endpoint by asking them to close connections when done, effectively
disabling keep-alive for this.

This means we don't need to disable keep-alives fully on all servers,
since the rest of endpoints are not affected (they are not streaming
endpoints).

Reference https://github.com/ipfs/go-ipfs/issues/5168

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-09-29 02:07:57 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Adrian Lanzafame
31474f6490
update go-cid and go-libp2p
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-09-24 11:35:38 +10:00
Hector Sanjuan
0c5de3a849 Add/improve godoc descriptions for some modules.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-23 14:14:06 +02:00
Hector Sanjuan
9c3826fdc5 Use cmdkit WebFile.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-21 02:12:36 +02:00
Hector Sanjuan
2061029313 Fix hanging add in sharness.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 12:08:49 +02:00
Hector Sanjuan
b6182e0621 rest/client: Implement .Add(paths), which adds from local files and web files
.Add(paths) will interpret http* paths as WebFiles. These are read performing
a GET request to the location. Otherwise, the path is interpreted as a local
disk file/folder, and read from disk. ipfs-cluster-ctl has been updated
accordingly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 11:07:23 +02:00
Hector Sanjuan
be651da0d0 Fix: disable KeepAlives in server
See discussion in https://github.com/ipfs/go-ipfs/issues/5168

We cannot stream responses with keep-alives enabled.

I prefer this to not be a client feature, as otherwise users might end up
shooting themselves in the foot.

Note, the price is a corrupted request body which gets added
normally and gives wrong hashes!

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:36 +02:00
Hector Sanjuan
00a757fc0b Proxy/add: flush AddedOutput
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
2675bf8040
Merge pull request #489 from ipfs/feat/dht
Feat: Enable DHT-based peer discovery and routing for cluster peers
2018-08-14 11:34:35 +02:00
Hector Sanjuan
c1363f3ad9 rest/client: clarify comment about overriding ssl configs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 10:28:25 +02:00
Hector Sanjuan
522791df8d Feat/rest/client: auto-handling of libp2p endpoints
This removes PeerAddr and uses APIAddr directly, figuring out if it is
a Peer multiaddress or not.

PeerAddr is actually kept for compatiblity.

It also fixes a bad panic when resolving returned 0 results

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 16:29:41 +02:00
Hector Sanjuan
f2dbd625a3 style
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:53:46 +02:00
Hector Sanjuan
bbf707d61e Fix allocations filter option
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:44 +02:00
Hector Sanjuan
50fc3c4e95 Address comments from review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05 Fix: close files. Close test files. Clean properly. Fix windows tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
26cbecca4c Fix tests
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 03:42:39 +02:00
Hector Sanjuan
0151f5e312 Fixes for adding: set default timeouts to 0. Improve flags and param names.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-08 21:11:26 +02:00
Hector Sanjuan
87f4fcf958 Make the adder modules ipld.DAGService modules.
This removes a bunch of the channel dance and block forwarding
by having the adder submodules be DAGServices themselves and take
Add() directly from the ipfsAdder.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
585cc69eed Put add endpoint in /add
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
6ffc8b8347 Update metrics regularly after a few blockputs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
3575f05753 Support adding Output (api.AddedOutput)
This was a long FIXME/TODO. Handling adding output and
reporting to the client of the progress of the adding process.

This attempts to do it. It is not sure that it works correctly
(response body being written while the multipart request is still being read)

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
8f1a15b279 Move adder.Params to api.AddParams. Re-use in other modules
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00