Commit Graph

231 Commits

Author SHA1 Message Date
Hector Sanjuan
b93b2fedd4 Fix interpreting Host parameter correctly.
We should deprecate passing in Host/Port in the config,
but in the meantime, it hardcoded /dns4/, meaning that if
someone placed an ipv6 address in there things would break badly
and weirdly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-19 21:19:47 +01:00
Hector Sanjuan
7c26f1ecb8 Fix HTTPs with DNS multiaddresses
Before we resolved all /dns*/ multiaddresses before we used them.

When using HTTPs, the Go HTTP Client only sees the resolved IP address
and it is unable to negotiate TLS with a cerficate because the request
is not going to the hostname the certificate is signed for, but to
the IP. This leverages a recent feature in go-multiaddr-net
and uses directly the user-provided hostname.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-19 21:15:37 +01:00
Hector Sanjuan
11cc0485c7 API/client: Do not clean folders mid-test.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:55:34 +01:00
Hector Sanjuan
ced9f2f7f1 Update to go-ipfs-files 2.0.3
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:29:44 +01:00
Hector Sanjuan
15413edbc6 Unrewrite deps
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
406d2b53f8 files2.0: Close subfiles in adder
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
c6944e4376 files2.0: update for no errors from NewMultiFileReader
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
00df384e1c go-ipfs-files 2.0 updates
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Kishan Sagathiya
3b86a94418 Put a note for deprecating json fields
Put a note for deprecating json fields that they are only here to
maintain compatibility and they will be removed in future

Start using env vars starting with `CLUSTER_IPFSPROXY`

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-17 08:24:17 +05:30
Kishan Sagathiya
8b33dbec03 Remove proxy_ and Proxy from proxy config
Remove proxy_ and Proxy from proxy config objects without breaking
compatibility with previous revisions

Fixes #616

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-13 00:21:21 +05:30
Kishan Sagathiya
e21d84dbee Issue #453 Extract the IPFS Proxy from ipfshttp
Warning about old config style(proxy inside ipfs connector) getting
removed in future

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-27 10:36:00 +05:30
Kishan Sagathiya
253185ad13 Merge branch 'master' of github.com:ipfs/ipfs-cluster into issue_453 2018-11-12 05:23:44 +05:30
Kishan Sagathiya
f98ed71ff7 Issue #453 Extract the IPFS Proxy from ipfshttp
Fixed some messages and comments

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-11 21:09:13 +05:30
Kishan Sagathiya
aef68f4101 Issue #453 Extract the IPFS Proxy from ipfshttp
Fixed breaking tests

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-11 16:23:36 +05:30
Adrian Lanzafame
e115021fa7
add envcfg to rest api
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-09 13:20:25 +10:00
Kishan Sagathiya
41edb04486 Issue #453 Extract the IPFS Proxy from ipfshttp
Added test for ipfsproxy config

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-04 09:05:53 +05:30
Kishan Sagathiya
56cc3b88d7 Issue #453 Extract the IPFS Proxy from ipfshttp
Added test for ipfsproxy

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-04 08:57:09 +05:30
Kishan Sagathiya
411eea664c Merge branch 'master' of github.com:ipfs/ipfs-cluster into issue_453 2018-11-03 20:24:15 +05:30
Hector Sanjuan
1d924bfc49 Remove shard testing folders
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-11-01 19:15:08 +01:00
Kishan Sagathiya
3a5ad6111a Issue #453 Extract the IPFS Proxy from ipfshttp
Changes as requieed
Rename IPFSProxy struct to Server

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-11-01 15:54:05 +05:30
Adrian Lanzafame
820df17359
add pinSerial.DecodeCid and benchs
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-30 21:07:26 +10:00
Hector Sanjuan
d63a5e2667 Fix race on ApplyTo
The FSM tries to decode an operation on top of the
*LogOp. We might still be using the *LogOp.Cid.Allocations
slice. We need to make a deep of *LogOp.Cid before
returning from ApplyTo.

This one was tricky...

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-29 12:03:47 +01:00
Hector Sanjuan
19b1124999 Make metrics human
Issue #572 exposes metrics but they carry the peer ID in binary.

This was ok with our internal codecs but it doesn't seem to work
very well with json, and makes the output format unusable.

This makes the Metric.Peer field a string.

Additinoally, fixes calling the command without arguments and displaying
the date in the right format.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-26 14:11:30 +02:00
Hector Sanjuan
9330ac82e2 Fix tests with latest libp2p
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-25 14:55:01 +02:00
Hector Sanjuan
b86f3cedb4 Files package has been extracted from cmdkit
Related: https://github.com/ipfs/go-ipfs-cmdkit/pull/31

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-24 01:28:05 +02:00
Hector Sanjuan
79fe5919d2 Fix: escape filter parameter query
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-22 23:45:33 +02:00
Hector Sanjuan
48c89fbe4d
Merge pull request #572 from kishansagathiya/issue_449
Issue #449 API endpoint for Peer Monitor metrics
2018-10-22 23:32:50 +02:00
Kishan Sagathiya
ec4588a5ce Issue #449 API endpoint for Peer Monitor metrics
Rename method PeerMonitorLatestMetrics to Metrics
Addressing first round of comment as in
https://github.com/ipfs/ipfs-cluster/pull/572#pullrequestreview-165367171

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 13:36:20 +05:30
Kishan Sagathiya
3ac7d6c9cc Issue #449 API endpoint for Peer Monitor metrics
Support the new endpoint for later metrics in `rest/api/client`

Support the new method created in `rest/api/client` in
ipfs-cluster-ctl. i.e. `ipfs-cluster-ctl health metrics <name>` would
show the peers and the last list of metrics logged for each as returned
by the Peer Monitor, in a friendly way.

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Kishan Sagathiya
fafbf7b929 Issue #449 API endpoint for Peer Monitor metrics
Opened new endpoint `GET /health/metrics/<name>` which would respond
with metrics of type <name>

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-21 12:49:56 +05:30
Hector Sanjuan
4e3553fb1f Fix: escape the add query parameters correctly in the client
When adding something that has a name with spaces, things fail
very badly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-19 20:19:52 +02:00
Hector Sanjuan
b0b826de39
Merge pull request #580 from ipfs/libp2p-6.0.19
Upgrade to libp2p-6.0.19. Update deps.
2018-10-18 12:07:46 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
86b74c059a Fix using new libp2p-http
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:47:23 +02:00
Hector Sanjuan
562ad713fc Update docs for sendResponse
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 13:43:57 +02:00
Hector Sanjuan
322e87dd59 Restapi: Add configurable response headers
By default, CORS headers allowing GET requests from everywhere are
set. This should facilitate the IPFS Web UI integration with the
Cluster API.

This commit refactors the sendResponse methods in the API, merging
them into one as it was difficult to follow the flows that actually
send something to the client. All tests now check the presence of
the configured headers too, to make sure no route was missed.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 11:42:25 +02:00
Kishan Sagathiya
fdb573c96f Issue #453 Extract the IPFS Proxy from ipfshttp
Adding more missing pieces in config
Use the right package(not the inbuilt one)
Setup rpc client for proxy in the cluster
Add back SetClient and Shutdown into Connector as they are required to
implement Component interface
Add `ipfsproxy` as into list of logging identifier and add its default
log level

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-14 22:42:50 +05:30
Kishan Sagathiya
6803df4e97 Issue #453 Extract the IPFS Proxy from ipfshttp
Go through `ipfsproxy` repo and change things that are inappropriate

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-14 18:00:19 +05:30
Kishan Sagathiya
d338f8de53 Issue #453 Extract the IPFS Proxy from ipfshttp
Added config
Change functions accordingly to add new `ipfsproxy` component

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-14 16:07:42 +05:30
Kishan Sagathiya
155a65cac3 Issue #453 Extract the IPFS Proxy from ipfshttp
Extract the IPFS Proxy from ipfshttp and make it an api module

The `ipfshttp` IPFSConnector implementation includes the so called IPFS
Proxy. An endpoint which offers an IPFS API, hijacking some interesting
requests and forwarding the rest to the ipfs daemon.

`ipfshttp` should contain an implementation of IPFSConnector whose only
task should be to talk to IPFS
A new module should be created, `api/ipfsproxy`, an API Component
implementation for Cluster. The whole proxy code should be moved here.

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-10-13 19:58:28 +05:30
Hector Sanjuan
b6306a6e61 Add endpoints output: improvements and compliance
This straigthens some mistakes with the outputs of the /add endpoints.

Currently, we had exactly the same output format which:

* was not exactly the ipfs API output format but was sort of similar
* made some weird concessions to be compatible (like having a string-type "size")
* was not aligned with Cluster API conventions (lowercase keys)

This corrects all this:

* The Cluster API /add output format now uses the right types and lowercase keys.
* `Hash` is now `Cid`, because the field carries a Cid.
* We copy error handling with request trailers from IPFS, and avoid carrying the
  errors in the output objects.
* The proxy now returns exactly the types as ipfs would
* We add the X-Chunked-Output: 1 header, which is custom and redundant, but
otherwise breaks js-ipfs-api integrations with the /add endpoint.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-04 00:57:36 +02:00
Hector Sanjuan
dc0b211324 Docs: Fix comment for WaitFor function
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-03 11:04:24 +02:00
Hector Sanjuan
b5631aeb8a Fix IPFS() method missing in Client interface
Seems we forgot about this one.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-01 20:09:26 +02:00
Adrian Lanzafame
000f2b3f6e api/rest: change *cid.Cid to cid.Cid in client iface
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-01 12:51:57 +02:00
Hector Sanjuan
6e83ef5f24 Fix #548: Re-enable keep-alives. Set Connection: Close when adding.
This is a workaround to have clients behave properly with the /add
endpoint by asking them to close connections when done, effectively
disabling keep-alive for this.

This means we don't need to disable keep-alives fully on all servers,
since the rest of endpoints are not affected (they are not streaming
endpoints).

Reference https://github.com/ipfs/go-ipfs/issues/5168

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-09-29 02:07:57 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Kishan Sagathiya
2cd4420ee8 Issue #446 Adding peername to PinInfo
Fixing errors and better code placement

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:20:37 +05:30
Kishan Sagathiya
773b4de1f0 Issue #446 Adding peername to PinInfo
Removed comments and code used for debugging

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30
Kishan Sagathiya
ef85ba8780 Issue #446 Adding peername to PinInfo
This commit adds peername to PinInfo and GlobalPinInfo so that we have
a nicer and more meaningfull output for `ipfs-cluster-ctl` queries like
`status`, `sync` and `recover`

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30
Kishan Sagathiya
ff48fb319a Issue #446 Adding peername to PinInfo
This commit adds peername to PinInfo and GlobalPinInfo so that we have
a nicer and more meaningfull output for queries like
`ipfs-cluster-ctl status`

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-09-24 23:04:16 +05:30