Commit Graph

31 Commits

Author SHA1 Message Date
Kishan Sagathiya
e9cf8abb3a Added tests for /monitor/metrics/{metrics_type}
Added API and client tests for GET /monitor/metrics/{metrics_type}

Fixes #587

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2018-12-21 08:24:40 +05:30
Hector Sanjuan
11cc0485c7 API/client: Do not clean folders mid-test.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:55:34 +01:00
Hector Sanjuan
15413edbc6 Unrewrite deps
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-12-17 14:16:38 +01:00
Łukasz Magiera
406d2b53f8 files2.0: Close subfiles in adder
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-12-17 14:16:38 +01:00
Hector Sanjuan
1d924bfc49 Remove shard testing folders
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-11-01 19:15:08 +01:00
Hector Sanjuan
4e3553fb1f Fix: escape the add query parameters correctly in the client
When adding something that has a name with spaces, things fail
very badly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-19 20:19:52 +02:00
Hector Sanjuan
7d16108751 Start using libp2p/go-libp2p-gorpc
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-17 15:28:03 +02:00
Hector Sanjuan
b6306a6e61 Add endpoints output: improvements and compliance
This straigthens some mistakes with the outputs of the /add endpoints.

Currently, we had exactly the same output format which:

* was not exactly the ipfs API output format but was sort of similar
* made some weird concessions to be compatible (like having a string-type "size")
* was not aligned with Cluster API conventions (lowercase keys)

This corrects all this:

* The Cluster API /add output format now uses the right types and lowercase keys.
* `Hash` is now `Cid`, because the field carries a Cid.
* We copy error handling with request trailers from IPFS, and avoid carrying the
  errors in the output objects.
* The proxy now returns exactly the types as ipfs would
* We add the X-Chunked-Output: 1 header, which is custom and redundant, but
otherwise breaks js-ipfs-api integrations with the /add endpoint.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-10-04 00:57:36 +02:00
Adrian Lanzafame
000f2b3f6e api/rest: change *cid.Cid to cid.Cid in client iface
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-10-01 12:51:57 +02:00
Joseph Lukasik
b98e549d92 Add Client interface, implemented by defaultClient
License: MIT
Signed-off-by: Joseph Lukasik <joseph@jgl.me>
2018-09-27 16:51:34 -07:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
bbf707d61e Fix allocations filter option
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:44 +02:00
Hector Sanjuan
50fc3c4e95 Address comments from review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05 Fix: close files. Close test files. Clean properly. Fix windows tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
3575f05753 Support adding Output (api.AddedOutput)
This was a long FIXME/TODO. Handling adding output and
reporting to the client of the progress of the adding process.

This attempts to do it. It is not sure that it works correctly
(response body being written while the multipart request is still being read)

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
8f1a15b279 Move adder.Params to api.AddParams. Re-use in other modules
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
65dc17a78b testfixing
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Wyatt Daviau
1c91a99492 flag cleanup:
silent flag removed for now
quiet flag renamed 'only-hashes' and implemented

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
2bc295ca9f address comments:
rebase, cleanup and separate tests,
AddParams type replaces map of strings

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
339532b5d1 tests up to date with rebase
sharness tests added

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
c4d1b34810 Tests to front end of adding pipeline
api client test
api test
one sharness test
refactoring of testingData for access from other packages
always wrap files added by cluster
remove unused flag 'progress'
changes to support hidden flag

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
238f3726f3 Pin datastructure updated to support sharding
4 PinTypes specify how CID is pinned
Changes to Pin and Unpin to handle different PinTypes
Tests for different PinTypes
Migration for new state format using new Pin datastructures
Visibility of the PinTypes used internally limited by default

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Hector Sanjuan
d3d1f960f5 Feat: Enable DHT-based peer discovery and routing for cluster peers
This uses go-libp2p-kad-dht as routing provider for the Cluster Peers.

This means that:

* A cluster peer can discover other Cluster peers even if they are
not in their peerstore file.
* We remove a bunch of code sending and receiving peers multiaddresses
when a new peer was added to the Cluster.
* PeerAdd now takes an ID and not a multiaddress. We do not need to
ask the new peer which is our external multiaddress nor broadcast
the new multiaddress to everyone. This will fix problems when bootstrapping
a new peer to the Cluster while not all the other peers are online.
* Adding a new peer does not mean to open connections to all peers
anymore. The number of connections will be made according to the DHT
parameters (this is good to have for future work)

The that detecting a peer addition in the watchPeers() function does
no longer mean that we have connected to it or that we know its
multiaddresses. Therefore it's no point to save the peerstore in these
events anymore.

Here a question opens, should we save the peerstore at all, and should we
save multiaddresses only for cluster peers, or for everyone known?
Currently, the peerstore is only updated on clean shutdown,
and it is updated with all the multiaddresses known, and not limited to
peer IDs in the cluster, (because, why not).

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-07-24 15:33:41 +02:00
Hector Sanjuan
41433298f5 Fix: --wait flag. Address comments
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-29 11:25:15 +02:00
Hector Sanjuan
851e78ad02 Add test for WaitFor
Actually test that WaitFor waits

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-29 11:23:40 +02:00
Hector Sanjuan
4ea873c297 api/rest/client: run endpoint tests in parallel
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-21 09:51:20 +01:00
Hector Sanjuan
bbe2407531 rest/client: add support for libp2p-tunneled http.
This adds support for libp2p-tunneled http to the rest api component.

If PeerAddr is specified in the configuration, then we will create a
libp2p host and communicate with the API using that.

Tests run now in both http and libp2p mode.

Note: pnet support not included, but coming up

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-15 17:14:22 +01:00
ZenGround0
4b26ccd144
Merge branch 'master' into feat/connectivity-graph 2018-01-23 08:34:43 -05:00
Wyatt Daviau
d2ef32f48f Testing and polishing connection graph
Added go tests
Refactored cluster connect graph to new file
Refactored dot file printing to new repo
Fixed code climate issues
Added sharness test

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-01-22 10:03:37 -05:00
Hector Sanjuan
5f45ba8bce Fix #277: Add min a max replication factor to rest API
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-16 16:36:06 +01:00
Hector Sanjuan
b2651b52ea Refactor some stuff to please codeclimate
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-08 17:05:21 +01:00