Commit Graph

117 Commits

Author SHA1 Message Date
Hector Sanjuan
623120fd50 Start cluster tests
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
65dc17a78b testfixing
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Hector Sanjuan
a96241941e WIP
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-07 20:12:05 +02:00
Wyatt Daviau
1c91a99492 flag cleanup:
silent flag removed for now
quiet flag renamed 'only-hashes' and implemented

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
c4d1b34810 Tests to front end of adding pipeline
api client test
api test
one sharness test
refactoring of testingData for access from other packages
always wrap files added by cluster
remove unused flag 'progress'
changes to support hidden flag

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:24 +02:00
Wyatt Daviau
1704295331 Refactor add & general cleanup
addFile function is now a Cluster method accessed by RPC
residue from attempting to stream responses removed
ipfs-cluster-ctl ls bug fixed
problem with importer/add not printing resolved
new test now checks for this

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
32df2ce224 support for replication factors during sharding
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
238f3726f3 Pin datastructure updated to support sharding
4 PinTypes specify how CID is pinned
Changes to Pin and Unpin to handle different PinTypes
Tests for different PinTypes
Migration for new state format using new Pin datastructures
Visibility of the PinTypes used internally limited by default

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
6b8bcdfbc3 Finishing feedback and adding new test
Also updating Makefile to fetch sharness more robustly

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
9f74f6f47d Addressing third round of comments
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
ae38d09208 Beginning add UX improvements--
1. Refactored importer endpoint, including writing cluster-specific
file adder, to get print info from importer
2. Refactored importer consumption to select equally from
channels of different output signals and manage context
timeouts correctly (only in local add here, sharding to follow)
3. Added output streaming and an error/termination handling protocol
4. Discovered that naive eager response streaming cuts off
reads from request data stream and breaks behavior, for
now all responses come after file ingestion.
5. Added ipfs add style flags (trickle, rawleaves etc.) and
refactored importer endpoint to take in these parameters
to provide identicle behavior to ipfs

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
77a61890ff Sharding rough draft:
sharding passes manual tests on single node cluster,
adding the shards of a directory and pinning the
clusterDAG to cluster/ipfs state

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
5725e34e3d polishing up print on restapi
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
9e3e42b19d client streams files
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:23 +02:00
Wyatt Daviau
9521885b63 ipfs-cluster-ctl add
parse file path arguments to cmdkit.File objects
no support for directories, symlinks or recursive add

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-08-07 20:11:13 +02:00
ia
3c1bd32e8b all: gofmt
Run standard gofmt command on project root.

License: MIT
Signed-off-by: Isaac Henri Ardis <isaac.ardis@gmail.com>
2018-06-14 11:55:03 +02:00
Hector Sanjuan
0889302c8d
Release 0.4.0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-30 13:26:10 +02:00
Hector Sanjuan
5b114383e6
Release 0.4.0-rc2
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-28 12:45:10 +02:00
Michael Muré
047fa48ec9
Makefile: add a source target to properly trigger a build on source change
Fix #425

License: MIT
Signed-off-by: Michael Muré <batolettre@gmail.com>
2018-05-18 01:46:19 +02:00
Hector Sanjuan
16c602e5c7
Release 0.4.0-rc1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-14 11:56:38 +02:00
Hector Sanjuan
a0a0898719
Merge pull request #393 from ipfs/docs/move-to-website
Docs/move to website
2018-05-01 11:14:17 +02:00
Hector Sanjuan
1d20f3de36 ipfs-cluster-ctl: do not provide "peers add"
This uses the PeerAdd endpoint which should NOT be used as the current
workflow states that the way to adding peers is bootstrapping.

Adding peers manually with this endpoint leads to split-head states very
easily. The fact that this operation is visible in ipfs-cluster-ctl
is only leading the users to bad places.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-04-27 10:27:52 +02:00
Hector Sanjuan
b08b3aba64 Docs: Move to website.
Updated READMEs, removed docs and point everything to website documentation.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-04-27 09:05:11 +02:00
Hector Sanjuan
ed55daf2c6
Release 0.3.5
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-29 13:43:36 +02:00
Hector Sanjuan
41433298f5 Fix: --wait flag. Address comments
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-29 11:25:15 +02:00
Hector Sanjuan
018e5dbaea Fix: --wait flag in ipfs-cluster-ctl
The --wait flag was being completely ignored unless --no-status was passed
too, which makes no sense because then it would print the wait status.

This waits when --wait and prints the status when --no-status is not passed.

If we have been waiting, the status comes from that. Otherwise we request it.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-29 11:23:40 +02:00
Adrian Lanzafame
4d48504779 rest/client: Add WaitFor utility function
address feedback and add WaitForPinnedStatus

address feedback and rework WaitFor

implement StatusFilter

address feedback and rework StatusFilter

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-03-27 14:56:49 +10:00
Hector Sanjuan
72876465d6
Release 0.3.5-rc1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-26 21:50:18 +02:00
Hector Sanjuan
09f4c9fce3 rest/libp2p-http: address lanzafame's review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-20 19:35:42 +01:00
Hector Sanjuan
23d2db15e7 ipfs-cluster-ctl: fix sharness
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-16 19:37:52 +01:00
Hector Sanjuan
73581b0471 ipfs-cluster-ctl: Allow libp2p multiaddress as --host. Use libp2p in that case.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-15 17:15:23 +01:00
Adrian Lanzafame
7715eff0f6 rest/client: ipfs-cluster-ctl: double default timeout
Change the rest/client's and ipfs-cluster-ctl's default
timeout from 60 to 120.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-03-10 11:20:56 +10:00
Adrian Lanzafame
c4468bab49 ipfs-cluster-ctl cmd: add no-status flag to pin add/rm
It simply skips the status call on the globalClient, along
with the associated time.Sleep(1000), if the `--no-status`
flag is provided with the command.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-03-07 11:46:40 +10:00
Hector Sanjuan
2e9bcec005
Release 0.3.4
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-02-20 20:25:44 +01:00
Hector Sanjuan
9f30638784
Release 0.3.3
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-02-12 21:13:35 +01:00
Wyatt Daviau
8f8c3afbd0 fix conn graph failures
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-02-01 17:29:14 -05:00
Hector Sanjuan
a180f1a5c5
Merge pull request #291 from ipfs/feat/connectivity-graph
Feat/connectivity graph
2018-01-26 12:42:18 +01:00
Hector Sanjuan
62cd18386c
Release 0.3.2
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-26 10:38:26 +01:00
ZenGround0
4b26ccd144
Merge branch 'master' into feat/connectivity-graph 2018-01-23 08:34:43 -05:00
Wyatt Daviau
78f14b6d2a addressing more comments
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-01-23 08:33:12 -05:00
Wyatt Daviau
d2ef32f48f Testing and polishing connection graph
Added go tests
Refactored cluster connect graph to new file
Refactored dot file printing to new repo
Fixed code climate issues
Added sharness test

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-01-22 10:03:37 -05:00
Wyatt
e712c87570 First draft of ConnectGraph collection:
added ConnectGraph type and serialization
added cli command hitting cluster api
added cluster api client method + endpoint calling into rpc
added rpc calling into main cluster component
added clustercomponent's function to collect ConnectGraph
added functionality in ipfsconn to retrieve ipfs swarm peers
added dot file printing given ConnectGraphSerial

License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
2018-01-22 09:07:12 -05:00
Hector Sanjuan
dcfc962f24 Feat #277: Address review comments
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-19 22:24:03 +01:00
Hector Sanjuan
b013850f94 Fear #277: Add test about wanted < 0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-19 22:24:03 +01:00
Hector Sanjuan
b12138f7ff Fix #277: Add support for replication-min and replication-max in ctl
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-19 22:23:34 +01:00
Hector Sanjuan
8eea4baf3d Fix #178: Document exit status for ipfs-cluster-ctl
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-12 16:58:56 +01:00
Hector Sanjuan
91b3763869
Release 0.3.1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-11 14:11:10 +01:00
Hector Sanjuan
d6bb5bb538 cluster-ctl: use standard error types
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-08 17:05:21 +01:00
Hector Sanjuan
6a243df4da api: Support /ws/ and /dns/ multiaddresses parsing. Log all errors
The multiaddresses protocols for websockets and dns are only registered
with init() function when loading the modules. ipfs-cluster-ctl
uses just the api, which did not load these modules so converting
from serialized types caused bad panics.

We have also ignored errors in the api library under the thinking that it
would only parse things serialized by us, but this has made parsing errors
to go unnoticed. From now, all errors are logged and some precautions
are taking to better handle the possibility of nil objects.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-08 01:05:49 +01:00
Hector Sanjuan
0c469b85e1 cluster-ctl: exit with error on errors.
As before, exit status 1 means a client application error and
exit status 2 means a server-returned error.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-12-07 12:00:28 +01:00