4 PinTypes specify how CID is pinned
Changes to Pin and Unpin to handle different PinTypes
Tests for different PinTypes
Migration for new state format using new Pin datastructures
Visibility of the PinTypes used internally limited by default
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
1. Refactored importer endpoint, including writing cluster-specific
file adder, to get print info from importer
2. Refactored importer consumption to select equally from
channels of different output signals and manage context
timeouts correctly (only in local add here, sharding to follow)
3. Added output streaming and an error/termination handling protocol
4. Discovered that naive eager response streaming cuts off
reads from request data stream and breaks behavior, for
now all responses come after file ingestion.
5. Added ipfs add style flags (trickle, rawleaves etc.) and
refactored importer endpoint to take in these parameters
to provide identicle behavior to ipfs
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
sharding passes manual tests on single node cluster,
adding the shards of a directory and pinning the
clusterDAG to cluster/ipfs state
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
parse file path arguments to cmdkit.File objects
no support for directories, symlinks or recursive add
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
This uses the PeerAdd endpoint which should NOT be used as the current
workflow states that the way to adding peers is bootstrapping.
Adding peers manually with this endpoint leads to split-head states very
easily. The fact that this operation is visible in ipfs-cluster-ctl
is only leading the users to bad places.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
The --wait flag was being completely ignored unless --no-status was passed
too, which makes no sense because then it would print the wait status.
This waits when --wait and prints the status when --no-status is not passed.
If we have been waiting, the status comes from that. Otherwise we request it.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
address feedback and add WaitForPinnedStatus
address feedback and rework WaitFor
implement StatusFilter
address feedback and rework StatusFilter
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
Change the rest/client's and ipfs-cluster-ctl's default
timeout from 60 to 120.
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
It simply skips the status call on the globalClient, along
with the associated time.Sleep(1000), if the `--no-status`
flag is provided with the command.
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
Added go tests
Refactored cluster connect graph to new file
Refactored dot file printing to new repo
Fixed code climate issues
Added sharness test
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
added ConnectGraph type and serialization
added cli command hitting cluster api
added cluster api client method + endpoint calling into rpc
added rpc calling into main cluster component
added clustercomponent's function to collect ConnectGraph
added functionality in ipfsconn to retrieve ipfs swarm peers
added dot file printing given ConnectGraphSerial
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
The multiaddresses protocols for websockets and dns are only registered
with init() function when loading the modules. ipfs-cluster-ctl
uses just the api, which did not load these modules so converting
from serialized types caused bad panics.
We have also ignored errors in the api library under the thinking that it
would only parse things serialized by us, but this has made parsing errors
to go unnoticed. From now, all errors are logged and some precautions
are taking to better handle the possibility of nil objects.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
As before, exit status 1 means a client application error and
exit status 2 means a server-returned error.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This adds the pakage api/rest/client which implements a go-client
for the REST API component. It also update the ipfs-cluster-ctl
tool to rely on it.
Originally, I wanted this to live it in it's own separate repository,
but the api client uses /api/types.go, which is part of cluster.
Therefore it would need to import all of cluster as a dependency.
ipfs-cluster-ctl would also need to import go-ipfs-cluster-api-client
as a dependency, creating circular gx deps which would be a mess to
maintain.
Only the splitting of cluster in multiple repositories (at least for
api, rest, ipfs-cluster-ctl, rest/client and test) would allow better
dependency management by allowing rest/client and the ctl tool
to only import what is needed, but this is something which brings
maintenance costs and can probably wait a bit until cluster is more stable.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This allows to call the Rest API's status and sync endpoints with a
"?local=true" parameter. This will trigger operations but only on the
local peer. Cluster *Local and RPC-*Local methods have been accordingly,
although they are aliases for the PinTracker methods (but otherwise they
would not be exposed in external APIs). ipfs-cluster-ctl has been updated to
support the new flag.
The rationaly behind this feature is that sometimes, a single cluster peer
(or the ipfs daemon in it) is misbehaving. The user then wants to Sync,
Recover, or see Status for that single peer. This is specially relevant
when working with big pinsets in larger clusters, as a Status() call will
be considerably more expensive when broadcasted everywhere.
Note that the Rest API keeps returning GlobalPinInfo objects even on local=true
calls. This ensures that the user always gets the same datatype from an endpoint.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
URL encoding was added for `ipfs-cluster-ctl pin add --name`. The
Map State version was return to version 2. I did not notice errors,
but if there are, Version 2 still is not in a stable release.
I added an option called --name or -n which specifies the name for
the added pin to be stored in consensus. Also added quotes around
name for allocations formatter.
This adds API, RPC calls to support RecoverAllLocal() (and expose RecoverLocal()
on the Rest API too). cluster-ctl is updated accordingly.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
