The rest/client does things like "return id.ToID(), err" which is nice.
But when there's an error the ID has Cid set to nil etc. This is now
causing error messages on the console which are very misleading because
they do not correspond to the actual error of the request.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
address feedback and add WaitForPinnedStatus
address feedback and rework WaitFor
implement StatusFilter
address feedback and rework StatusFilter
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
The IPFS() methods returns an ipfs Shell pointing to the ipfs-cluster
proxy endpoint. The location can be customized (via ProxyAddr configuration
option) or it is assumed to be the same as the PeerAddr/APIAddr, with
a different port (the default).
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This adds support for libp2p-tunneled http to the rest api component.
If PeerAddr is specified in the configuration, then we will create a
libp2p host and communicate with the API using that.
Tests run now in both http and libp2p mode.
Note: pnet support not included, but coming up
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This commits allows restapi to serve/tunnel http on a libp2p stream.
NewWitHost(...) allows to provide a libp2p host during initialization
which is then used to obtain a listener with go-libp2p-gostream.
Alternatively, if the configuration provides an ID, PrivateKey and Libp2pListenAddr,
a host is created directly by us and used to get the listener.
The protocol tag used is provided by the p2phttp library which will
be used by the client.
All tests now run against the libp2p node too.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This adds support for parameters to create a libp2p host
in the REST API configuration: ID, PrivateKey and ListenMultiaddr.
These parameters default to nil/empty and are ommited in the default
configuration. They are only supposed to be used when the user wants
the REST API to use a different libp2p host than a provided one (upcoming
changes).
Pnet protector not supported yet in this case. Underlying basic auth
should cover that front. Will implement if someone has a usecase.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
Change the rest/client's and ipfs-cluster-ctl's default
timeout from 60 to 120.
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
Added go tests
Refactored cluster connect graph to new file
Refactored dot file printing to new repo
Fixed code climate issues
Added sharness test
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
added ConnectGraph type and serialization
added cli command hitting cluster api
added cluster api client method + endpoint calling into rpc
added rpc calling into main cluster component
added clustercomponent's function to collect ConnectGraph
added functionality in ipfsconn to retrieve ipfs swarm peers
added dot file printing given ConnectGraphSerial
License: MIT
Signed-off-by: Wyatt Daviau <wdaviau@cs.stanford.edu>
This ensures that we don't re-pin something which is already correctly
pinned with the same allocations.
It also ensures that we do re-pin something when the replication
factor associated to it changes.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This PR replaces ReplicationFactor with ReplicationFactorMax
and ReplicationFactor min.
This allows a CID to be pinned even though the desired
replication factor (max) is not reached, and prevents triggering
re-pinnings when the replication factor has not crossed the
lower threshold (min).
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
Raft will fail to take a snapshot when applied index is
different from the last index. Therefore, we wait for
all updates to be aplied before snapshotting.
If still it doesn't work, we retry a few times.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
We had a problem happening when assigning the returned *api.Error
to default 'error' type.
Things like "if err != nil" would not work even when *api.Error is nil
I'm not sure why this happens, but this is very confusing for the user
integrating on top. It is better that we just return plain go errors.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
The multiaddresses protocols for websockets and dns are only registered
with init() function when loading the modules. ipfs-cluster-ctl
uses just the api, which did not load these modules so converting
from serialized types caused bad panics.
We have also ignored errors in the api library under the thinking that it
would only parse things serialized by us, but this has made parsing errors
to go unnoticed. From now, all errors are logged and some precautions
are taking to better handle the possibility of nil objects.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
Apparently, cancelling the request context closes the response body
prematurely, before it's being fully read.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This allows taking advantage of connection keep alive by having the
api client re-use the same connection. Additionally, an option
to close connections after every request is provided.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
* Set default logging facility
* Remove old keep-alive comment in tests
* Use a port for TestPeersWithErrors which is not default
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
This adds the pakage api/rest/client which implements a go-client
for the REST API component. It also update the ipfs-cluster-ctl
tool to rely on it.
Originally, I wanted this to live it in it's own separate repository,
but the api client uses /api/types.go, which is part of cluster.
Therefore it would need to import all of cluster as a dependency.
ipfs-cluster-ctl would also need to import go-ipfs-cluster-api-client
as a dependency, creating circular gx deps which would be a mess to
maintain.
Only the splitting of cluster in multiple repositories (at least for
api, rest, ipfs-cluster-ctl, rest/client and test) would allow better
dependency management by allowing rest/client and the ctl tool
to only import what is needed, but this is something which brings
maintenance costs and can probably wait a bit until cluster is more stable.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
I've modified the peer identifier to be 'peername'. I've also
modified the TestLoadJSON to check that it is correctly read from
config and set to a default if empty.
Also added 'peername' fields to configurations for various tests.
This allows to call the Rest API's status and sync endpoints with a
"?local=true" parameter. This will trigger operations but only on the
local peer. Cluster *Local and RPC-*Local methods have been accordingly,
although they are aliases for the PinTracker methods (but otherwise they
would not be exposed in external APIs). ipfs-cluster-ctl has been updated to
support the new flag.
The rationaly behind this feature is that sometimes, a single cluster peer
(or the ipfs daemon in it) is misbehaving. The user then wants to Sync,
Recover, or see Status for that single peer. This is specially relevant
when working with big pinsets in larger clusters, as a Status() call will
be considerably more expensive when broadcasted everywhere.
Note that the Rest API keeps returning GlobalPinInfo objects even on local=true
calls. This ensures that the user always gets the same datatype from an endpoint.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
I renamed Hostname to simply Name as to not imply relation to DNS.
Removed quotes from formatter, used helper function setting config,
and added defensive error check.
