Commit Graph

122 Commits

Author SHA1 Message Date
Hector Sanjuan
e186dbe2c2 Undo extra delays
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-02 15:24:26 +02:00
Hector Sanjuan
8b08dfeed8 Pintracker: rename and fmting.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-02 15:24:26 +02:00
Hector Sanjuan
9856bcdb94 Pintracker: remove timeouts
Pinning/unpinning timeouts are controlled by the ipfs connector component.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-02 15:24:26 +02:00
Hector Sanjuan
5709e5d03c pintracker: do not register operation after putting it in channel
This creates a race condition where the items may have been
already pinned before the operation is registered in the tracker.

This may result in operations being left in the tracker and potentially
never completed.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-05-02 15:24:26 +02:00
Adrian Lanzafame
f68c7f5354 ipfshttp: add pin/unpin specific timeouts
and get the tests passing and add Pin/UnpinQueued
tracker statuses back in.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-05-02 15:24:26 +02:00
Adrian Lanzafame
1eade86209 pintracker: add filtering of operationCtxs as they
come off the pin/unpin channels.

Also fix a race condition in the operationTracker.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-05-02 15:24:26 +02:00
Adrian Lanzafame
ab2a883a3d pintracker/mappintracker: separate status and operation concepts
The TrackerStatuses were starting to be used to convey the inflight
status of an 'operation', instead of just the status of the Pin.
I have separated out any thing related to 'operations' and
an operation's 'phases'.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-05-02 15:24:26 +02:00
Adrian Lanzafame
a2f59b26af ipfshttp/config: add ClientPostTimeout value
ipfshttp: cancel POST request when timeout reached

ipfshttp/config: fix config test

ipfshttp: use struct styling for multi-line func calls

ipfshttp/config: add general ClientTimeout

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-05-02 15:24:26 +02:00
Adrian Lanzafame
5316c3bb4c typos and style nitpicks
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-05-02 15:24:26 +02:00
Hector Sanjuan
4de7cb0ed6 Fix: Recover does not pin recursively
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-27 14:11:21 +02:00
Hector Sanjuan
f469966d02 Fix tests with bad json
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-09 17:30:06 +01:00
Hector Sanjuan
fb4812ec79 Feat #326: Adds "refs -r" pinning method support + multiple pin workers
This fixes #326. It adds a new `pin_method` configuration option to the
`ipfshttp` component allows to configure it to perform `refs -r <cid>` before
the `pin/add` call. By fetching content before pinning, we don't have
a global lock in place, and we can have several pin-requests to
ipfs in parallel.

It also adds a `concurrent_pins` option to the pin tracker, which
launches more pin workers so it can potentially trigger more pins at
the same time. This is a minimal intervention in the pintracker as #308
is still pending.

Documentation for the configuration file has been updated.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-03-09 15:01:29 +01:00
Hector Sanjuan
b013850f94 Fear #277: Add test about wanted < 0
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-19 22:24:03 +01:00
Hector Sanjuan
4549282cba Fix #277: Introduce maximum and minimum replication factor
This PR replaces ReplicationFactor with ReplicationFactorMax
and ReplicationFactor min.

This allows a CID to be pinned even though the desired
replication factor (max) is not reached, and prevents triggering
re-pinnings when the replication factor has not crossed the
lower threshold (min).

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-01-16 16:36:06 +01:00
Hector Sanjuan
e824aea55e RecoverAll: Implement RecoverAllLocal() which recovers all pins in a peer
This adds API, RPC calls to support RecoverAllLocal() (and expose RecoverLocal()
on the Rest API too). cluster-ctl is updated accordingly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-11-30 01:53:31 +01:00
Hector Sanjuan
11a8926236 MapPinTracker: support configuration section
This also generates a default configuration section when it
doesn't exist, so it's backwards compatible.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2017-11-29 14:42:50 +01:00
Hector Sanjuan
856252e5c6 Fix #88: Run SyncAllLocal() regularly on peers.
It makes a pin ls requests to ipfs and makes sure the pin tracker is
up to date.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-04-05 23:29:22 +02:00
Hector Sanjuan
8753d50c18 Increase delay in Tracker Untrack test
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-21 13:53:19 +01:00
Hector Sanjuan
e3a52e1d7d Make tests less prone to fail
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-17 17:05:22 +01:00
Hector Sanjuan
03a931b8df Fix logging
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-14 17:32:00 +01:00
Hector Sanjuan
e2efef8469 go lint, go vet, put the Consensus component behind interface.
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-14 16:37:29 +01:00
Hector Sanjuan
37fab27ba6 Move MapPinTracker to its own submodule and add tests for it
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-14 16:10:45 +01:00