Commit Graph

1197 Commits

Author SHA1 Message Date
Hector Sanjuan
d900ff8ff5
Merge pull request #509 from ipfs/feat/client-add
rest/client: Implement .Add(paths), which adds from local files and web files
2018-08-20 12:41:15 +02:00
Hector Sanjuan
2061029313 Fix hanging add in sharness.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 12:08:49 +02:00
Hector Sanjuan
6c28d6bf3e Fix PinLsCid and pins with maxdepth 0 after migration
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 11:39:34 +02:00
Hector Sanjuan
b6182e0621 rest/client: Implement .Add(paths), which adds from local files and web files
.Add(paths) will interpret http* paths as WebFiles. These are read performing
a GET request to the location. Otherwise, the path is interpreted as a local
disk file/folder, and read from disk. ipfs-cluster-ctl has been updated
accordingly.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-20 11:07:23 +02:00
Hector Sanjuan
63fa1ba213
Merge pull request #505 from ipfs/fix/495-proxy-add
Fix #495: Hijack proxy /add correctly. Disable keep-alives.
2018-08-20 10:29:20 +02:00
Hector Sanjuan
77df433c2d
Merge pull request #504 from ipfs/fix/pinls
IPFSPinLsCid should try to find pins in the direct pinset too.
2018-08-20 10:10:32 +02:00
Hector Sanjuan
8d3510e98a
Merge pull request #507 from ipfs/fix/fast-repo-stat
IPFSHttp: Add "size-only=true" flag to repo/stat calls
2018-08-20 10:10:05 +02:00
Hector Sanjuan
2f59eaf966
Merge pull request #503 from ipfs/fix/sharding-state-upgrade
State migrations for sharding
2018-08-20 10:09:49 +02:00
Hector Sanjuan
5d991d651b
Merge pull request #508 from ipfs/feat/fix-testclusterspeerrejoin
Fix: TestClustersPeerRejoin was failing a lot since DHT feature.
2018-08-20 10:09:34 +02:00
Hector Sanjuan
be651da0d0 Fix: disable KeepAlives in server
See discussion in https://github.com/ipfs/go-ipfs/issues/5168

We cannot stream responses with keep-alives enabled.

I prefer this to not be a client feature, as otherwise users might end up
shooting themselves in the foot.

Note, the price is a corrupted request body which gets added
normally and gives wrong hashes!

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:36 +02:00
Hector Sanjuan
0709652d38 Run sharness tests verbose
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
ff8476e9b7 Fix: TestClustersPeerRejoin was failing a lot since DHT feature.
This uses Join rather than PeerAdd. Join ensures a DHT bootstrap
after successfully joining.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
00a757fc0b Proxy/add: flush AddedOutput
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
b2914c2516 Fix #495: Hijack proxy /add correctly.
No more hacks around /add. This uses the local adder when hijacking /add.
It supports the parameters and works pretty well with the ipfs CLI, showing
progress and everything.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-18 02:48:35 +02:00
Hector Sanjuan
eba96c3a26 Merge branch 'feat/fix-testclusterspeerrejoin' into fix/pinls
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-17 23:07:09 +02:00
Hector Sanjuan
5a0d4ed9a2 Merge branch 'feat/fix-testclusterspeerrejoin' into fix/fast-repo-stat
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-17 23:04:26 +02:00
Hector Sanjuan
c8e713134f Fix: TestClustersPeerRejoin was failing a lot since DHT feature.
This uses Join rather than PeerAdd. Join ensures a DHT bootstrap
after successfully joining.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-17 22:54:53 +02:00
Hector Sanjuan
ee8d7f7098 IPFSHttp: Add "size-only=true" flag to repo/stat calls
This will make them very fast.

Fixes #280

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-17 16:53:40 +02:00
Hector Sanjuan
0f2a978bcd
Merge pull request #458 from ipfs/fix/340-races
Issue #340: fix some data races
2018-08-17 10:55:23 +02:00
Hector Sanjuan
e66a68dbef State migrations for sharding
This adds state migration to new state format version 5.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-15 13:28:24 +02:00
Hector Sanjuan
2e7e71d13f IPFSPinLsCid should try to find pins in the direct pinset too.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-15 13:19:39 +02:00
Hector Sanjuan
5bbc699bb4 Issue #340: Fix some data races
Unfortunately, there are still some data races in yamux
https://github.com/libp2p/go-libp2p/issues/396 so we can't
enable this by default.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-15 12:27:01 +02:00
Hector Sanjuan
e86c2e6c59
Merge pull request #460 from ipfs/statelesstracker
implements a stateless pintracker
2018-08-15 11:04:38 +02:00
Hector Sanjuan
949abb25f0 Align stateless tracker with sharding
Also: Fixes #500

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 20:51:01 +02:00
Hector Sanjuan
10fa7a13b5 Allow selecting pintracker with ipfs-cluster-service
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 19:57:14 +02:00
Hector Sanjuan
b8194143c2 Test with stateless tracker. Make map tracker default for tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 19:18:26 +02:00
Hector Sanjuan
95683ac8f0 Remove unreachable code
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 18:38:38 +02:00
Hector Sanjuan
584f532bb1 Fix tests after rebase to master
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 15:18:16 +02:00
Adrian Lanzafame
c6eada9db5 uses new gorpc method to distinguish err type
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:34:02 +02:00
Adrian Lanzafame
33f56e8867 stateless: reduce num of methods
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:33:39 +02:00
Adrian Lanzafame
bec77546ce optracker: fix complexity of filter fn
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:33:39 +02:00
Adrian Lanzafame
df2753dfc6 implements a stateless pintracker
Also updates to the optracker to make retrieving information easier.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-08-14 13:33:36 +02:00
Hector Sanjuan
505794707d
Merge pull request #484 from ipfs/feat/sharding-v1
Sharding: support adding content directly to IPFS Cluster
2018-08-14 13:29:25 +02:00
Hector Sanjuan
df768b0b17 Align go-libp2p-kad-dht version
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 12:09:13 +02:00
Hector Sanjuan
1b8967aa46 Merge branch 'master' into feat/sharding-v1
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-14 11:40:03 +02:00
Hector Sanjuan
2675bf8040
Merge pull request #489 from ipfs/feat/dht
Feat: Enable DHT-based peer discovery and routing for cluster peers
2018-08-14 11:34:35 +02:00
Hector Sanjuan
9baffd3d4f
Merge pull request #498 from ipfs/fix/client-resolve-panic
Feat/rest/client: auto-handling of libp2p endpoints
2018-08-14 11:33:58 +02:00
Hector Sanjuan
f4455d3733 Address comments
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 11:56:16 +02:00
Hector Sanjuan
c1363f3ad9 rest/client: clarify comment about overriding ssl configs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-13 10:28:25 +02:00
Hector Sanjuan
522791df8d Feat/rest/client: auto-handling of libp2p endpoints
This removes PeerAddr and uses APIAddr directly, figuring out if it is
a Peer multiaddress or not.

PeerAddr is actually kept for compatiblity.

It also fixes a bad panic when resolving returned 0 results

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 16:29:41 +02:00
Hector Sanjuan
f2dbd625a3 style
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:53:46 +02:00
Hector Sanjuan
bbf707d61e Fix allocations filter option
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:44 +02:00
Hector Sanjuan
7b826a5335 Dependency upgrades
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 14:39:34 +02:00
Hector Sanjuan
f707621a39 Allow to use adder only once
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-10 11:51:27 +02:00
Hector Sanjuan
3c80169174 dag -> dgs
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:24:10 +02:00
Hector Sanjuan
50fc3c4e95 Address comments from review
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 13:22:47 +02:00
Hector Sanjuan
c2312cbb05 Fix: close files. Close test files. Clean properly. Fix windows tests.
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 12:05:27 +02:00
Hector Sanjuan
26cbecca4c Fix tests
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 03:42:39 +02:00
Hector Sanjuan
031ff7183b Fix sharness
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 02:41:30 +02:00
Hector Sanjuan
78b8f47c14 Fix: Wrap-in-directory
Usually we had wrap-in-directory enabled by default because otherwise
we had an error when adding single, non-directory files.

wrap-in-directory happens automatically when adding more than one file
so that was no problem. Thigns also worked when adding a folder and Wrap was
disabled. The only case was adding a single with wrap disabled (a default option).

This patches the ipfsadd/add.go file to remember the last added file so that
we can use it's Cid as the resulting root of the adding process without
having to fetch it from our dummy dagservice.

We have to pass this CID to our Finalize() functions, because it turns out that
in this case (single file without wrap-in-directory), the last block added to the DAG
is not the IPFS root (the ipfsadd/Adder adds the mfs root folder last always).

This was the case when wrap-in-directory was enabled by default.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2018-08-09 01:16:30 +02:00