Commit Graph

400 Commits

Author SHA1 Message Date
Kishan Sagathiya
e1faf12bae ipfsproxy: hijack repo/gc and trigger cluster-wide GC
This adds hijacking of the repo/gc endpoint to the proxy to do cluster-wide gc.
2019-12-06 13:08:57 +01:00
Hector Sanjuan
ba021e8bb4 restapi: fix logging
Logging with Infof when there is nothing to format.
2019-12-02 11:56:25 +01:00
Hector Sanjuan
71217ac5d1 restapi client: support talking to unix sockets
Setting APIAddr to a Unix socket multiaddress will just work.
2019-12-02 11:56:16 +01:00
Kishan Sagathiya
a178e5988a Avoid out of index error in failover (#951)
Fixes panic when using the load balancing client.
2019-11-13 12:15:29 +01:00
Hector Sanjuan
f788a83d23 Fix lbclient test with new multiaddresses 2019-11-07 20:19:07 +01:00
Hector Sanjuan
249d9007d2 Merge branch 'master' into feat/cluster-gc 2019-11-07 18:35:42 +01:00
Kishan Mohanbhai Sagathiya
4d8ef92b3d health/graph: Improve graph
Mark local, trusted peers. Add peernames. Improve display.
2019-11-07 10:47:29 +01:00
Hector Sanjuan
e34bddd3ab Do not start Autonat when not used
Avoid chaining options as not necessary.

Add functions for common options and protector creation.
2019-11-05 12:51:18 +01:00
Kishan Mohanbhai Sagathiya
c7f37ce9f4 Use quic and use tls by default in restapi and restapi client
(cherry picked from commit 7ab1cd66ed17fda923b149628cae18ea96d1cf45)
2019-11-05 12:51:18 +01:00
Hector Sanjuan
af29bbc944 Pin-expiration: error when pinning expired pins 2019-11-05 12:22:52 +01:00
Kishan Mohanbhai Sagathiya
e4e1cbea6e Fix #481: Pin expiration
This adds a new PinOption: ExpireAt.

The StateSync ticker will check and unpin expired pins from the Cluster.

ipfs-cluster-ctl supports an "expire-in" which gives a duration.
2019-11-05 10:40:48 +01:00
Kishan Sagathiya
31534a429b Fix #374: health metrics improvements
- Human-sizes for freespace metrics. Display whether if metric is
expires in something like "expires in 3m".
- When not passing metric name `ipfs-cluster-ctl health metrics` hits
the the metrics endpoint which returns a list of available metrics and
displays to user
- Humanize metrics output
- Sort metrics output
2019-10-24 16:37:26 +02:00
Kishan Mohanbhai Sagathiya
492b5612e7 Add ability to run Garbage Collector on all peers
- cluster method, ipfs connector method, rpc and rest apis,
command, etc for repo gc
    - Remove extra space from policy generator
    - Added special timeout for `/repo/gc` call to IPFS
    - Added `RepoGCLocal` cluster rpc method, which will be used to run gc
    on local IPFS daemon
    - Added peer name to the repo gc struct
    - Sorted with peer ids, while formatting(only affects cli
    results)
    - Special timeout setting where timeout gets checked from last update
    - Added `local` argument, which would run gc only on contacted peer
2019-10-22 11:13:19 +05:30
Hector Sanjuan
582968f44e
Merge pull request #915 from ipfs/feat/proxy-logging
API logs for IPFS Proxy
2019-09-27 12:07:34 +02:00
Hector Sanjuan
ce20e86fdb
Merge pull request #907 from ipfs/feat/local-add
`--local` flag for `ctl add`
2019-09-23 13:13:59 -04:00
Hector Sanjuan
a98292bfa6
Merge pull request #893 from ipfs/feat/recover-all
Pin recover on all peers
2019-09-23 13:13:48 -04:00
Kishan Mohanbhai Sagathiya
adc7c5b131 Merge branch 'master' of github.com:ipfs/ipfs-cluster into test2 2019-09-10 20:03:05 +07:00
Kishan Mohanbhai Sagathiya
7f6c45812f Fix test 2019-09-10 15:27:47 +07:00
Kishan Mohanbhai Sagathiya
bc357400f6 API logs for IPFS Proxy 2019-09-10 13:59:03 +07:00
Hector Sanjuan
12141a562e
Merge pull request #913 from ipfs/fix/add-params
Fix: consolidate parsing of PinOptions
2019-09-09 14:52:26 +02:00
Hector Sanjuan
963bf8877f
Merge pull request #894 from ipfs/feat/http-logging
Logging for Cluster HTTP APIs
2019-09-09 10:30:01 +02:00
Hector Sanjuan
18dad223b2
Merge pull request #912 from ipfs/fix/allocations
Fix: handling allocations
2019-09-09 10:22:14 +02:00
Kishan Mohanbhai Sagathiya
9cb1cdeaff Merge branch 'master' into feat/recover-all 2019-09-08 17:06:43 +07:00
Kishan Mohanbhai Sagathiya
b1748d13c4 Use absolute path 2019-09-07 19:05:57 +07:00
Hector Sanjuan
11d5f6a3f2 Fix: consolidate parsing of PinOptions
This removes duplicated code to parse PinOptions.
2019-09-06 18:38:55 +02:00
Hector Sanjuan
96752e4e58 Fix: handling allocations
* pin() should not allocate if allocations are already provided
* pin() should not skip pinning if the exact same pin exists
  * Additionally this was unreliable as it allocated it before
    so the pin may have existed but the allocations may have been
    artificially changed.
* pin() re-uses existing pin when pin options are the same and thus
  avoids changing the allocations of a pin.

As a side effect, this fixes re-allocations which were broken: peers
called `shouldPeerRepinCid()` and instead of repinning that single
cid proceeded to repin the full state. For every pin.

Additionally tests have been adapted. It may be that some re-alloc tests
were very unreliable for the problems above.
2019-09-06 17:56:00 +02:00
Hector Sanjuan
9eb0deee4e
Merge pull request #891 from ipfs/feat/metadata
Metadata can be set and shown from `ctl`
2019-09-06 15:46:29 +02:00
Hector Sanjuan
2d389428cd Merge branch 'master' into feat/metadata 2019-09-06 15:17:05 +02:00
Hector Sanjuan
81ea1e76bc Merge branch 'master' into feat/sort-responses 2019-09-06 15:13:54 +02:00
Hector Sanjuan
f45c8544a7
Merge pull request #890 from ipfs/feat/allocations-for-add
Added UserAllocations support for `add`
2019-09-06 15:03:22 +02:00
Hector Sanjuan
d63a7fd641
Merge pull request #877 from ipfs/fix/ipfs-to-p2p
Use `p2p` protocol name over `ipfs` for multiaddr
2019-09-06 15:00:36 +02:00
Kishan Mohanbhai Sagathiya
5f57feab54 Addressed reviews on tests 2019-08-31 11:15:16 +05:30
Kishan Mohanbhai Sagathiya
db11f77396 Remove unwanted configs 2019-08-30 09:13:18 +05:30
Kishan Mohanbhai Sagathiya
6c863df9fe Merge branch 'master' of github.com:ipfs/ipfs-cluster into feat/http-logging 2019-08-29 21:37:04 +05:30
Kishan Mohanbhai Sagathiya
d43c8fb4a4 Make api logger work 2019-08-29 20:49:53 +05:30
Kishan Mohanbhai Sagathiya
49af782444 Use custom io.Writer for logging, Use go test instead of sharness tests 2019-08-28 22:13:23 +05:30
Kishan Mohanbhai Sagathiya
a684c9567d Add with option --local
This commit introduces `--local` option for `ctl add` which would add
content only the local ipfs peer and then pin it according to pin
options (fetching from the local peer)
For achieving this, a new local dag service is introduced
2019-08-28 18:39:30 +05:30
Kishan Mohanbhai Sagathiya
62a998a694 Use custom log formatter when logging to stdout
- Instead of using a boolean config element(send logs to std out or
file), use a string element that would store path of the log file.
That way user has freedom to choose the filepath. Use std out if this
path is empty.
2019-08-27 14:41:15 +05:30
Kishan Mohanbhai Sagathiya
7b35524779 Addressed reviews 2019-08-26 14:30:45 +05:30
Hector Sanjuan
9c73c2bdbd
Merge pull request #887 from ipfs/fix/small-issues
Fixes for some small issues
2019-08-24 14:24:38 +02:00
Hector Sanjuan
aab5f9bd0b Enable p2p addresses in a single place 2019-08-23 22:45:32 +02:00
Kishan Mohanbhai Sagathiya
c32281f9e0 Logging for Cluster HTTP APIs
This commit introduces logging for Cluster HTTP APIs.
It adds a config element `send_logs_to_file`, which tells whether logs
should be saved in a file or shown in standard output.

Requests are logged as per Apache Common Log Format (CLF)
http://httpd.apache.org/docs/2.2/logs.html#common

Fixes #574
2019-08-21 21:03:38 +05:30
Kishan Mohanbhai Sagathiya
512bf6a13b Pin recover on all peers
- recover works without `--local` flag as well (recovers all pins on all
peers)
- remove extra space from rpc policy

Fixes #763
2019-08-21 11:19:07 +05:30
Kishan Mohanbhai Sagathiya
e6d183fd70 Addressed reviews 2019-08-20 10:49:28 +05:30
Kishan Mohanbhai Sagathiya
0b6c376bad Metadata can be set and shown from ctl
This commit adds
- `metadata` option to `pin add` and `add` commands
- ability to set metadata from `/add` rest API
2019-08-19 10:29:06 +05:30
Kishan Mohanbhai Sagathiya
6bd897ac47 Added UserAllocations support for add 2019-08-16 18:13:06 +05:30
Kishan Mohanbhai Sagathiya
6656b80a00 Some more occurences of /ipfs
and use  SwapToP2pMultiaddrs (very helpful since ipfs still send
addresses with `/ipfs` tag)
2019-08-16 11:56:09 +05:30
Kishan Sagathiya
edd5e33a1b
Update api/rest/restapi.go 2019-08-14 19:49:15 +05:30
Hector Sanjuan
6743f92e51 Fix: do not set PinUpdate when cid is Undef
It gets set to 'b' and then prints an error when parsing.
2019-08-14 14:09:46 +02:00
Kishan Mohanbhai Sagathiya
229953f943 Merge branch 'master' of github.com:ipfs/ipfs-cluster into fix/api-improvements 2019-08-14 09:05:56 +05:30