Commit Graph

152 Commits

Author SHA1 Message Date
Hector Sanjuan
4ea830f74e Feat: ipfs-cluster-follow
This adds a new cluster command: ipfs-cluster-follow.

This command allows initializing and running follower peers as configured by a
remote-source configuration. The command can list configured peers
and obtain information for each of them.

Peers are launched with the rest API listening on a local unix socket. The
command can be run to list the items in the cluster pinset using this
endpoint. Alternatively, if no socket is present, the peer will be assumed to
be offline and the pin list will be directly read from the datastore.

Cluster peers launched with this command (and their configurations) are
compatible with ipfs-cluster-ctl and ipfs-cluster-service. We purposely do not
support most configuration options here. Using ipfs-cluster-ctl or launching
the peers using ipfs-cluster-service is always an option when the usecase
deviates from that supported by ipfs-cluster-follow.

Examples:

$ ipfs-cluster-follow -> list configured peers
$ ipfs-cluster-follow --help
$ ipfs-cluster-follow <clusterName> init <url>
$ ipfs-cluster-follow <clusterName> info
$ ipfs-cluster-follow <clusterName> run
$ ipfs-cluster-follow <clusterName> list
2019-12-07 15:38:59 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils
Service: extract some methods to cmdutils
2019-12-06 11:08:58 +01:00
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt
Fix: disable default libp2p API endpoint with CRDT consensus
2019-12-05 16:14:35 +01:00
Kishan Sagathiya
30fd5ee6a0 Feat: Allow and run multiple informer components (#962)
This introduces the possiblity of running Cluster with multiple informer components. The first one in the list is the used for "allocations". This is just groundwork for working with several informers in the future.
2019-12-05 15:08:43 +01:00
Hector Sanjuan
f6ea95bccf Service/cmdutils: fix small typos (review) 2019-12-02 16:08:47 +01:00
Hector Sanjuan
5cc9ed0c4f ipfs-cluster-service: extract some helpers to cmdutils
This will allow better re-use from other CLIs.
2019-12-02 11:57:24 +01:00
Hector Sanjuan
454efe83fd ipfs-cluster-service: rename cluster node -> cluster peer in help.
We use "peer" everywhere. This is an old leftover.
2019-12-02 11:57:13 +01:00
Hector Sanjuan
578ee29bdf Fix: disable default libp2p API endpoint with CRDT consensus 2019-12-02 11:54:51 +01:00
Hector Sanjuan
7c5d86ee86 Fix graph not displaying all ipfs nodes when all-ipfs-peers enabled 2019-11-08 12:48:02 +01:00
Hector Sanjuan
f9b52e02b9 Fix graph tests failing and randomly failing 2019-11-07 20:18:50 +01:00
Hector Sanjuan
249d9007d2 Merge branch 'master' into feat/cluster-gc 2019-11-07 18:35:42 +01:00
Hector Sanjuan
5f265b017a Change colors and shapes 2019-11-07 12:49:51 +01:00
Kishan Mohanbhai Sagathiya
4d8ef92b3d health/graph: Improve graph
Mark local, trusted peers. Add peernames. Improve display.
2019-11-07 10:47:29 +01:00
Kishan Mohanbhai Sagathiya
56ef75b50c Use TLS instead of secio for security 2019-11-05 12:50:46 +01:00
Hector Sanjuan
dcb6c0805b
Merge pull request #946 from ipfs/dockerfiles-improvs
Dockerfiles: download modules in advance
2019-11-05 12:27:51 +01:00
Hector Sanjuan
af29bbc944 Pin-expiration: error when pinning expired pins 2019-11-05 12:22:52 +01:00
Kishan Mohanbhai Sagathiya
e4e1cbea6e Fix #481: Pin expiration
This adds a new PinOption: ExpireAt.

The StateSync ticker will check and unpin expired pins from the Cluster.

ipfs-cluster-ctl supports an "expire-in" which gives a duration.
2019-11-05 10:40:48 +01:00
Hector Sanjuan
073a0dfa58 Dockerfiles: download modules in advance
This should speed up Dockerfile builds, particularly when building after
updating the code since the modules will not be to be re-downloaded.
2019-10-28 10:23:18 +01:00
Kishan Sagathiya
7b499263af service init --randomports:
Allow initialize cluster with random ports that are ununsed (at the moment of initialization)
2019-10-24 17:54:50 +02:00
Kishan Sagathiya
31534a429b Fix #374: health metrics improvements
- Human-sizes for freespace metrics. Display whether if metric is
expires in something like "expires in 3m".
- When not passing metric name `ipfs-cluster-ctl health metrics` hits
the the metrics endpoint which returns a list of available metrics and
displays to user
- Humanize metrics output
- Sort metrics output
2019-10-24 16:37:26 +02:00
Kishan Mohanbhai Sagathiya
492b5612e7 Add ability to run Garbage Collector on all peers
- cluster method, ipfs connector method, rpc and rest apis,
command, etc for repo gc
    - Remove extra space from policy generator
    - Added special timeout for `/repo/gc` call to IPFS
    - Added `RepoGCLocal` cluster rpc method, which will be used to run gc
    on local IPFS daemon
    - Added peer name to the repo gc struct
    - Sorted with peer ids, while formatting(only affects cli
    results)
    - Special timeout setting where timeout gets checked from last update
    - Added `local` argument, which would run gc only on contacted peer
2019-10-22 11:13:19 +05:30
Hector Sanjuan
ce20e86fdb
Merge pull request #907 from ipfs/feat/local-add
`--local` flag for `ctl add`
2019-09-23 13:13:59 -04:00
Kishan Mohanbhai Sagathiya
fe57e33726 Addressed reviews 2019-09-21 14:07:26 +05:30
Hector Sanjuan
3c4859c74c
Release 0.11.0 2019-09-13 18:57:25 +02:00
Hector Sanjuan
d21dffa923
Release 0.11.0-rc10 2019-09-13 10:52:13 +02:00
Kishan Mohanbhai Sagathiya
adc7c5b131 Merge branch 'master' of github.com:ipfs/ipfs-cluster into test2 2019-09-10 20:03:05 +07:00
Hector Sanjuan
cf189799f2
Release 0.11.0-rc9 2019-09-09 14:54:52 +02:00
Hector Sanjuan
4e75bee03d
Merge pull request #909 from ipfs/fix/rpl-confusion
Explanation of 0 replication factor
2019-09-09 10:31:18 +02:00
Hector Sanjuan
2d389428cd Merge branch 'master' into feat/metadata 2019-09-06 15:17:05 +02:00
Hector Sanjuan
f45c8544a7
Merge pull request #890 from ipfs/feat/allocations-for-add
Added UserAllocations support for `add`
2019-09-06 15:03:22 +02:00
Hector Sanjuan
d63a7fd641
Merge pull request #877 from ipfs/fix/ipfs-to-p2p
Use `p2p` protocol name over `ipfs` for multiaddr
2019-09-06 15:00:36 +02:00
Kishan Mohanbhai Sagathiya
7155fa87c3 Explanation of 0 replication factor 2019-09-02 09:22:50 +05:30
Kishan Mohanbhai Sagathiya
a684c9567d Add with option --local
This commit introduces `--local` option for `ctl add` which would add
content only the local ipfs peer and then pin it according to pin
options (fetching from the local peer)
For achieving this, a new local dag service is introduced
2019-08-28 18:39:30 +05:30
Hector Sanjuan
1cfea47696
Release 0.11.0-rc8 2019-08-27 12:55:41 +02:00
Kishan Mohanbhai Sagathiya
cd86211766 Sharness test, dont print metadata in cmd 2019-08-27 09:32:34 +05:30
Hector Sanjuan
46cd5327fa
Release 0.11.0-rc7 2019-08-26 14:58:49 +02:00
Kishan Mohanbhai Sagathiya
7b35524779 Addressed reviews 2019-08-26 14:30:45 +05:30
Hector Sanjuan
33f111c44d mDNS: attach mDNS inside the Cluster. Allow interval configuration.
Setting up mDNS outside the Cluster is dirtier and allows less configuration.

This adds MDNSInterval to the cluster config options and allow disabling it
when the option is set to 0.
2019-08-24 17:24:18 +02:00
Hector Sanjuan
aab5f9bd0b Enable p2p addresses in a single place 2019-08-23 22:45:32 +02:00
Kishan Mohanbhai Sagathiya
0b6c376bad Metadata can be set and shown from ctl
This commit adds
- `metadata` option to `pin add` and `add` commands
- ability to set metadata from `/add` rest API
2019-08-19 10:29:06 +05:30
Kishan Mohanbhai Sagathiya
6bd897ac47 Added UserAllocations support for add 2019-08-16 18:13:06 +05:30
Kishan Mohanbhai Sagathiya
6656b80a00 Some more occurences of /ipfs
and use  SwapToP2pMultiaddrs (very helpful since ipfs still send
addresses with `/ipfs` tag)
2019-08-16 11:56:09 +05:30
Hector Sanjuan
5e4c8d9418 Ctl: increase host-name spacing in status output
Allow for slightly longer hostnames
2019-08-14 14:10:11 +02:00
Hector Sanjuan
dc2e73dfc2
Release 0.11.0-rc6 2019-08-13 20:25:53 +02:00
Hector Sanjuan
dc3170b1d2 cluster: add mDNS service discovery
I always thought the libp2p node would do this, but it is not the case.
With this, CRDT peers are able to autodiscover on local networks.
2019-08-13 19:32:38 +02:00
Hector Sanjuan
5303a7f721
Merge pull request #874 from ipfs/feat/865-init-consensus
Service: Select consensus on "init" (not on "daemon")
2019-08-13 10:21:45 +02:00
Hector Sanjuan
3b3e3493a8 Service: init: improve log messages
Align with previous behaviour and make sure it is logged that
the configuration file was written.

Do not say "peerstore written with 0 entries" as that might be
taken like an error.
2019-08-12 16:58:06 +02:00
Hector Sanjuan
d8c20adc4e
Merge pull request #869 from ipfs/fix/732-pin-update-the-good-way
Fix #732: Introduce native pin/update
2019-08-12 12:15:30 +02:00
Kishan Mohanbhai Sagathiya
684f58894d Use string flag instead of string slice
Because string slice is not very user friendly. Using string instead
would allow us to pass a comma separated list of arguments in one option
2019-08-12 11:31:26 +02:00
Hector Sanjuan
2311bdac19 daemon: do consensus things inside a switch 2019-08-12 10:39:49 +02:00