Commit Graph

87 Commits

Author SHA1 Message Date
Hector Sanjuan
a56ce73f92 Switch to building with -trimpath instead of asm and gc trimpath flags
Kudos to https://github.com/ipfs/distributions/pull/314
2020-10-20 14:32:20 +02:00
Kishan Mohanbhai Sagathiya
ae8e74453b Fix #937: Print full working configuration at startup
Only when using debug mode

Co-authored-by: Hector Sanjuan <code@hector.link>
2020-05-15 01:33:04 +02:00
Hector Sanjuan
7ffd18e41b Feat: upgrade to dual DHT 2020-04-14 22:03:24 +02:00
Hector Sanjuan
65ad4bd632 cluster/daemons: Close the datastore AFTER the DHT.
Avoids panics.

This also removes the abnormality of cluster closing a datastore
that it did not create.
2020-04-02 16:29:41 +02:00
Hector Sanjuan
ab74987bc8 clusterhost: place dht records in persistened DHT
With this cluster peers will start remembering provider records accross
re-starts.
2020-04-01 20:15:48 +02:00
Hector Sanjuan
b3853caf36 Dependency ugprade: changes needed
* Libp2p protectors no longer needed, use PSK directly
* Generate cluster 32-byte secret here (helper gone from pnet)
* Switch to go-log/v2 in all places
* DHT bootstrapping not needed. Adjust DHT options for tests.
* Do not rely on dissappeared CidToDsKey and DsKeyToCid functions fro dshelp.
* Disable QUIC (does not support private networks)
* Fix tests: autodiscovery started working properly
2020-03-22 14:50:25 +01:00
Hector Sanjuan
c80a207e41 Fix #1012: Trim paths on Makefile builds
This results in more reproduceable builds.

Additionally, sources command to list all go-files and not just those
one directory below.
2020-03-06 14:49:19 +01:00
Hector Sanjuan
5d44275d0e
Chore: Add license file with URLS (#1014)
* Chore: Add license file with URLS

Source: 59df5e58a2

* Chore: license copyright updates
2020-03-06 13:52:19 +01:00
Hector Sanjuan
531379b1d9
Feature: Support multiple listeners in configuration
* add ipv6 listening addresses to the default config

* ipfsproxy: support multiple listeners. Add default ipv6.

* mm

* restapi: support multiple listen addresses. enable ipv6

* cluster_config: format default listen addresses

* commands: update for multiple listeners. Fix randomports for udp and ipv6.

* ipfs-cluster-service: fix randomports test

* multiple listeners: fix remaining tests

* golint

* Disable ipv6 in defaults

It is not supported by docker by default. It is not supported in travis-CI
build environments. User can enable it now manually.

* proxy: disable ipv6 in test

* ipfshttp: fix test

Co-authored-by: @RubenKelevra <cyrond@gmail.com>
2020-02-28 11:16:16 -05:00
Yang Hau
7986d94242
fix: Fix typos (#1001)
Fix typos in files
2020-02-03 10:30:04 +01:00
Hector Sanjuan
60ce5b3228 Update copyrights to 2020x 2019-12-20 14:40:34 +01:00
Hector Sanjuan
0fa01d5435 Distribute the correct dual license with ctl and service 2019-12-20 14:04:48 +01:00
Hector Sanjuan
2391d2d246
Merge pull request #960 from ipfs/feat/log-env-flags
Environment variable to set loglevels for individual identifiers
2019-12-16 17:20:17 +01:00
Kishan Sagathiya
5258a4d428 Remove map pintracker (#944)
This removes mappintracker and sets stateless tracker as the default (and only) pintracker component.

Because the stateless tracker matches the cluster state with only ongoing operations being kept on memory, and additional information provided by ipfs-pin-ls, syncing operations are not necessary. Therefore the Sync/SyncAll operations are removed cluster-wide.
2019-12-12 21:22:54 +01:00
Hector Sanjuan
f587b453c8
Merge pull request #970 from ipfs/feat/service-extract-to-cmdutils
Service: extract some methods to cmdutils
2019-12-06 11:08:58 +01:00
Kishan Mohanbhai Sagathiya
a136509522 No need to verify log levels 2019-12-06 11:22:08 +05:30
Hector Sanjuan
418587a946
Merge pull request #968 from ipfs/fix/libp2p-api-with-crdt
Fix: disable default libp2p API endpoint with CRDT consensus
2019-12-05 16:14:35 +01:00
Kishan Sagathiya
30fd5ee6a0 Feat: Allow and run multiple informer components (#962)
This introduces the possiblity of running Cluster with multiple informer components. The first one in the list is the used for "allocations". This is just groundwork for working with several informers in the future.
2019-12-05 15:08:43 +01:00
Kishan Mohanbhai Sagathiya
df36995213 detail description and example for how to customize logging 2019-12-04 23:22:38 +05:30
Kishan Mohanbhai Sagathiya
794646ff12 Addressed reviews 2019-12-04 23:10:35 +05:30
Hector Sanjuan
f6ea95bccf Service/cmdutils: fix small typos (review) 2019-12-02 16:08:47 +01:00
Hector Sanjuan
5cc9ed0c4f ipfs-cluster-service: extract some helpers to cmdutils
This will allow better re-use from other CLIs.
2019-12-02 11:57:24 +01:00
Hector Sanjuan
454efe83fd ipfs-cluster-service: rename cluster node -> cluster peer in help.
We use "peer" everywhere. This is an old leftover.
2019-12-02 11:57:13 +01:00
Hector Sanjuan
578ee29bdf Fix: disable default libp2p API endpoint with CRDT consensus 2019-12-02 11:54:51 +01:00
Kishan Mohanbhai Sagathiya
e99abf7688 Do not modify global var ipfscluster.LoggingFacilities 2019-11-25 17:18:50 +05:30
Kishan Mohanbhai Sagathiya
f0c83f59b4 Don't error if a loglevel is passed twice, but warn 2019-11-25 16:57:48 +05:30
Kishan Mohanbhai Sagathiya
047d3fe615 Use a single log-level flag 2019-11-25 16:41:49 +05:30
Kishan Mohanbhai Sagathiya
361b819139 Try to set all loglevels with a single flag 2019-11-25 12:57:59 +05:30
Kishan Mohanbhai Sagathiya
1fd4e41c99 Use urfave/cli's environment variable support for loglevels
- introduce a new flag for loglevels for individual components
- handle all log related flags together so that we can maintain priority
among log level flags (loglevel < component-loglevel < debug)
2019-11-16 15:37:57 +05:30
Kishan Mohanbhai Sagathiya
d8929aa5a6 Environment variable to set loglevels for individual identifiers
- Example: export LOG_FACS="cluster:debug,raft:debug"
2019-11-14 22:54:26 +05:30
Kishan Mohanbhai Sagathiya
56ef75b50c Use TLS instead of secio for security 2019-11-05 12:50:46 +01:00
Hector Sanjuan
073a0dfa58 Dockerfiles: download modules in advance
This should speed up Dockerfile builds, particularly when building after
updating the code since the modules will not be to be re-downloaded.
2019-10-28 10:23:18 +01:00
Kishan Sagathiya
7b499263af service init --randomports:
Allow initialize cluster with random ports that are ununsed (at the moment of initialization)
2019-10-24 17:54:50 +02:00
Hector Sanjuan
d63a7fd641
Merge pull request #877 from ipfs/fix/ipfs-to-p2p
Use `p2p` protocol name over `ipfs` for multiaddr
2019-09-06 15:00:36 +02:00
Hector Sanjuan
33f111c44d mDNS: attach mDNS inside the Cluster. Allow interval configuration.
Setting up mDNS outside the Cluster is dirtier and allows less configuration.

This adds MDNSInterval to the cluster config options and allow disabling it
when the option is set to 0.
2019-08-24 17:24:18 +02:00
Hector Sanjuan
aab5f9bd0b Enable p2p addresses in a single place 2019-08-23 22:45:32 +02:00
Kishan Mohanbhai Sagathiya
6656b80a00 Some more occurences of /ipfs
and use  SwapToP2pMultiaddrs (very helpful since ipfs still send
addresses with `/ipfs` tag)
2019-08-16 11:56:09 +05:30
Hector Sanjuan
dc3170b1d2 cluster: add mDNS service discovery
I always thought the libp2p node would do this, but it is not the case.
With this, CRDT peers are able to autodiscover on local networks.
2019-08-13 19:32:38 +02:00
Hector Sanjuan
5303a7f721
Merge pull request #874 from ipfs/feat/865-init-consensus
Service: Select consensus on "init" (not on "daemon")
2019-08-13 10:21:45 +02:00
Hector Sanjuan
3b3e3493a8 Service: init: improve log messages
Align with previous behaviour and make sure it is logged that
the configuration file was written.

Do not say "peerstore written with 0 entries" as that might be
taken like an error.
2019-08-12 16:58:06 +02:00
Kishan Mohanbhai Sagathiya
684f58894d Use string flag instead of string slice
Because string slice is not very user friendly. Using string instead
would allow us to pass a comma separated list of arguments in one option
2019-08-12 11:31:26 +02:00
Hector Sanjuan
2311bdac19 daemon: do consensus things inside a switch 2019-08-12 10:39:49 +02:00
Hector Sanjuan
f32a53c4e6 Daemon: "init" do not TrustAll when passing --peers 2019-08-12 10:25:55 +02:00
Hector Sanjuan
063c5f1b78 Service: Select consensus on "init" (not on "daemon")
Fixes #865.

This makes the necessary changes so that consensu is selected on "init" with a
flag set, by default, to "crdt". This generates only a "crdt" or a "raft"
section, not both.

If the configuration file has a "raft" section, "raft" will be used to start
the daemon. If it has a "crdt" section, "crdt" will be used. If it has none or
both sections, an error will happen.

This also affects "state *" commands, which will now autoselect how to work
from the existing configuration.
2019-08-09 19:20:53 +02:00
Hector Sanjuan
00e78a6b6d
Daemon: support remote configuration (#868)
* Daemon: support remote configuration

This:

* Adds support for fetching the configuration from a remote HTTP location:

`ipfs-cluster-service init http://localhost:8080/ipfs/Qm...` will instruct
cluster to read the configuration file from ipfs on start (potentially making
use of ipns and dnslink).

This is done by creating a `service.json` like `{ "source": <url> }`.

The source is then read when loading that configuration every time the daemon starts.

This allows to let users always use a mutating remote configuration, potentially
adding/removing trusted peers from the list or adjusting other things.

* Configuration and state helpers from ipfs-cluster-service have been extracted
to its own cmdutils package. This will help supporting something like an
`ipfs-cluster-follow` command in the next releases.

* Allows to disable the rest api by not defining it in the configuration (I thought
this was already so, but apparently only affected the ipfsproxy).

* Removes informer/allocator configurations from the daemon (--alloc). No one used
a non default pair. In fact, it was potentially buggy to use the reposize one.
2019-08-09 12:56:27 +02:00
Hector Sanjuan
eace9031dc
Merge pull request #856 from ipfs/feat/force-consensus-choice
service: Make --consensus a mandatory flag
2019-07-30 11:32:49 +02:00
Kishan Sagathiya
e0c38a2ddf Auto-trust bootstrap peers (#857)
Fix #834: Auto-trust bootstrap peers (#857)

Add bootstrap peers to list of trusted peers for crdt.
2019-07-29 10:17:43 +02:00
Hector Sanjuan
6188d6ff52 service: Make --consensus a mandatory flag
It has a few implications to launch a raft peer when you wanted to do crdt and vice-versa.

Same when exporting and exporting states.

Users starting cluster peers should be explicit about their consensus choice.

Also, if we ever want to make `crdt` the default, we can't do that before
making `raft` non-default first. I don't like to break things but otherwise
the experience for new users wanting to try crdts might be aweful.
2019-07-26 18:15:41 +02:00
Kishan Sagathiya
e7b731e0e4 Fix #835: service: init --peers
* Init should take a list of peers

This commit adds `--peers` option to `ipfs-cluster-service init`

`ipfs-cluster-service init --peers <multiaddress,multiaddress>`

- Adds and writes the given peers to the peerstore file
- For raft config section, adds the peer IDs to the `init_peerset`
- For crdt config section, add the peer IDs to the `trusted_peers`
2019-07-25 10:47:44 +02:00
Kishan Sagathiya
b96c2b664d Do not load API components removed from the config (#836)
* Do not load API components removed from the config

This commit introduces a map that would keep track of whether components
for a component were missing or not from the JSON config file. This map
can be check while creating cluster to avoid loading a component.

It would consider component only if the component is fully missing from the
config.
Say the component in question is `ipfsproxy` which is under `api`
section.
This would use defaults for `ipfsproxy` and load IPFS proxy.

```
{
"api":{
	"ipfsproxy": {}
	}
}
```

However, this would not load IPFS proxy
```
{
"api":{}
}
```
2019-07-18 17:33:14 +02:00