Commit Graph

1532 Commits

Author SHA1 Message Date
Adrian Lanzafame
3d4412ffd4
expose badger loading mode conf
For simplicity, I haven't exposed the entire
badger config. Also, due to go default values and
the badger implementation, I have had to extract the
two loading mode config values so we can determine
whether it has actually has been set by the user in
the json config file.

License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-21 22:42:40 +10:00
Hector Sanjuan
e523215ee2
Merge pull request #789 from ipfs/go-ds-crdt-dep
Update go-ds-crdt
2019-05-21 13:29:46 +01:00
Hector Sanjuan
44d93d61e0 fix timeouts in crdt 2019-05-21 11:55:48 +02:00
Hector Sanjuan
c59873e076 update deps 2019-05-21 11:34:33 +02:00
Hector Sanjuan
9ffd08520d Downgrade discovery 2019-05-21 11:05:51 +02:00
Hector Sanjuan
8c8e449bc9 Update go-ds-crdt 2019-05-21 10:38:37 +02:00
Hector Sanjuan
30ba6f82dd
Merge pull request #502 from ipfs/feat/run-tests-with-race
Fix #340: Run tests with -race
2019-05-20 22:56:50 +01:00
Hector Sanjuan
8e6eefb714 Tests: multiple fixes
This fixes multiple issues in and around tests while
increasing ttls and delays in 100ms. Multiple issues, including
races, tests not running with consensus-crdt missing log messages
and better initialization have been fixed.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2019-05-20 23:45:04 +02:00
Hector Sanjuan
9692e368f3
Merge pull request #779 from ipfs/feat/failfast
Tests: add failfast, bump to go1.12, adjust codecov
2019-05-17 13:07:48 +01:00
Hector Sanjuan
18e30967a4 Disable some sharness tests adding with cid-version=1
go-ipfs is not publishing new docker containers and therefore
we cannot test with the new base32 output changes.
2019-05-17 13:58:26 +02:00
Hector Sanjuan
c23bb9abb0 Bump deps 2019-05-17 13:58:26 +02:00
Hector Sanjuan
c00b90b86d Fix panic in test 2019-05-17 13:54:13 +02:00
Hector Sanjuan
bfd4b7d3ee Tests: add failfast, bump to go1.12, adjust codecov 2019-05-17 13:54:13 +02:00
Hector Sanjuan
5f7510bd2d
Merge pull request #780 from ipfs/fix/identity-docs
service: multiple fixes around init and identities
2019-05-17 11:57:31 +01:00
Hector Sanjuan
ff92a660d2
Merge pull request #713 from ipfs/feat/monitor-ring
Change the underlying data structure of a metrics.Window
2019-05-17 11:56:47 +01:00
Hector Sanjuan
d51c2a0377 Merge branch 'master' into feat/monitor-ring 2019-05-16 15:46:30 +02:00
Hector Sanjuan
e62d10f83a service: multiple fixes around init and identities
* Fix error messages (they must be in the form "doing something")
* Improve/reword some error messages
* Document the identity.json existance in the cli docs
* Fix a bunch of typos
* Fix missing folder path in the --help
* Fix cluster not locking when configuration is not there but folder is
* Fix force flag not overriding the config overwrite prompt
* Fix deletion of Raft state on re-init (not necessary if identity persists)
* Fix overwriting on identity (should not be overwritten if already exists)

Much of this paves the way to be able to run without service.json:

* Either taking default values (and using env vars) - maybe someday
* Either by getting a configuration template it from somewhere (ipfs, http)
  at runtime - sooner.
2019-05-16 15:31:36 +02:00
Hector Sanjuan
5be1b6532f
Merge pull request #714 from ipfs/feat/monitor-accrual
Monitoring and re-allocation revamp: Accrual failure detection
2019-05-16 13:46:02 +01:00
Hector Sanjuan
2e8f3ad4d3 Tests: Do not hang indifinitely when clusters are not healthy 2019-05-16 14:11:56 +02:00
Hector Sanjuan
6caf78a57b monitor config: make threshold optional in the configuration
takes default when not set.
2019-05-16 12:52:40 +02:00
Hector Sanjuan
7a66fc3484
Merge pull request #775 from ipfs/feat/rpc-auth
Feat: RPC Authorization
2019-05-16 11:08:13 +01:00
Hector Sanjuan
305096da94 policygen: Improve help text 2019-05-16 12:07:02 +02:00
Hector Sanjuan
f428a3b669
Merge pull request #766 from ipfs/issue_760
Separate Identity and Configuration
2019-05-16 09:24:44 +01:00
Kishan Mohanbhai Sagathiya
50c9aa6d17 There should be identity.json sharness/config
There should be identity.json stripped out from service.json in
sharness/config

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-16 09:39:56 +05:30
Hector Sanjuan
59fdff97f2 policygen: use format.Source() directly in code. 2019-05-15 10:46:03 +02:00
Kishan Mohanbhai Sagathiya
64390be061 No need for deferred recover
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-14 14:47:28 +05:30
Kishan Sagathiya
9822848e92
Update config/identity.go
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-14 14:43:45 +05:30
Kishan Sagathiya
a681d25cee
Update config/identity.go
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-14 14:43:34 +05:30
Hector Sanjuan
b1996970c5 ipfscluster.go: Add stops at the end of sentences 2019-05-13 23:22:08 +02:00
Hector Sanjuan
21032f2101 Raft: remove TODO. Trust all peers. 2019-05-13 23:22:08 +02:00
Hector Sanjuan
d468ea5d31 crdt: add test for DistrustPeer 2019-05-13 23:22:08 +02:00
Hector Sanjuan
816449d9a5 policygen: fix typo 2019-05-13 23:22:08 +02:00
Hector Sanjuan
de2e64e1e0 RPC Auth: make policygen.go generate a full rpc_policy.go
So that the file can be replaced. Helpers: "make" and "make install"
2019-05-13 23:22:08 +02:00
Hector Sanjuan
a2d8ce2ab6
Avoid using Sprintf("%s.%s")
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-10 11:52:37 +01:00
Hector Sanjuan
fba230fff9
Fix policygen.go path in comment
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-10 11:50:42 +01:00
Hector Sanjuan
a0eeddfae7 Test: remove removed endpoints from mock RPC 2019-05-09 22:52:02 +02:00
Hector Sanjuan
dbc52ae981 rpc auth: golint 2019-05-09 22:36:03 +02:00
Hector Sanjuan
6530808298 rpc auth: adjust tests to work with trusted-peer-enabled crdts component 2019-05-09 21:24:56 +02:00
Hector Sanjuan
a86c7cae2b rpc auth: handle some auth errors gracefully
particuarly we will ignore authorization errors for some broadcasts and somply
not include those responses in the assembled one.
2019-05-09 21:23:49 +02:00
Hector Sanjuan
949e6f2364 RPC auth: Support Trusted Peers in CRDT consensus component.
TrustedPeers are specified in the configuration. Additional peers
can be added at runtime with Trust/Distrust functions.

Unfortunately we cannot use consensus.PeerAdd as a way to trust a peer as
cluster.PeerAdd+Join can be called by any peer and this calls
consensus.PeerAdd.

The result is consensus.PeerAdd doing a lot in Raft while consensus.Trust does
nothing, while in CRDTs consensus.Trust does something but consensus.PeerAdd
does nothing. But this is more or less consistent.
2019-05-09 19:48:40 +02:00
Hector Sanjuan
c5a2e7fdc5 RPC auth: Fix tests
I cannot have RPCAPIs expose a SvcID() method as gorpc will warn about it not
having the right signature. So I have created an RPCServiceID() method instead.
2019-05-09 16:33:59 +02:00
Hector Sanjuan
70f4cad613 RPC Auth: start using the RPC policy in the RPC server. 2019-05-09 15:14:26 +02:00
Hector Sanjuan
1a4ff6a500 RPC auth: Define Default RPC policy and carry it in the cluster config
While right now it is not allowed for the user to overwrite any entries
in the default policy from the JSON, this should be possible (and easy)
in the future.
2019-05-09 14:48:23 +02:00
Hector Sanjuan
2ed48b6ac4 RPC auth: Rework PeerAdd and Join
PeerAdd called RPC endpoints for `LogMetric` and `ConnectSwarms`
remotely. However, I think similar effect can be achieved by calling
these from the Join() function locally.

In particular, ConnectSwarms was called when maybe the joining peer did not
even know about the other peers in the Cluster. Now this is delayed until some
ping metrics have come through.
2019-05-09 14:19:07 +02:00
Hector Sanjuan
40fb0761da RPC Auth: Add policygen.go: a tool to list all RPC endpoints. 2019-05-09 14:18:12 +02:00
Kishan Mohanbhai Sagathiya
654c376a57 Fixed sharness test with new identity
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-09 00:13:29 +05:30
Kishan Mohanbhai Sagathiya
f05af75abc Tests for identity separation
Added tests for identity.go and modifies others according ly

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-08 21:54:59 +05:30
Hector Sanjuan
26b0949b5e
Merge pull request #769 from ipfs/deps/update
Update deps
2019-05-08 13:24:21 +01:00
Hector Sanjuan
5fc5c6d38c Re-update 2019-05-08 13:33:22 +02:00
Hector Sanjuan
8b19f812c6
Merge pull request #773 from ipfs/feat/component-rpc-apis
RPC: Give each component a different RPC Service
2019-05-07 10:57:43 +01:00