Commit Graph

2188 Commits

Author SHA1 Message Date
Hector Sanjuan
3e61377f86 Remove some unreachable code 2019-05-17 16:36:30 +02:00
Hector Sanjuan
39481cb613 Issue #497: Better way of handling progress and timeouts for pins & refs 2019-05-17 16:26:27 +02:00
Kishan Mohanbhai Sagathiya
0c31342459 Pin timeouts should start from the last block
Close reset channel
No need for done in checkTimeout

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
aecee5bcaf Use empty struct for channels
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
1b90d871c2 Reset the timer less often 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
7cc09e0efd Dont reset if timer expired 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
9db07ea4c1 Avoid racy behaviour while using timer.Reset 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
c70b77d0bd Remove the data race because of elapsed 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
4e3eadfcb1 Pin timeouts should start from the last block
- a doPostCtx can happen in Pin and then pass the response reader to
either a handleRefsProgress or handlePinProgress
- fixed timeout in Unpin
2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
2b1c25ea52 Remove redundant annotations 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
3ce7f5a56a Unnecessary error printing 2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
1d62d8301e Pin timeouts should start from the last block
Parse response and check for errors
2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
3db5e45069 Pin timeouts should start from the last block
Use timer for checking timeouts
2019-05-17 14:17:47 +02:00
Kishan Mohanbhai Sagathiya
9244daabfb Pin timeouts should start from the last block
IPFSConnector Pin operation should not have a timeout since the start
of the operation, but a timeout since the last block was received,
thus only cancelling operations when there are no more blocks provided
in the network
2019-05-17 14:17:47 +02:00
Hector Sanjuan
9692e368f3
Merge pull request #779 from ipfs/feat/failfast
Tests: add failfast, bump to go1.12, adjust codecov
2019-05-17 13:07:48 +01:00
Hector Sanjuan
18e30967a4 Disable some sharness tests adding with cid-version=1
go-ipfs is not publishing new docker containers and therefore
we cannot test with the new base32 output changes.
2019-05-17 13:58:26 +02:00
Hector Sanjuan
c23bb9abb0 Bump deps 2019-05-17 13:58:26 +02:00
Hector Sanjuan
c00b90b86d Fix panic in test 2019-05-17 13:54:13 +02:00
Hector Sanjuan
bfd4b7d3ee Tests: add failfast, bump to go1.12, adjust codecov 2019-05-17 13:54:13 +02:00
Hector Sanjuan
5f7510bd2d
Merge pull request #780 from ipfs/fix/identity-docs
service: multiple fixes around init and identities
2019-05-17 11:57:31 +01:00
Hector Sanjuan
ff92a660d2
Merge pull request #713 from ipfs/feat/monitor-ring
Change the underlying data structure of a metrics.Window
2019-05-17 11:56:47 +01:00
Hector Sanjuan
d51c2a0377 Merge branch 'master' into feat/monitor-ring 2019-05-16 15:46:30 +02:00
Hector Sanjuan
e62d10f83a service: multiple fixes around init and identities
* Fix error messages (they must be in the form "doing something")
* Improve/reword some error messages
* Document the identity.json existance in the cli docs
* Fix a bunch of typos
* Fix missing folder path in the --help
* Fix cluster not locking when configuration is not there but folder is
* Fix force flag not overriding the config overwrite prompt
* Fix deletion of Raft state on re-init (not necessary if identity persists)
* Fix overwriting on identity (should not be overwritten if already exists)

Much of this paves the way to be able to run without service.json:

* Either taking default values (and using env vars) - maybe someday
* Either by getting a configuration template it from somewhere (ipfs, http)
  at runtime - sooner.
2019-05-16 15:31:36 +02:00
Hector Sanjuan
5be1b6532f
Merge pull request #714 from ipfs/feat/monitor-accrual
Monitoring and re-allocation revamp: Accrual failure detection
2019-05-16 13:46:02 +01:00
Hector Sanjuan
2e8f3ad4d3 Tests: Do not hang indifinitely when clusters are not healthy 2019-05-16 14:11:56 +02:00
Hector Sanjuan
6caf78a57b monitor config: make threshold optional in the configuration
takes default when not set.
2019-05-16 12:52:40 +02:00
Hector Sanjuan
7a66fc3484
Merge pull request #775 from ipfs/feat/rpc-auth
Feat: RPC Authorization
2019-05-16 11:08:13 +01:00
Hector Sanjuan
305096da94 policygen: Improve help text 2019-05-16 12:07:02 +02:00
Hector Sanjuan
f428a3b669
Merge pull request #766 from ipfs/issue_760
Separate Identity and Configuration
2019-05-16 09:24:44 +01:00
Kishan Mohanbhai Sagathiya
50c9aa6d17 There should be identity.json sharness/config
There should be identity.json stripped out from service.json in
sharness/config

License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-16 09:39:56 +05:30
Hector Sanjuan
59fdff97f2 policygen: use format.Source() directly in code. 2019-05-15 10:46:03 +02:00
Kishan Mohanbhai Sagathiya
64390be061 No need for deferred recover
License: MIT
Signed-off-by: Kishan Mohanbhai Sagathiya <kishansagathiya@gmail.com>
2019-05-14 14:47:28 +05:30
Kishan Sagathiya
9822848e92
Update config/identity.go
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-14 14:43:45 +05:30
Kishan Sagathiya
a681d25cee
Update config/identity.go
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-14 14:43:34 +05:30
Hector Sanjuan
b1996970c5 ipfscluster.go: Add stops at the end of sentences 2019-05-13 23:22:08 +02:00
Hector Sanjuan
21032f2101 Raft: remove TODO. Trust all peers. 2019-05-13 23:22:08 +02:00
Hector Sanjuan
d468ea5d31 crdt: add test for DistrustPeer 2019-05-13 23:22:08 +02:00
Hector Sanjuan
816449d9a5 policygen: fix typo 2019-05-13 23:22:08 +02:00
Hector Sanjuan
de2e64e1e0 RPC Auth: make policygen.go generate a full rpc_policy.go
So that the file can be replaced. Helpers: "make" and "make install"
2019-05-13 23:22:08 +02:00
Hector Sanjuan
a2d8ce2ab6
Avoid using Sprintf("%s.%s")
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-10 11:52:37 +01:00
Hector Sanjuan
fba230fff9
Fix policygen.go path in comment
Co-Authored-By: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2019-05-10 11:50:42 +01:00
Hector Sanjuan
a0eeddfae7 Test: remove removed endpoints from mock RPC 2019-05-09 22:52:02 +02:00
Hector Sanjuan
dbc52ae981 rpc auth: golint 2019-05-09 22:36:03 +02:00
Hector Sanjuan
6530808298 rpc auth: adjust tests to work with trusted-peer-enabled crdts component 2019-05-09 21:24:56 +02:00
Hector Sanjuan
a86c7cae2b rpc auth: handle some auth errors gracefully
particuarly we will ignore authorization errors for some broadcasts and somply
not include those responses in the assembled one.
2019-05-09 21:23:49 +02:00
Hector Sanjuan
949e6f2364 RPC auth: Support Trusted Peers in CRDT consensus component.
TrustedPeers are specified in the configuration. Additional peers
can be added at runtime with Trust/Distrust functions.

Unfortunately we cannot use consensus.PeerAdd as a way to trust a peer as
cluster.PeerAdd+Join can be called by any peer and this calls
consensus.PeerAdd.

The result is consensus.PeerAdd doing a lot in Raft while consensus.Trust does
nothing, while in CRDTs consensus.Trust does something but consensus.PeerAdd
does nothing. But this is more or less consistent.
2019-05-09 19:48:40 +02:00
Hector Sanjuan
c5a2e7fdc5 RPC auth: Fix tests
I cannot have RPCAPIs expose a SvcID() method as gorpc will warn about it not
having the right signature. So I have created an RPCServiceID() method instead.
2019-05-09 16:33:59 +02:00
Hector Sanjuan
70f4cad613 RPC Auth: start using the RPC policy in the RPC server. 2019-05-09 15:14:26 +02:00
Hector Sanjuan
1a4ff6a500 RPC auth: Define Default RPC policy and carry it in the cluster config
While right now it is not allowed for the user to overwrite any entries
in the default policy from the JSON, this should be possible (and easy)
in the future.
2019-05-09 14:48:23 +02:00
Hector Sanjuan
2ed48b6ac4 RPC auth: Rework PeerAdd and Join
PeerAdd called RPC endpoints for `LogMetric` and `ConnectSwarms`
remotely. However, I think similar effect can be achieved by calling
these from the Join() function locally.

In particular, ConnectSwarms was called when maybe the joining peer did not
even know about the other peers in the Cluster. Now this is delayed until some
ping metrics have come through.
2019-05-09 14:19:07 +02:00