cascade/ipfs-cluster
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,574 Commits 3 Branches 70 Tags 15 MiB
a0d93fc62c
Commit Graph

1574 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hector Sanjuan
6d77954327 Fix multiple problems with state migration 
Additionally, remove persisting the state version to the go-datastore. In the
future versions of the state, there is not a global format anymore (with a
global version). Instead, every pin element can potentially be stored in a
different version.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-19 19:46:03 +00:00
Hector Sanjuan
d57b81490f State: Use go-datastore to implement the state interface 
Since the beginning, we have used a Go map to store the shared state (pinset)
in memory. The mapstate knew how to serialize itself so that libp2p-raft would
know how to write to disk when it:

* Saved snapshots of the state on shutdown
* Sent the state to a newcomer peer

hashicorp.Raft assumes an in-memory state which is snapshotted from time to
time and read from disk on boot.

This commit adds a `dsstate` implementation of the state interface using
`go-datastore`. This allows to effortlessly switch to a disk-backed state in
the future (as we will need), and also have at our disposal the different
implementations and utilities of Datastore for fine-tuning (caching, batching
etc.).

`mapstate` has been reworked to use dsstate. Ideally, we would not even need
`mapstate`, as it would suffice to initialize `dsstate` with a
`MapDatastore`. BUT, we still need it separate to be able to auto-migrate to
the new format.

This will be the last migration with the current system. Once this has been
released and users have been able to upgrade we will just remove `mapstate` as
it is now.

License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
 2019-02-19 18:31:14 +00:00
Hector Sanjuan
3059ab387a
Merge pull request #663 from roignpar/issue_656 
Read config values from env on init command
 2019-02-19 17:48:47 +00:00
Hector Sanjuan
2b6a8d2a07
Merge pull request #679 from ipfs/update/libp2p-6.0.35 
Update to libp2p-6.0.35 and general dep alignment
 2019-02-19 16:45:45 +00:00
Hector Sanjuan
d1617ac79c Update to libp2p-6.0.35 and general dep alignment 
No more duplicates with protobuf

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-19 15:28:12 +00:00
Hector Sanjuan
cc8dd7eb53
Merge pull request #675 from ipfs/update/libp2p-6.0.33 
Dependency alignment for libp2p-6.0.33, dag libraries and others
 2019-02-19 13:27:34 +00:00
Robert Ignat
06482e5fce Add ApplyEnvVars test to observations config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:51:39 +02:00
Robert Ignat
50844b9e5b Add ApplyEnvVars test to stateless config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:51:04 +02:00
Robert Ignat
368f1de6bc Add ApplyEnvVars test to maptracker config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:50:17 +02:00
Robert Ignat
38886dae42 Add ApplyEnvVars test to pubsubmon config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:48:15 +02:00
Robert Ignat
40d1077af2 Add ApplyEnvVars test to monbasic config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:47:28 +02:00
Robert Ignat
be8c56f949 Add ApplyEnvVars test to ipfshttp config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:47:02 +02:00
Robert Ignat
e38ceab222 Add ApplyEnvVars test to numpin config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:46:15 +02:00
Robert Ignat
08580c3d5c Add ApplyEnvVars test to disk config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:45:42 +02:00
Robert Ignat
bac982c5aa Add ApplyEnvVars test to raft config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:43:54 +02:00
Robert Ignat
36ee0f84eb Add ApplyEnvVars test to ipfsproxy config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 17:40:51 +02:00
Robert Ignat
78ac49a2d5 Fix env config tests 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-18 13:36:51 +02:00
Hector Sanjuan
b32e12ab25
Merge pull request #677 from ipfs/0.9.0 
Release 0.9.0
 2019-02-18 10:40:28 +00:00
Adrian Lanzafame
bccccccbdc
gx publish 0.9.0 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-18 20:31:10 +10:00
Adrian Lanzafame
6509dfff56
Release 0.9.0 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-18 20:31:07 +10:00
Adrian Lanzafame
d2c002319c
Merge pull request #664 from ipfs/0.9.0 
v0.9.0 changelog
 2019-02-18 20:21:56 +10:00
Hector Sanjuan
90d79f28ff Reference a couple other issues in changelog 
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-18 10:04:42 +00:00
Adrian Lanzafame
40b60d3e6c
Add Go API changes and fix date 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-18 18:13:38 +10:00
Hector Sanjuan
dcd8bb85d1
Merge pull request #676 from ipfs/fix/cancelled-context-repo-stat 
Fix context cancelled too early.
 2019-02-17 20:58:37 +00:00
Hector Sanjuan
72d41790bd Ipfshttp: Fix: context tracing for sendInformerMetrics 
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-16 00:11:07 +00:00
Hector Sanjuan
f22d01976d Fix context cancelled too early. 
Every 10th pin the ipfs connector calls repo stat.

Because the call is async, the context was cancelled immediately and failed.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-15 18:45:04 +00:00
Robert Ignat
523e109ef1 Create LoadJSONFileAndEnv config method for convenience 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-15 19:32:48 +02:00
Robert Ignat
168cf76224 Change ApplyEnvVars strategy for all config components 
Get jsonConfig from Config, apply env vars to it, load jsonConfig
back into Config.

License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-15 19:07:20 +02:00
Robert Ignat
47252f8e62 Merge remote-tracking branch 'upstream/master' into issue_656 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-15 16:45:32 +02:00
Hector Sanjuan
10d6a37304 Update deps: fix the things that need fixing 
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-15 12:40:53 +00:00
Hector Sanjuan
d4a4b5526d Dependency alignment for libp2p-6.0.33, dag libraries and others 
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-15 12:09:05 +00:00
Hector Sanjuan
d49884bfd0
Merge pull request #671 from roignpar/rename_observations_config 
Rename observations Config fields
 2019-02-15 10:43:52 +00:00
Hector Sanjuan
73ba101900
Merge pull request #674 from ipfs/fix/pprof-urls 
bring pprof urls in line with default
 2019-02-15 10:34:33 +00:00
Adrian Lanzafame
c3f5a6002d
bring pprof urls in line with default 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-15 12:31:44 +10:00
Robert Ignat
bac23c0243 Rename observations Config fields 
* StatsReportingInterval -> ReportingInterval
* TracingSamplingProb -> SamplingProb
* TracingServiceName -> ServiceName

License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-13 23:49:02 +02:00
Robert Ignat
aa5d545164 Implement ApplyEnvVars for ipfshttp Config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-13 23:13:28 +02:00
Robert Ignat
580020db54 Use separate envConfigKeys for metrics and tracing 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-13 23:10:13 +02:00
Robert Ignat
15ac8cd79b Use ParseDurations when applying json to cluster Config 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-12 16:24:06 +02:00
Adrian Lanzafame
b4f0eb3c95
gx publish 0.9.0-rc1 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-12 21:01:53 +10:00
Adrian Lanzafame
17ffcf2b1c
Release 0.9.0-rc1 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-12 21:01:50 +10:00
Robert Ignat
da58eae2c9 Add ipfsproxy NodeHTTPS config field to ToJSON 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-11 18:08:55 +02:00
Adrian Lanzafame
ecef9ea762
Merge pull request #649 from ipfs/remove/snaps 
Fix #593: Remove snaps
 2019-02-09 13:11:05 +10:00
Robert Ignat
21a539159f Remove json config overriding with env vars 
in LoadJSON for restapi and cluster Config

License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-09 00:00:58 +02:00
Robert Ignat
032f02802f Implement ApplyEnvVars for all ComponentConfigs 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-08 23:57:16 +02:00
Adrian Lanzafame
fe5e30d4d9
v0.9.0 changelog 
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
 2019-02-08 11:48:03 +10:00
Robert Ignat
ed30ac1ab4 Add ApplyEnvVars() to ComponentConfig interface 
* cluster and restapi configs can also get values from environment variables
* other config components don't read any values from the environment

License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-07 20:51:20 +02:00
Hector Sanjuan
fcea4f41cf
Merge pull request #662 from roignpar/patch-2 
update build status link in README
 2019-02-07 16:36:58 +00:00
roignpar
743ff9dcd8
update build status link in README 
License: MIT
Signed-off-by: Robert Ignat <robert.ignat91@gmail.com>
 2019-02-07 13:42:10 +02:00
Hector Sanjuan
4324889c79
Merge pull request #659 from ipfs/fix/update-json-config 
Fix/update json config
 2019-02-06 22:43:51 +00:00
Hector Sanjuan
789ab57c2c Fix #658: Improve test 
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
 2019-02-06 22:41:29 +00:00