Skip to content
Permalink
Branch: master
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
903 lines (646 sloc) 61.1 KB

IPFS Cluster Changelog

v0.10.0 - 2019-03-07

Summary

As we get ready to introduce a new CRDT-based "consensus" component to replace Raft, IPFS Cluster 0.10.0 prepares the ground with substancial under-the-hood changes. many performance improvements and a few very useful features.

First of all, this release requires users to run state upgrade (or start their daemons with ipfs-cluster-service daemon --upgrade). This is the last upgrade in this fashion as we turn to go-datastore-based storage. The next release of IPFS Cluster will not understand or be able to upgrade anything below 0.10.0.

Secondly, we have made some changes to internal types that should greatly improve performance a lot, particularly calls involving large collections of items (pin ls or status). There are also changes on how the state is serialized, avoiding unnecessary in-memory copies. We have also upgraded the dependency stack, incorporating many fixes from libp2p.

Thirdly, our new great features:

  • ipfs-cluster-ctl pin add/rm now supports IPFS paths (/ipfs/Qmxx.../..., /ipns/Qmxx.../..., /ipld/Qm.../...) which are resolved automatically before pinning.
  • All our configuration values can now be set via environment variables, and these will be reflected when initializing a new configuration file.
  • Pins can now specify a list of "priority allocations". This allows to pin items to specific Cluster peers, overriding the default allocation policy.
  • Finally, the REST API supports adding custom metadata entries as key=value (we will soon add support in ipfs-cluster-ctl). Metadata can be added as query arguments to the Pin or PinPath endpoints: POST /pins/<cid-or-path>?meta-key1=value1&meta-key2=value2...

Note that on this release we have also removed a lot of backwards-compatiblity code for things older than version 0.8.0, which kept things working but printed respective warnings. If you're upgrading from an old release, consider comparing your configuration with the new default one.

List of changes

Features
Bug fixes

Upgrading notices

This release needs an state upgrade before starting the Cluster daemon. Run ipfs-cluster-service state upgrade or run it as ipfs-cluster-service daemon --upgrade. We recommend backing up the ~/.ipfs-cluster folder or exporting the pinset with ipfs-cluster-service state export.

Configuration changes

Configurations now respects environment variables for all sections. They are in the form:

CLUSTER_COMPONENTNAME_KEYNAMEWITHOUTSPACES=value

Environment variables will override service.json configuration options when defined and the Cluster peer is started. ipfs-cluster-service init will reflect the value of any existing environment variables in the new service.json file.

REST API

The main breaking change to the REST API corresponds to the JSON representation of CIDs in response objects:

  • Before: "cid": "Qm...."
  • Now: "cid": { "/": "Qm...."}

The new CID encoding is the default as defined by the cid library. Unfortunately, there is no good solution to keep the previous representation without copying all the objects (an innefficient technique we just removed). The new CID encoding is otherwise aligned with the rest of the stack.

The API also gets two new "Path" endpoints:

  • POST /pins/<ipfs|ipns|ipld>/<path>/... and
  • DELETE /pins/<ipfs|ipns|ipld>/<path>/...

Thus, it is equivalent to pin a CID with POST /pins/<cid> (as before) or with POST /pins/ipfs/<cid>.

The calls will however fail when a non-compliant IPFS path is provided: POST /pins/<cid>/my/path will fail because all paths must start with the /ipfs, /ipns or /ipld components.

Go APIs

This release introduces lots of changes to the Go APIs, including the Go REST API client, as we have started returning pointers to objects rather than the objects directly. The Pin will now take api.PinOptions instead of different arguments corresponding to the options. It is aligned with the new PinPath and UnpinPath.

Other

As pointed above, 0.10.0's state migration is a required step to be able to use future version of IPFS Cluster.


v0.9.0 - 2019-02-18

Summary

IPFS Cluster version 0.9.0 comes with one big new feature, OpenCensus support! This allows for the collection of distributed traces and metrics from the IPFS Cluster application as well as supporting libraries. Currently, we support the use of Jaeger as the tracing backend and Prometheus as the metrics backend. Support for other OpenCensus backends will be added as requested by the community.

List of changes

Features
Bug Fixes

No bugs were fixed from the previous release.

Deprecated

Upgrading notices

Configuration changes

No changes to the existing configuration.

There are two new configuration sections with this release:

tracing section

The tracing section configures the use of Jaeger as a tracing backend.

    "tracing": {
      "enable_tracing": false,
      "jaeger_agent_endpoint": "/ip4/0.0.0.0/udp/6831",
      "sampling_prob": 0.3,
      "service_name": "cluster-daemon"
    }
metrics section

The metrics section configures the use of Prometheus as a metrics collector.

    "metrics": {
      "enable_stats": false,
      "prometheus_endpoint": "/ip4/0.0.0.0/tcp/8888",
      "reporting_interval": "2s"
    }
REST API

No changes to the REST API.

Go APIs

The Go APIs had the minor change of having a context.Context parameter added as the first argument to those that didn't already have it. This was to enable the proporgation of tracing and metric values.

The following is a list of interfaces and their methods that were affected by this change:

  • Component
    • Shutdown
  • Consensus
    • Ready
    • LogPin
    • LogUnpin
    • AddPeer
    • RmPeer
    • State
    • Leader
    • WaitForSync
    • Clean
    • Peers
  • IpfsConnector
    • ID
    • ConnectSwarm
    • SwarmPeers
    • RepoStat
    • BlockPut
    • BlockGet
  • Peered
    • AddPeer
    • RmPeer
  • PinTracker
    • Track
    • Untrack
    • StatusAll
    • Status
    • SyncAll
    • Sync
    • RecoverAll
    • Recover
  • Informer
    • GetMetric
  • PinAllocator
    • Allocate
  • PeerMonitor
    • LogMetric
    • PublishMetric
    • LatestMetrics
  • state.State
    • Add
    • Rm
    • List
    • Has
    • Get
    • Migrate
  • rest.Client
    • ID
    • Peers
    • PeerAdd
    • PeerRm
    • Add
    • AddMultiFile
    • Pin
    • Unpin
    • Allocations
    • Allocation
    • Status
    • StatusAll
    • Sync
    • SyncAll
    • Recover
    • RecoverAll
    • Version
    • IPFS
    • GetConnectGraph
    • Metrics

These interface changes were also made in the respective implementations. All export methods of the Cluster type also had these changes made.

Other

No other things.


v0.8.0 - 2019-01-16

Summary

IPFS Cluster version 0.8.0 comes with a few useful features and some bugfixes. A significant amount of work has been put to correctly handle CORS in both the REST API and the IPFS Proxy endpoint, fixing some long-standing issues (we hope once are for all).

There has also been heavy work under the hood to separate the IPFS HTTP Connector (the HTTP client to the IPFS daemon) from the IPFS proxy, which is essentially an additional Cluster API. Check the configuration changes section below for more information about how this affects the configuration file.

Finally we have some useful small features:

  • The ipfs-cluster-ctl status --filter option allows to just list those items which are still pinning or queued or error etc. You can combine multiple filters. This translates to a new filter query parameter in the /pins API endpoint.
  • The stream-channels=false query parameter for the /add endpoint will let the API buffer the output when adding and return a valid JSON array once done, making this API endpoint behave like a regular, non-streaming one. ipfs-cluster-ctl add --no-stream acts similarly, but buffering on the client side. Note that this will cause in-memory buffering of potentially very large responses when the number of added files is very large, but should be perfectly fine for regular usage.
  • The ipfs-cluster-ctl add --quieter flag now applies to the JSON output too, allowing the user to just get the last added entry JSON object when adding a file, which is always the root hash.

List of changes

Features
Bug fixes

Upgrading notices

This release comes with some configuration changes that are important to notice, even though the peers will start with the same configurations as before.

Configuration changes
ipfsproxy section

This version introduces a separate ipfsproxy API component. This is reflected in the service.json configuration, which now includes a new ipfsproxy subsection under the api section. By default it looks like:

    "ipfsproxy": {
      "node_multiaddress": "/ip4/127.0.0.1/tcp/5001",
      "listen_multiaddress": "/ip4/127.0.0.1/tcp/9095",
      "read_timeout": "0s",
      "read_header_timeout": "5s",
      "write_timeout": "0s",
      "idle_timeout": "1m0s"
   }

We have however added the necessary safeguards to keep backwards compatibility for this release. If the ipfsproxy section is empty, it will be picked up from the ipfshttp section as before. An ugly warning will be printed in this case.

Based on the above, the ipfshttp configuration section loses the proxy-related options. Note that node_multiaddress stays in both component configurations and should likely be the same in most cases, but you can now potentially proxy requests to a different daemon than the one used by the cluster peer.

Additional hidden configuration options to manage custom header extraction from the IPFS daemon (for power users) have been added to the ipfsproxy section but are not shown by default when initializing empty configurations. See the documentation for more details.

restapi section

The introduction of proper CORS handling in the restapi component introduces a number of new keys:

      "cors_allowed_origins": [
        "*"
      ],
      "cors_allowed_methods": [
        "GET"
      ],
      "cors_allowed_headers": [],
      "cors_exposed_headers": [
        "Content-Type",
        "X-Stream-Output",
        "X-Chunked-Output",
        "X-Content-Length"
      ],
      "cors_allow_credentials": true,
      "cors_max_age": "0s"

Note that CORS will be essentially unconfigured when these keys are not defined.

The headers key, which was used before to add some CORS related headers manually, takes a new empty default. We recommend emptying headers from any CORS-related value.

REST API

The REST API is fully backwards compatible:

  • The GET /pins endpoint takes a new ?filter=<filter> option. See ipfs-cluster-ctl status --help for acceptable values.
  • The POST /add endpoint accepts a new ?stream-channels=<true|false> option. By default it is set to true.
Go APIs

The signature for the StatusAll method in the REST client module has changed to include a filter parameter.

There may have been other minimal changes to internal exported Go APIs, but should not affect users.

Other

Proxy requests which are handled by the Cluster peer (/pin/ls, /pin/add, /pin/rm, /repo/stat and /add) will now attempt to fully mimic ipfs responses to the header level. This is done by triggering CORS pre-flight for every hijacked request along with an occasional regular request to /version to extract other headers (and possibly custom ones).

The practical result is that the proxy now behaves correctly when dropped instead of IPFS into CORS-aware contexts (like the browser).


v0.7.0 - 2018-11-01

Summary

IPFS Cluster version 0.7.0 is a maintenance release that includes a few bugfixes and some small features.

Note that the REST API response format for the /add endpoint has changed. Thus all clients need to be upgraded to deal with the new format. The rest/api/client has been accordingly updated.

List of changes

Features
Bug fixes

Upgrading notices

Configuration changes

The configurations from previous versions are compatible, but a new headers key has been added to the restapi section. By default it gets CORS headers which will allow read-only interaction from any origin.

Additionally, all fields from the main cluster configuration section can now be overwrriten with environment variables. i.e. CLUSTER_SECRET, or CLUSTER_DISABLEREPINNING.

REST API

The /add endpoint stream now returns different objects, in line with the rest of the API types.

Before:

type AddedOutput struct {
	Error
	Name  string
	Hash  string `json:",omitempty"`
	Bytes int64  `json:",omitempty"`
	Size  string `json:",omitempty"`
}

Now:

type AddedOutput struct {
	Name  string `json:"name"`
	Cid   string `json:"cid,omitempty"`
	Bytes uint64 `json:"bytes,omitempty"`
	Size  uint64 `json:"size,omitempty"`
}

The /add endpoint no longer reports errors as part of an AddedOutput object, but instead it uses trailer headers (same as go-ipfs). They are handled in the client.

Go APIs

The AddedOutput object has changed, thus the api/rest/client from older versions will not work with this one.

Other

No other things.


v0.6.0 - 2018-10-03

Summary

IPFS version 0.6.0 is a new minor release of IPFS Cluster.

We have increased the minor release number to signal changes to the Go APIs after upgrading to the new cid package, but, other than that, this release does not include any major changes.

It brings a number of small fixes and features of which we can highlight two useful ones:

  • the first is the support for multiple cluster daemon versions in the same cluster, as long as they share the same major/minor release. That means, all releases in the 0.6 series (0.6.0, 0.6.1 and so on...) will be able to speak among each others, allowing partial cluster upgrades.
  • the second is the inclusion of a PeerName key in the status (PinInfo) objects. ipfs-cluster-status will now show peer names instead of peer IDs, making it easy to identify the status for each peer.

Many thanks to all the contributors to this release: @lanzafame, @meiqimichelle, @kishansagathiya, @cannium, @jglukasik and @mike-ngu.

List of changes

Features
Bugfixes

Upgrading notices

Configuration changes

There are no changes to the configuration file on this release.

REST API

There are no changes to the REST API.

Go APIs

We have upgraded to the new version of the cid package. This means all *cid.Cid arguments are now cid.Cid.

Other

We are now using go-1.11 to build and test cluster. We recommend using this version as well when building from source.


v0.5.0 - 2018-08-23

Summary

IPFS Cluster version 0.5.0 is a minor release which includes a major feature: adding content to IPFS directly through Cluster.

This functionality is provided by ipfs-cluster-ctl add and by the API endpoint /add. The upload format (multipart) is similar to the IPFS /add endpoint, as well as the options (chunker, layout...). Cluster add generates the same DAG as ipfs add would, but it sends the added blocks directly to their allocations, pinning them on completion. The pin happens very quickly, as content is already locally available in the allocated peers.

The release also includes most of the needed code for the Sharding feature, but it is not yet usable/enabled, pending features from go-ipfs.

The 0.5.0 release additionally includes a new experimental PinTracker implementation: the stateless pin tracker. The stateless pin tracker relies on the IPFS pinset and the cluster state to keep track of pins, rather than keeping an in-memory copy of the cluster pinset, thus reducing the memory usage when having huge pinsets. It can be enabled with ipfs-cluster-service daemon --pintracker stateless.

The last major feature is the use of a DHT as routing layer for cluster peers. This means that peers should be able to discover each others as long as they are connected to one cluster peer. This simplifies the setup requirements for starting a cluster and helps avoiding situations which make the cluster unhealthy.

This release requires a state upgrade migration. It can be performed with ipfs-cluster-service state upgrade or simply launching the daemon with ipfs-cluster-service daemon --upgrade.

List of changes

Features
Bugfixes

Upgrading notices

Configuration files

IMPORTANT: 0s is the new default for the read_timeout and write_timeout values in the restapi configuration section, as well as proxy_read_timeout and proxy_write_timeout options in the ipfshttp section. Adding files to cluster (via the REST api or the proxy) is likely to timeout otherwise.

The peerstore file (in the configuration folder), no longer requires listing the multiaddresses for all cluster peers when initializing the cluster with a fixed peerset. It only requires the multiaddresses for one other cluster peer. The rest will be inferred using the DHT. The peerstore file is updated only on clean shutdown, and will store all known multiaddresses, even if not pertaining to cluster peers.

The new stateless PinTracker implementation uses a new configuration subsection in the pin_tracker key. This is only generated with ipfs-cluster-service init. When not present, a default configuration will be used (and a warning printed).

The state_sync_interval default has been increased to 10 minutes, as frequent syncing is not needed with the improvements in the PinTracker. Users are welcome to update this setting.

REST API

The /add endpoint has been added. The replication_factor_min and replication_factor_max options (in POST allocations/<cid>) have been deprecated and subsititued for replication-min and replication-max, although backwards comaptibility is kept.

Keep Alive has been disabled for the HTTP servers, as a bug in Go's HTTP client implementation may result adding corrupted content (and getting corrupted DAGs). However, while the libp2p API endpoint also suffers this, it will only close libp2p streams. Thus the performance impact on the libp2p-http endpoint should be minimal.

Go APIs

The Config.PeerAddr key in the rest/client module is deprecated. APIAddr should be used for both HTTP and LibP2P API endpoints. The type of address is automatically detected.

The IPFSConnector Pin call now receives an integer instead of a Recursive flag. It indicates the maximum depth to which something should be pinned. The only supported value is -1 (meaning recursive). BlockGet and BlockPut calls have been added to the IPFSConnector component.

Other

As noted above, upgrade to state format version 5 is needed before starting the cluster service.


v0.4.0 - 2018-05-30

Summary

The IPFS Cluster version 0.4.0 includes breaking changes and a considerable number of new features causing them. The documentation (particularly that affecting the configuration and startup of peers) has been updated accordingly in https://cluster.ipfs.io . Be sure to also read it if you are upgrading.

There are four main developments in this release:

  • Refactorings around the consensus component, removing dependencies to the main component and allowing separate initialization: this has prompted to re-approach how we handle the peerset, the peer addresses and the peer's startup when using bootstrap. We have gained finer control of Raft, which has allowed us to provide a clearer configuration and a better start up procedure, specially when bootstrapping. The configuration file no longer mutates while cluster is running.
  • Improvements to the pintracker: our pin tracker is now able to cancel ongoing pins when receiving an unpin request for the same CID, and vice-versa. It will also optimize multiple pin requests (by only queuing and triggering them once) and can now report whether an item is pinning (a request to ipfs is ongoing) vs. pin-queued (waiting for a worker to perform the request to ipfs).
  • Broadcasting of monitoring metrics using PubSub: we have added a new monitor implementation that uses PubSub (rather than RPC broadcasting). With the upcoming improvements to PubSub this means that we can do efficient broadcasting of metrics while at the same time not requiring peers to have RPC permissions, which is preparing the ground for collaborative clusters.
  • We have launched the IPFS Cluster website: https://cluster.ipfs.io . We moved most of the documentation over there, expanded it and updated it.

List of changes

Features
Bugsfixes:

Upgrading notices

Configuration file

This release introduces breaking changes to the configuration file. An error will be displayed if ipfs-cluster-service is started with an old configuration file. We recommend re-initing the configuration file altogether.

  • The peers and bootstrap keys have been removed from the main section of the configuration
  • You might need to provide Peer multiaddresses in a text file named peerstore, in your ~/.ipfs-cluster folder (one per line). This allows your peers how to contact other peers.
  • A disable_repinning option has been added to the main configuration section. Defaults to false.
  • A init_peerset has been added to the raft configuration section. It should be used to define the starting set of peers when a cluster starts for the first time and is not bootstrapping to an existing running peer (otherwise it is ignored). The value is an array of peer IDs.
  • A backups_rotate option has been added to the raft section and specifies how many copies of the Raft state to keep as backups when the state is cleaned up.
  • An ipfs_request_timeout option has been introduced to the ipfshttp configuration section, and controls the timeout of general requests to the ipfs daemon. Defaults to 5 minutes.
  • A pin_timeout option has been introduced to the ipfshttp section, it controls the timeout for Pin requests to ipfs. Defaults to 24 hours.
  • An unpin_timeout option has been introduced to the ipfshttp section. it controls the timeout for Unpin requests to ipfs. Defaults to 3h.
  • Both pinning_timeout and unpinning_timeout options have been removed from the maptracker section.
  • A monitor/pubsubmon section configures the new PubSub monitoring component. The section is identical to the existing monbasic, its only option being check_interval (defaults to 15 seconds).

The ipfs-cluster-data folder has been renamed to raft. Upon ipfs-cluster-service daemon start, the renaming will happen automatically if it exists. Otherwise it will be created with the new name.

REST API

There are no changes to REST APIs in this release.

Go APIs

Several component APIs have changed: Consensus, PeerMonitor and IPFSConnector have added new methods or changed methods signatures.

Other

Calling ipfs-cluster-service without subcommands no longer runs the peer. It is necessary to call ipfs-cluster-service daemon. Several daemon-specific flags have been made subcommand flags: --bootstrap and --alloc.

The --bootstrap flag can now take a list of comma-separated multiaddresses. Using --bootstrap will automatically run state clean.

The ipfs-cluster-ctl no longer has a peers add subcommand. Peers should not be added this way, but rather bootstrapped to an existing running peer.


v0.3.5 - 2018-03-29

This release comes full with new features. The biggest ones are the support for parallel pinning (using refs -r rather than pin add to pin things in IPFS), and the exposing of the http endpoints through libp2p. This allows users to securely interact with the HTTP API without having to setup SSL certificates.

There are no breaking API changes and all configurations should be backwards compatible. The api/rest/client provides a new IPFS() method.

We recommend updating the service.json configurations to include all the new configuration options:

  • The pin_method option has been added to the ipfshttp section. It supports refs and pin (default) values. Use refs for parallel pinning, but only if you don't run automatic GC on your ipfs nodes.
  • The concurrent_pins option has been added to the maptracker section. Only useful with refs option in pin_method.
  • The listen_multiaddress option in the restapi section should be renamed to http_listen_multiaddress.

This release will require a state upgrade. Run ipfs-cluster-service state upgrade in all your peers, or start cluster with ipfs-cluster-service daemon --upgrade.


v0.3.4 - 2018-02-20

This release fixes the pre-built binaries.


v0.3.3 - 2018-02-12

This release includes additional ipfs-cluster-service state subcommands and the connectivity graph feature.

APIs have not changed in this release. The /health/graph endpoint has been added.


v0.3.2 - 2018-01-25

This release includes a number of bufixes regarding the upgrade and import of state, along with two important features:

These release is compatible with previous versions of ipfs-cluster on the API level, with the exception of the ipfs-cluster-service version command, which returns x.x.x-shortcommit rather than ipfs-cluster-service version 0.3.1. The former output is still available as ipfs-cluster-service --version.

The replication_factor option is deprecated, but still supported and will serve as a shortcut to set both replication_factor_min and replication_factor_max to the same value. This affects the configuration file, the REST API and the ipfs-cluster-ctl pin add command.


v0.3.1 - 2017-12-11

This release includes changes around the consensus state management, so that upgrades can be performed when the internal format changes. It also comes with several features and changes to support a live deployment and integration with IPFS pin-bot, including a REST API client for Go.

This release should stay backwards compatible with the previous one. Nevertheless, some REST API endpoints take the local flag, and matching new Go public functions have been added (RecoverAllLocal, SyncAllLocal...).


v0.3.0 - 2017-11-15

This release introduces Raft 1.0.0 and incorporates deep changes to the management of the cluster peerset.

Bugfixes:

This releases introduces some changes affecting the configuration file and some breaking changes affecting go and the REST APIs:

  • The consensus.raft section of the configuration has new options but should be backwards compatible.
  • The Consensus component interface has changed, LogAddPeer and LogRmPeer have been replaced by AddPeer and RmPeer. It additionally provides Clean and Peers methods. The consensus/raft implementation has been updated accordingly.
  • The api.ID (used in REST API among others) object key ClusterPeers key is now a list of peer IDs, and not a list of multiaddresses as before. The object includes a new key ClusterPeersAddresses which includes the multiaddresses.
  • Note that --bootstrap and --leave flags when calling ipfs-cluster-service will be stored permanently in the configuration (see ipfs/ipfs-cluster#235).

v0.2.1 - 2017-10-26

This is a maintenance release with some important bugfixes.

The fix for 32-bit architectures has required a change in the IPFSConnector interface (FreeSpace() and Reposize() return uint64 now). The current implementation by the ipfshttp module has changed accordingly.


v0.2.0 - 2017-10-23

This release introduces some breaking changes affecting configuration files and go integrations:

  • Config: The old configuration format is no longer valid and cluster will fail to start from it. Configuration file needs to be re-initialized with ipfs-cluster-service init.
  • Go: The restapi component has been renamed to rest and some of its public methods have been renamed.
  • Go: Initializers (New<Component>(...)) for most components have changed to accept a Config object. Some initializers have been removed.

Note, when adding changelog entries, write links to issues as @<issuenumber> and then replace them with links with the following command:

sed -i -r 's/@([0-9]+)/[ipfs\/ipfs-cluster#\1](https:\/\/github.com\/ipfs\/ipfs-cluster\/issues\/\1)/g' CHANGELOG.md
You can’t perform that action at this time.