Release N.1.8.3.0

The release includes a feature and four improvements and bugfixes.

• Support for path manifests is introduced

  Path manifests are a simple and optional extension for Arweave gateways; they allow users to upload a small metadata transaction (the path manifest), which maps user-definable subpaths with other Arweave transaction IDs. This allows users to create logical groupings of content, for example, a directory of related files, or the files and assets that make up a web application.

  For more details, check the wiki page about path manifests. It describes the schema and provides an example.

• The benchmark was made more precise

  Now it collects the actual number of hashes the mining processes generate instead of estimating them from the number of mined blocks. As a result,
  the benchmark fluctuates less and reports slightly higher numbers.

  Link.

• The propagation of blocks and transactions was improved slightly

  Link.

• A bug was fixed where nodes would sometimes fail to join the network

  Link.

• An unstable memory monitoring process was removed that sometimes caused the nodes to stop mining

  Link.

Upgrade instructions:

1. Shut down arweave-server. To make sure the shutdown was successful, run ps aux | grep beam | grep -v grep. The output should be empty.
2. Run git fetch --all --tags && git checkout N.1.8.3.0 && git reset --hard N.1.8.3.0
3. Start arweave-server again with the same arguments as before.

For more details on how to run the node, please see our mining guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server or email us on team@arweave.org.

Release N.1.8.2.0

The release contains three improvements.

1. RandomX tuning

Three RandomX parameters (used to be always off) can be now enabled or disabled via the command line:

• JIT

  Switched on by default on Linux. Can be disabled via disable randomx_jit.

  The performance boost can be ~ 5x.

• Hardware AES

  Switched on by default. Can be disabled via disable randomx_hardware_aes.

  The performance improvement is about 30%.

• Large pages

  Switched off by default. The OS needs to be configured before it can be used.

  On Ubuntu, to see the current values, execute cat /proc/meminfo | grep HugePages.
  To set a value, sudo sysctl -w vm.nr_hugepages=2000. 2000 * 2 MB = 4GB. Consider
  increasing the value if you have more memory. 400 multiplied by the number of logical cores
  you are mining on might work best in case you have so much RAM. Check the RandomX
  repository for more details - https://github.com/tevador/RandomX.

  Should give the performance improvement around 2x.

2. Fewer forks

The issue was addressed when nodes would reject a height + 1 block from an alternative
chain to incur additional cost for the potential minority to store content, which is
considered illicit by the majority. Nodes now make an explicit content scan and only
wait for yet another block if the scan fails.

3. Pessimistic pricing

The /price endpoint was made more pessimistic, to ensure almost every accepted transaction
is not invalidated before being included into a block when the network difficulty decreases.

Upgrade instructions:

1. Shut down arweave-server. To make sure the shutdown was successful, run ps aux | grep beam | grep -v grep. The output should be empty.
2. Run git fetch --all --tags && git checkout N.1.8.2.0 && git reset --hard N.1.8.2.0
3. Start arweave-server again with the same arguments as before.

For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server or email us on team@arweave.org.

Release N.1.8.1.0

**New features in the release:**

1. A new interface to query TX metadata using GraphQL.

The main issues addressed by the new interface are:

- Use an existing and well-defined query language to let apps fetch data from the weave
- Reduce the number of roundtrips required to get an exploitable dataset to use in an app on the permaweb
- Allow querying through user-defined relationships between transactions

GraphQL requests are processed by the POST /arql endpoint. The ArQL language
is still supported - the requests are dispatched to the corresponding backend
according to the received query.

The new queries are resolved using the same backing database as the ArQL queries,
meaning that the feature only presents negligible performance implications compared
to the previous implementation.

GraphQL queries are validated against the schema defined in priv/schema.graphql.

2. Webhooks.

A node can be configured to post blocks and transactions to the specified HTTP endpoints.
Failed requests are retried.

**Bugfixes and improvements:**

1. Fixes a race condition during block verification that causes valid blocks
   to occasionally be rejected (~13% of the time), leading to a lower chance of
   forks occurring.
2. The efficiency of block application is improved.

**Upgrade instructions:**

1. Shut down arweave-server. To make sure the shutdown was successful, run `ps aux | grep beam | grep -v grep`. The output should be empty.
2. Run `git fetch --all --tags && git checkout N.1.8.1.0 && git reset --hard N.1.8.1.0`
3. Start arweave-server again with the same arguments as before.

For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server or email us on team@arweave.org.

Release N.1.8.0.0

The release comes with a hard fork which activates on 2019-08-29 approximately at 14:00 CET.

**New features in the release:**

1. Multiple transactions per wallet per block.

After the fork, multiple transactions made from the same wallet can be posted to a node and included into one or more subsequent blocks.

The client-side support (JS SDK, arweave-deploy, browser extensions) is coming soon. In case you want to prepare transactions yourself, query the new GET /tx_anchor endpoint and put the result into the last_tx field of your transaction. You can use the same "anchor" for multiple transactions. Until the fork activates, there is still a limitation of one transaction per wallet per block. For backward-compatibility reasons, we are keeping support for the old way of building transactions using the last transaction of the wallet for last_tx. Subsequently, the current apps continue functioning. Note though that only one transaction built the old way per wallet per block is included into the block, even after the fork.

2. Transaction data size limit has increased to 10 MB.

3. Linear difficulty.

Effective from the first block after the fork, the network difficulty is a number the hash must be greater than. `2 ** 256 / (2 ** 256 - Diff)` is an average number of hashes one needs to try to mine a block. The new format allows us to do smooth increments and decrements of the difficulty, proportional to the ratio of the actual time to the target time. As a result, block time becomes more stable.

**Bugfixes and improvements:**

1. The performance of transaction verification was improved to account for the increase in the expected number and data volume of transactions.
2. The GET /wallet/[Addr]/txs(/[EarliestTX) endpoint is now based on the ArQL index and works much faster for the wallets with many transactions.
3. The POST /arql endpoint returns results sorted. arweave.net nodes have been already running this for a while.
4. There are updates to the pooling mechanics.
5. Various reliability improvements were made to ensure smooth node operation.

**Upgrade instructions:**

1. Shut down arweave-server. To make sure the shutdown was successful, run `ps aux | grep beam | grep -v grep`. The output should be empty.
2. Run `git fetch --all --tags && git checkout N.1.8.0.0 && git reset --hard N.1.8.0.0`
3. Start arweave-server again with the same arguments as before.

For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord https://discord.gg/bC5W2T or email us on team@arweave.org.

Release N.1.7.1.0

Arweave node release N.1.7.1.0 is out!

New features
   1. Gateway

      The gateway is a separate HTTP listener with TLS support that
      improves on how content hosted on the permaweb is served to web
      browsers. It's disabled by default. Here are the key features:

         * Listens on the standard HTTP and HTTPS ports.
         * Serve pages from isolated origins to allow browsers to
           properly sandbox each page's local data such as cookies,
           and make these inaccessible to other pages hosted on the
           same gateway.
         * Supports custom domain names with HTTPS, in other words,
           map domain names to pages on the Permaweb.

      Don't hesitate to reach out for help setting up a gateway.

   2. Stricter checks of blocks as soon as they are received from the
      network.

Bug fixes

   1. Crash when a malformed ARQL query is received.

   2. Invalid default value for disk_space if not specified by the
      user.

Upgrade instructions

   1. Shut down arweave-server. To make sure the shutdown was
      successful, run `ps aux | grep beam | grep -v grep`. The output
      should be empty.

   2. Run `git fetch --all --tags && git checkout N.1.7.1.0 && git
      reset --hard N.1.7.1.0`

   3. Start arweave-server again with the same arguments as before.

   For more details on how to run the node, please see our mining
   guide - https://docs.arweave.org/info/mining/mining-guide.

If you have any issues upgrading or would like to know more about the
release, feel free to reach out to us in the #mining channel on
Discords https://discord.gg/bC5W2T or email us on team@arewave.org.

Release N.1.7.0.0

We're very happy to announce the N.1.7.0.0 release of the Arweave node!

N.1.7.0.0 includes a lot of changes since the last release, N.1.6.3.1,
from three months ago. This release enables a hard fork scheduled to
activate at 2019-07-08 UTC. Please make sure to upgrade all your nodes
before the fork happens.

New features

   1. RandomX proof-of-work algorithm

      RandomX is a PoW algorithm that is optimized for general-purpose
      CPUs. RandomX uses random code execution together with several
      memory-hard techniques to maximize the efficiency of
      general-purpose CPUs and therefore make GPU accelerated miners
      and ASIC based miners less likely to be advantageous.

      The switch is a hard fork and will be effective from block
      height 235200 which will happen already around 2019-07-08 UTC.
      Please test and upgrade your miners well in advance before the
      fork happens.

      The mining benchmark is updated with a new argument for choosing
      RandomX hashing or the previous SHA-384 hashing. E.g.
      `./arweave-server benchmark randomx max_miners 4`. A bug was
      also fixed which halved all the performance numbers.

      More information available in commit 5148a682
      (https://github.com/ArweaveTeam/arweave/commit/5148a6826db98c49c59ef3eaff17f2c657851939).

   2. Two-way integration with IPFS

      Arweave is launching an IPFS incentivisation layer, providing
      permanent storage for the IPFS network. Read more about it here:
      https://medium.com/@arweave/arweave-ipfs-persistence-for-the-interplanetary-file-system-9f12981c36c3

      Uploading Arweave transactions to IPFS

         All Arweave transactions with the tag "IPFS-Add" will be
         uploaded and pinned to your local IPFS node, if this feature
         is enabled. This will help you make your transactions
         available on IPFS with minimal effort. See
         doc/ar-ipfs-howto.md
         (https://github.com/ArweaveTeam/arweave/blob/N.1.7.0.0/doc/ar-ipfs-howto.md)
         for how to enable this.

      Import IPFS objects to Arweave

         New API endpoints are available where IPFS CIDs can be posted.
         The Arweave node then downloads the object from IPFS and
         uploads it to Arweave using your wallet. This makes IPFS
         objects permanently stored. See doc/ipfs-ar-daemon.md
         (https://github.com/ArweaveTeam/arweave/blob/N.1.7.0.0/doc/ipfs-ar-daemon.md)
         for how to enable and use this feature.

      We have been running this IPFS integration for quite some time
      but it's still in the early stage, especially on the
      documentation side which is far from finished. We're happy to
      help you with the IPFS integration in the mining channel on our
      Discord server (see below).

   3. Content policy improvements

      The content policy feature (see the `content_policy` and
      `transaction_blacklist` startup arguments) is improved. Content
      denied by your policy will not be stored on your node.

    See content_policy_docs.md
    (https://github.com/ArweaveTeam/arweave/blob/N.1.7.0.0/content_policy_docs.md)

   4. Custom data directory

      There is a new startup argument called `data_dir` which gives
      the possibility to store the data (blocks, transactions and
      metadata) in any other directory instead of having it the
      application directory.

   5. Config file

      All startup arguments can be put in a JSON based config file and
      be loaded with the new startup argument `config_file`.

      E.g. `arweave-server config_file my-config-file.json`

   6. Export block and transaction metadata

      Metadata for all blocks and transactions can be exported to CSV
      files. See the `app_block_tx_export` module
      (src/apps/app_block_tx_export.erl
      (https://github.com/ArweaveTeam/arweave/blob/N.1.7.0.0/src/apps/app_block_tx_export.erl)).

Reliability and operations improvements

   1. Cowboy

      The HTTP server Elli is replaced with Cowboy.

   2. Reduced memory consumption

      Several improvements are made to reduce the memory consumption
      and keep it more even and less 'spiky'.

   3. Mixed improvements to logging, reduced disk operations, tweaked
      timeouts, overall fault-tolerance.

Bug fixes

   1. Polling mode stalls when a certain block doesn't exist on disk.

   2. The price endpoint is not pessimistic regarding future price,
      resulting in transactions becoming invalid before they are mined
      (due to the price increasing).

Upgrade instructions

   Please make sure to upgrade all of your nodes well in advance
   before the fork happens on 2019-07-08, but test upgrading only one
   node to begin with.

   1. Shut down arweave-server. To make sure the shutdown was
      successful, run `ps aux | grep beam | grep -v grep`. The output
      should be empty.

   2. Run `git fetch --all --tags && git checkout N.1.7.0.0 && git
      reset --hard N.1.7.0.0`

   3. Start arweave-server again with the same arguments as before.

   For more details on how to run the node, please see our mining
   guide - https://docs.arweave.org/info/mining/mining-guide.

   Pay attention to the recommended list of trusted peers (the `peer`
   startup argument). The list has been updated in the mining guide
   since the previous release. Only add peers you trust as this
   argument.

   If you get GCC compile errors, make sure you’re using GCC version 7
   or later.

The best way to get support for upgrading and running this release is
in the "mining" channel on our "Arweave Dev Talk" Discord server
(https://discord.gg/RZN9ab). You can also email to team@arweave.org.

Release N.1.6.3.0

1. The `block/hash/[hash]` and `block/height/[height]` HTTP endpoints do not return a hash list by default anymore. Viewing blocks in a browser/other HTTP client is now much faster!
2. Reliability improvements of the time measurement for difficulty adjustments.
3. Various reliability improvements for the mining worker processes.
4. New mining benchmark tool, using real candidate block production to much more accurately measure mining performance. You can run it with `./arweave-server benchmark`.
Note: As the new benchmarking tool tests real block production speed, not just pure hashing power, it may appear that the effective hashing rate has decreased. This is false. As it happens, the changes mentioned in point 3 lead to improved mining performance.

Upgrade instructions:

1. Shut down arweave-server. To make sure the shutdown was successful, run `ps aux | grep beam`. The output should only have a single line with `grep beam` at the end.
2. git fetch --all --tags && git checkout master && git reset --hard N.1.6.3.0.
3. Start arweave-server again with the same arguments as before.

For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide.

Pay attention to the recommended list of peers. It has not been updated since the last release. These peers are all operated by the Arweave organization, so use them if you trust them - alternatively join on the IP address of another trusted node.

N.1.6.0.0

The new features in 1.6:

• Extremely fast ( O(1) ) independent hash and block list verification.
• Tweaks to the difficulty adjustment algorithm.
• Redistribution of soft on-boarding rewards over Y1 (in ‘network time’).

Bonus: A wallet name generator that allows you to find addresses that start with a given prefix.

N.1.5.0.0 FINAL

Bump version and release numbers

N.1.5.0.0-RC-6

Bump release number.