…the rpc name

2248:  feat: add verbosity param to chain related rpc r=quake a=quake

This PR adds an optional `verbosity ` param to chain related rpc, returns data in hex format without calculated hash values, it will improve performance in some scenarios, ~~for example, ckb-indexer can scan 2,000,000 blocks in 13 minutes with this change (before: 20 minutes)~~ **This benchmark is not correct, I will rerun it and update**

=== updated ckb-indexer benchmark result ===
scan 2,000,000 blocks, run 3 times, avg:
before: 17 minutes 40 seconds
after: 12 minutes 30 seconds


=== response example ===
hex format: 
```
echo '{                                                                                     
    "id": 2,     
    "jsonrpc": "2.0",
    "method": "get_tip_header",
    "params": ["0x1"]
}' \
| tr -d '\n' \
| curl -H 'content-type: application/json' -d @- \
http://localhost:8114

{"jsonrpc":"2.0","result":"0x000000002872ae19e93f872d7401000027c92700000000009e06003d02080700903e152458ff060ddf290e2b876d94211be8982f4b10ba63b9b017abb587bb75f59cd510ec3e3e13ff331a41d954f3ed9b9980cdcea5c88b5e7bf6fcfcd2bc2c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005ed103b7d5f09434388f10dcba93240032b96b8ee42e3a01001579760524020700043bd6280000002b99f9d300010000","id":2}
```

v.s

json format
```
echo '{
    "id": 2,
    "jsonrpc": "2.0",
    "method": "get_tip_header",
    "params": []    
}' \
| tr -d '\n' \
| curl -H 'content-type: application/json' -d @- \
http://localhost:8114
{"jsonrpc":"2.0","result":{"compact_target":"0x19ae7228","dao":"0x5ed103b7d5f09434388f10dcba93240032b96b8ee42e3a010015797605240207","epoch":"0x708023d00069e","hash":"0x50f0a70e5fdbce353a46f018c1a6f1dd15cb7470fff8ebcb2b7b418cd1a5faab","nonce":"0x100d3f9992b00000028d63b0400","number":"0x27c927","parent_hash":"0x903e152458ff060ddf290e2b876d94211be8982f4b10ba63b9b017abb587bb75","proposals_hash":"0x0000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1742d873fe9","transactions_root":"0xf59cd510ec3e3e13ff331a41d954f3ed9b9980cdcea5c88b5e7bf6fcfcd2bc2c","uncles_hash":"0x0000000000000000000000000000000000000000000000000000000000000000","version":"0x0"},"id":2}
```

Co-authored-by: quake <quake.wang@gmail.com>

2251: feat: deprecated rpc method by adding `deprecated.` prefix to the rpc name r=doitian,zhangsoledad,yangby-cryptape a=quake

resolve nervosnetwork#2247, rpc method can be deprecated by adding `deprecated.` prefix to the rpc name:
```diff
+    #[rpc(name = "deprecated.get_cells_by_lock_hash")]
-    #[rpc(name = "get_cells_by_lock_hash")]
    fn get_cells_by_lock_hash
```

Co-authored-by: quake <quake.wang@gmail.com>

…041 temporarily

2266: fix: use forked metrics and forked sentry to fix RUSTSEC-2020-0041 temporarily r=yangby-cryptape a=yangby-cryptape

### Commits

- [chore(deps): bump crossbeam-channel from 0.4.3 to 0.4.4](nervosnetwork@728f5c3)
- [chore: remove direct dependencies of metrics exporters and observers](nervosnetwork@de3173a)
- [chore(deps): bump sentry from 0.16.0 to 0.17.0](nervosnetwork@1453b1a)
- [fix: use patched versions of metrics and sentry to fix RUSTSEC-2020-0041 temporarily](nervosnetwork@6aaae31)
  - [Changes of forked `metrics`](nervosnetwork/metrics-rs@4fd13f6)
  - [Changes of forked `sentry`](nervosnetwork/sentry-rust@0e964f5)

### References

- [RUSTSEC-2020-0041](../../../../rustsec/advisory-db/pull/381)

Co-authored-by: Boyu Yang <yangby@cryptape.com>

2236: refactor: rewrite discovery r=TheWaWaR,doitian a=driftluo

1. Discovery remove `Both` registration
2. Rewrite discovery protocol, the original implementation is too complicated and difficult to read, and there is a lot of unnecessary consumption
3. Use bloom filter to filter known address

Later, I will write these implementations back into Tentacle to avoid these problems with reference implementations

Co-authored-by: driftluo <driftluo@foxmail.com>

2274: chore: amend a log message which could cause users to be panic r=doitian,TheWaWaR a=yangby-cryptape

For example, the 8~10 lines of the log file for our all integration tests at present are:

> Failed to open AddrManager db, file: "XXX/addr_manager.db", error: Os { code: 2, kind: NotFound, message: "No such file or directory" }
> Failed to open BanList db, file: "XXX/ban_list.db", error: Os { code: 2, kind: NotFound, message: "No such file or directory" } 
> receiving shutdown signal ...

I think it's a bit confusing and scary.

Co-authored-by: Boyu Yang <yangby@cryptape.com>

2258: chore: integration test - log case name before start executing r=doitian,keroro520 a=chuijiaolianying



Co-authored-by: chuijiaolianying <chuijiaolianying@gmail.com>

2276: feat: improve migration progress display r=quake,doitian a=zhangsoledad

Add migrarion processbar, improve migration display

![2020-09-13 21-09-08 的屏幕截图](https://user-images.githubusercontent.com/3198439/93019008-e165c000-f606-11ea-8f9f-302a41aeefe4.png)


Co-authored-by: zhangsoledad <787953403@qq.com>

2276: feat: improve migration progress display r=quake,doitian a=zhangsoledad

Add migrarion processbar, improve migration display

![2020-09-13 21-09-08 的屏幕截图](https://user-images.githubusercontent.com/3198439/93019008-e165c000-f606-11ea-8f9f-302a41aeefe4.png)

Co-authored-by: zhangsoledad <787953403@qq.com>

2275: chore: all leftover registrations of `both` are cleared r=quake,TheWaWaR a=driftluo

Clean up the historical `Both` registration issues

Co-authored-by: driftluo <driftluo@foxmail.com>

2257: feat: add ping_peers rpc r=doitian,driftluo a=quake

This PR added a `ping_peers` rpc, which requests that a ping be sent to all connected peers, to measure ping time. 

for reviewer's reference: the code change of `network/src/network.rs` is using struct deconstruction and scoped clone to replace the original multiple rename clone.

before:
```
let foo = Arc::new(...);
let foo_clone = foo.clone();
thread::spawn(move || {
    let foo_clone_1 = foo_clone.clone();
    thread::spawn(move || {
        foo_clone_1.bar();
    });
    foo_clone.bar();
});
foo.bar();
```

now:
```
let foo = Arc::new(...);
{
    let foo = foo.clone();
    thread::spawn(move || {
        {
            let foo = foo.clone();
            thread::spawn(move || {
                foo.bar();
            });
        }
        foo.bar();
    });
}
foo.bar();
```

makes it easier to read and add new variables.


Co-authored-by: quake <quake.wang@gmail.com>

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.74 to 0.2.77.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](rust-lang/libc@0.2.74...0.2.77)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

…ver is on

2265: feat: add nMinimumChainWork config r=doitian,quake a=driftluo

The mainnet has been online for a long time, and it is time to add a minimum workload proof to prevent possible node attacks during the initial synchronization.

I chose a total difficulty of 500k block height, header first sync will take about 1-2 minutes to reach this position, then start downloading the blocks.

Co-authored-by: driftluo <driftluo@foxmail.com>

2285: fix: fix the problem of disconnection caused by uncertainty r=quake,yangby-cryptape a=driftluo



Co-authored-by: driftluo <driftluo@foxmail.com>

2284: chore: remove tmp directory dependencies r=jjyr,zhangsoledad,yangby-cryptape a=quake

I got a `PermissionDenied` error when running a lightnode on android which is using the `ckb-network` as a lib:
```
08-28 15:35:55.052  9853  9881 I RustStdoutStderr: 2020-08-28 15:35:55.052 +09:00 NetworkRuntime WARN ckb_network::services::dump_peer_store  Dump peer store error, path: "/storage/emulated/0/Android/data/rust.example.android/files/db/peer_store" error: Io(Custom { kind: PermissionDenied, error: PathError { path: "/data/local/tmp/.tmponp0Nt", err: Os { code: 13, kind: PermissionDenied, message: "Permission denied" } } })
```

This PR removes tmp directory dependencies, make it possible to using `ckb-network` as a lib on android and also running ckb full node on a system without tmp directory mouting

Co-authored-by: quake <quake.wang@gmail.com>

2278: chore: integration test - put case log together r=quake,yangby-cryptape a=chuijiaolianying

When debug integration case in local, we may run a single case lots time, then there is a big problem how to find the logs under an execution.

Old log structure:
```
├── ckb-it-get_blocks_timeout-net-G65fZ4
├── ckb-it-get_blocks_timeout-node0-LZRUJE
├── ckb-it-get_blocks_timeout-node1-11uL3C
```
New log structure:
```
├── ckb-it-get_blocks_timeout-UccnLI
│   ├── net
│   ├── node0
│   └── node1
```

Another change is updating `node.working_dir` and `net.working_dir` from `String` to `PathBuf`

Co-authored-by: chuijiaolianying <chuijiaolianying@gmail.com>

2283: fix: resolve network background service cleanup issue when rpc tcp server is on r=doitian,TheWaWaR,driftluo,yangby-cryptape a=quake

network background services (peer store dump, etc) can not be dropped automatically when rpc tcp service is on, this PR drop the rpc_server explicitly to resolve this issue. (I don't know the root reason, but it works)

Co-authored-by: quake <quake.wang@gmail.com>

…ver is on

Co-authored-by: quake <quake.wang@gmail.com>

2282: feat: create log monitor for integration test check status between nodes r=quake,keroro520 a=chuijiaolianying



Co-authored-by: chuijiaolianying <chuijiaolianying@gmail.com>

2263: chore(ci): Do not send integration test failures in forks. r=quake,keroro520,chuijiaolianying,yangby-cryptape a=doitian



Co-authored-by: ian <ian@nervos.org>

2269: feat: redesign cell store r=quake,driftluo a=zhangsoledad

## Description
Previous cell storage is inefficient, resolve transactions need fetch cell from block body, this PR proposal a new live cell storage schema, optimize resolve transaction bottleneck, I run a benchmark (500 2-in-2out secp lock txs/per block), `process block` get 4x faster.
It precondition `chain freezer` which intends to achieve historical block.

## Breaking changes
remove `get_cells_by_lock_hash` rpc.




2279: chore(deps): bump libc from 0.2.74 to 0.2.77 r=yangby-cryptape a=dependabot-preview[bot]

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.74 to 0.2.77.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/rust-lang/libc/releases">libc's releases</a>.</em></p>
<blockquote>
<h2>0.2.77</h2>
<p>Bump patch version to 0.2.77</p>
<h2>0.2.76</h2>
<p>Bump patch version to 0.2.76</p>
<h2>0.2.75</h2>
<p>Bump patch version to 0.2.75</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/rust-lang/libc/commit/c2a184b255c2d7548e17b66f19f0330192ce6484"><code>c2a184b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1879">#1879</a> from xonatius/0.2.77</li>
<li><a href="https://github.com/rust-lang/libc/commit/59544d95bcd1bce6f104752341556529ff9ce6e2"><code>59544d9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1887">#1887</a> from psumbera/master</li>
<li><a href="https://github.com/rust-lang/libc/commit/2a0b8fe39094a75529858e9d5f9b52cac60d2af2"><code>2a0b8fe</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1886">#1886</a> from sunfishcode/main</li>
<li><a href="https://github.com/rust-lang/libc/commit/c44fece72034203267860ccfd4f292d1577b5c0b"><code>c44fece</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1885">#1885</a> from MeiK2333/master</li>
<li><a href="https://github.com/rust-lang/libc/commit/9f6a7ceee819e847eb4e0218c4452ac1d0cffad4"><code>9f6a7ce</code></a> add pthread_getattr_np() and pthread_attr_getstack() support for illumos and ...</li>
<li><a href="https://github.com/rust-lang/libc/commit/a02da55c4b526e88769da6b745037999d8cc37d3"><code>a02da55</code></a> Define several more constants for WASI.</li>
<li><a href="https://github.com/rust-lang/libc/commit/7cc22dbabd6e17b3db0ba0f31345cf3c9bf9f4fc"><code>7cc22db</code></a> Add getitimer and setitimer for macOS</li>
<li><a href="https://github.com/rust-lang/libc/commit/af7749c1e52fbf4da14c08e3ec654c8119f079a8"><code>af7749c</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1882">#1882</a> from alindima/master</li>
<li><a href="https://github.com/rust-lang/libc/commit/d21380bb501b71f909b6366c733341474e36fa4a"><code>d21380b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1881">#1881</a> from JohnColanduoni/apple-o-symlink</li>
<li><a href="https://github.com/rust-lang/libc/commit/f19436de85b08a92f4e7f6ea1f0bb424ff3c5b90"><code>f19436d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1880">#1880</a> from Thomasdezeeuw/io-max-consts</li>
<li>Additional commits viewable in <a href="https://github.com/rust-lang/libc/compare/0.2.74...0.2.77">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=libc&package-manager=cargo&previous-version=0.2.74&new-version=0.2.77)](https://dependabot.com/compatibility-score/?dependency-name=libc&package-manager=cargo&previous-version=0.2.74&new-version=0.2.77)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)



</details>

Co-authored-by: zhangsoledad <787953403@qq.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

2191: chore: upload release for deployment to qiniu r=yangby-cryptape a=doitian

Build prod-with-debug and upload to qiniu directly in CI.

The job has been tested in https://travis-ci.com/github/nervosnetwork/ckb/builds/177427769

Co-authored-by: ian <ian@nervos.org>

2260: feat: add get_transaction_proof / verify_transaction_proof rpc r=doitian,TheWaWaR a=quake



Co-authored-by: quake <quake.wang@gmail.com>

2268: refactor: add some transaction checking utils r=quake,chuijiaolianying a=keroro520

Add [`test/src/util/check.rs`](https://github.com/nervosnetwork/ckb/pull/2268/files#diff-56a7c1b37b9c48a7b20036328fdb8125) some transaction checking utils.

Co-authored-by: keroro <keroroxx520@gmail.com>

2259: feat: add clear_banned_addresses rpc r=keroro520,zhangsoledad a=quake



Co-authored-by: quake <quake.wang@gmail.com>

* Change interface `fn before_run()` => `fn before_run() -> Vec<Node>`
* Change interface `fn run(&mut Net)` => `fn run(&mut Vec<Node>)`
* Change interfaces `fn modify_ckb_config` and `modify_chain_spec`
* Initialize context within `before_run`, which is originally at
  worker.rs
* Use struct name as spec name
* Remove `protocols` and `connect_all` from `Setup` options
* Remove `TestProtocol`. `ckb_network::SupportProtocols` is enough.

* Simplify the struct fields and interfaces. Reduce from 12 fields to 6 fields.
* Simplify the workflow of initializing ckb node. `Node` can be initialized via loading configs from working directory. Meanwhile we provide easy-to-use interfaces for modifying node's configs.
* Allocate rpc port and p2p port dynamically.

* ls test/template: ckb.toml, specs/integration.toml, specs/cells/always_success
* We use it to initialize nodes' working directory

* `Spec` would not implicitly create `Net` instance any more. Coder should explicitly create `Net` on demand.
* Simplify interfaces. Also removed several interfaces.

2286: feat: redesign the testing framework r=quake,chuijiaolianying a=keroro520

Redesign the interfaces and workflow of Spec:

  * Change interface `fn before_run()` => `fn before_run() -> Vec<Node>`
  * Change interface `fn run(&mut Net)` => `fn run(&mut Vec<Node>)`
  * Change interfaces `fn modify_ckb_config` and `modify_chain_spec`
  * Initialize context within `before_run`, which is originally at worker.rs
  * Use struct name as spec name
  * Remove `protocols` and `connect_all` from `Setup` options
  * Remove `TestProtocol`. `ckb_network::SupportProtocols` is enough.

And some refactor to `Node` and `Net`.

Co-authored-by: keroro <keroroxx520@gmail.com>

2281: feat: add tx subscription r=keroro520,zhangsoledad,driftluo,yangby-cryptape a=quake

This PR added a `new_transaction` topic to subscription rpc, user will get notified when new transaction is submitted to pool.

tcp tx subscription example:
```
telnet localhost 18114
> {"id": 2, "jsonrpc": "2.0", "method": "subscribe", "params": ["new_transaction"]}
< {"jsonrpc":"2.0","result":"0x3","id":2}
< {"jsonrpc":"2.0","method":"subscribe","params":{"result":"... pool transaction entry json string...","subscription":"0x3"}}
```

Co-authored-by: quake <quake.wang@gmail.com>

2298: ci: skip RUSTSEC-2020-0043 temporarily r=quake,zhangsoledad a=yangby-cryptape

[`ws` allows remote attacker to run the process out of memory.](https://rustsec.org/advisories/RUSTSEC-2020-0043)
Since it is no longer actively maintained, we couldn't fix it in the short term, skip it temporarily to avoid blocking other PRs.

p.s. That issue was reported before one year, but there is still no reply.

Co-authored-by: Boyu Yang <yangby@cryptape.com>