Add update docs script to CI #9219

cpurta · 2018-07-25T21:01:37Z

Added a script to CI that will use the jsonrpc tool to update rpc
documentation then commit and push those to the wiki repo.

Resolves #8835

Added a script to CI that will use the jsonrpc tool to update rpc documentation then commit and push those to the wiki repo.

parity-cla-bot · 2018-07-25T21:01:40Z

It looks like @cpurta signed our Contributor License Agreement. 👍

Many thanks,

Parity Technologies CLA Bot

5chdn · 2018-07-26T20:34:36Z

.gitlab-ci.yml

+  stage: docs
+  only:
+    - tags
+    - master


might be sufficient to have this on tags only.

I agree. Having that for every commit to master would get annoying pretty quick.

5chdn · 2018-07-26T20:37:02Z

scripts/gitlab-rpc-docs.sh

+
+setup_git() {
+    git config user.email "runner@parity.com"
+    git config user.name "Parity Runner"


Does this user exist? Do we need to create some user that has access?

I would doubt that user exists. I just put that there as a placeholder for now. You will need to create a user in gitlab for the runner to use. Can then update those lines with the correct email and username.

@General-Beck please suggest what we should use here :)

MB devops@parity.io ?

5chdn · 2018-07-30T09:32:14Z

Scripts LGTM.

I created a test pipeline, let's see how it works out. https://gitlab.parity.io/parity/parity/pipelines/21869

5chdn · 2018-07-31T07:56:15Z

Failure unrelated. Should be fixed now: https://gitlab.parity.io/parity/parity/pipelines/21887

cpurta · 2018-07-31T15:07:06Z

Looks like that job failed since the repo is called parity and not parity-ethereum when trying to copy. Will update the script.

Copy correct parity repo to jsonrpc folder before attempting to build docs since the CI runner clones the repo as parity and not parity-ethereum.

General-Beck

Yes, most likely so it is more correct. Because we did not rename the repo on the gitlab.

debris · 2018-08-01T07:59:58Z

@5chdn please merge if it looks good to you :)

cpurta · 2018-08-08T23:40:25Z

@5chdn anything else needed for this PR?

5chdn · 2018-08-20T20:46:27Z

Sorry, I was sick, I'll restart the CI once again.

https://gitlab.parity.io/parity/parity/pipelines/22544

5chdn · 2018-08-21T19:06:48Z

docs failed: https://gitlab.parity.io/parity/parity/-/jobs/95442

fs.js:904
  return binding.readdir(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, scandir '/home/gitlab-runner/builds/4e7a1ea4/0/parity/parity/jsonrpc/.parity/rpc/src/v1/traits'
    at Object.fs.readdirSync (fs.js:904:18)
    at getMethodsFromRustTraits (/home/gitlab-runner/builds/4e7a1ea4/0/parity/parity/jsonrpc/scripts/helpers/parsed-rpc-traits.js:63:6)
    at Object.<anonymous> (/home/gitlab-runner/builds/4e7a1ea4/0/parity/parity/jsonrpc/scripts/helpers/parsed-rpc-traits.js:70:1)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @parity/jsonrpc@2.1.6 build:markdown: `node scripts/build-rpc-markdown.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @parity/jsonrpc@2.1.6 build:markdown script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/gitlab-runner/.npm/_logs/2018-08-21T19_04_30_599Z-debug.log
ls: cannot access 'jsonrpc/docs': No such file or directory
Switched to a new branch 'rpcdoc-update-v2.1.0-rc2'
On branch rpcdoc-update-v2.1.0-rc2
nothing to commit, working directory clean
On branch rpcdoc-update-v2.1.0-rc2
nothing to commit, working directory clean
error: Terminal is dumb, but EDITOR unset
Please supply the message using either -m or -F option.
fatal: could not read Username for 'https://github.com': No such device or address
ERROR: Job failed: exit status 1

cpurta · 2018-08-21T20:11:30Z

It appears to be due to not correctly copying the parity folder into the jsonrpc repo that is cloned. Working on a fix now.

Update remote config in wiki repo before pushing changes using a github token for authentication. Add message to wiki tag when pushing changes. Use project directory to correctly copy parity code base into the jsonrpc repo for doc generation.

cpurta · 2018-08-21T21:18:56Z

Fixed that error and a few others that popped up when running the job with a local gitlab CI runner. One thing that I forgot about was that this change will need a github api token environment variable needed in Gitlab CI settings (referenced as GITHUB_TOKEN in the script). This allows for the runner to be able to push code using the script. The script will fail until there is a valid token present on the runner.

5chdn

This LGTM, @General-Beck could you finalize the PR with a GITHUB_TOKEN?

5chdn · 2018-08-24T16:13:54Z

Ok, I'll merge this now as-is and we will fix the final bit together with #8765

Thanks

* Add update docs script to CI Added a script to CI that will use the jsonrpc tool to update rpc documentation then commit and push those to the wiki repo. * fix gitlab ci lint * Only apply jsonrpc docs update on tags * Update gitlab-rpc-docs.sh * Copy correct parity repo to jsonrpc folder Copy correct parity repo to jsonrpc folder before attempting to build docs since the CI runner clones the repo as parity and not parity-ethereum. * Fix JSONRPC docs CI job Update remote config in wiki repo before pushing changes using a github token for authentication. Add message to wiki tag when pushing changes. Use project directory to correctly copy parity code base into the jsonrpc repo for doc generation. * Fix set_remote_wiki function call in CI

* master: evmbin: Fix gas_used issue in state root mismatch and handle output better (#9418) Update hardcoded sync (#9421) Add block reward contract config to ethash and allow off-chain contracts (#9312) Private packets verification and queue refactoring (#8715) Update tobalaba.json (#9419) docs: add parity ethereum logo to readme (#9415) build: update rocksdb crate (#9414) Updating the CI system (#8765) Better support for eth_getLogs in light mode (#9186) Add update docs script to CI (#9219) `gasleft` extern implemented for WASM runtime (kip-6) (#9357) block view! removal in progress (#9397) Prevent sync restart if import queue full (#9381) nonroot CentOS Docker image (#9280) ethcore: kovan: delay activation of strict score validation (#9406)

* parity-version: bump beta to 2.0.2 * remove ssl from dockerfiles, closes #8880 (#9195) * snap: remove ssl dependencies from snapcraft definition (#9222) * parity-version: bump beta to 2.0.3 * Remove all dapp permissions related settings (#9120) * Completely remove all dapps struct from rpc * Remove unused pub use * Remove dapp policy/permission func in ethcore * Remove all dapps settings from rpc * Fix rpc tests * Use both origin and user_agent * Address grumbles * Address grumbles * Fix tests * Check if synced when using eth_getWork (#9193) (#9210) * Check if synced when using eth_getWork (#9193) * Don't use fn syncing * Fix identation * Fix typo * Don't check for warping * rpc: avoid calling queue_info twice on eth_getWork * Fix potential as_usize overflow when casting from U256 in miner (#9221) * Allow old blocks from peers with lower difficulty (#9226) Previously we only allow downloading of old blocks if the peer difficulty was greater than our syncing difficulty. This change allows downloading of blocks from peers where the difficulty is greater then the last downloaded old block. * Update Dockerfile (#9242) * Update Dockerfile fix Docker build * fix dockerfile paths: parity -> parity-ethereum (#9248) * Propagate transactions for next 4 blocks. (#9265) Closes #9255 This PR also removes the limit of max 64 transactions per packet, currently we only attempt to prevent the packet size to go over 8MB. This will only be the case for super-large transactions or high-block-gas-limit chains. Patching this is important only for chains that have blocks that can fit more than 4k transactions (over 86M block gas limit) For mainnet, we should actually see a tiny bit faster propagation since instead of computing 4k pending set, we only need `4 * 8M / 21k = 1523` transactions. Running some tests on `dekompile` node right now, to check how it performs in the wild. * Update tobalaba.json (#9313) * Fix load share (#9321) * fix(light_sync): calculate `load_share` properly * refactor(api.rs): extract `light_params` fn, add test * style(api.rs): add trailing commas * ethcore: fix pow difficulty validation (#9328) * ethcore: fix pow difficulty validation * ethcore: validate difficulty is not zero * ethcore: add issue link to regression test * ethcore: fix tests * ethcore: move difficulty_to_boundary to ethash crate * ethcore: reuse difficulty_to_boundary and boundary_to_difficulty * ethcore: fix grumbles in difficulty_to_boundary_aux * Light client `Provide default nonce in transactions when it´s missing` (#9370) * Provide `default_nonce` in tx`s when it´s missing When `nonce` is missing in a `EthTransaction` will cause it to fall in these cases provide `default_nonce` value instead! * Changed http:// to https:// on Yasm link (#9369) Changed http:// to https:// on Yasm link in README.md * Provide `default_nonce` in tx`s when it´s missing When `nonce` is missing in a `EthTransaction` will cause it to fall in these cases provide `default_nonce` value instead! * Address grumbles * ethcore: kovan: delay activation of strict score validation (#9406) * Better support for eth_getLogs in light mode (#9186) * Light client on-demand request for headers range. * Cache headers in HeaderWithAncestors response. Also fulfills request locally if all headers are in cache. * LightFetch::logs fetches missing headers on demand. * LightFetch::logs limit the number of headers requested at a time. * LightFetch::logs refactor header fetching logic. * Enforce limit on header range length in light client logs request. * Fix light request tests after struct change. * Respond to review comments. * Add update docs script to CI (#9219) * Add update docs script to CI Added a script to CI that will use the jsonrpc tool to update rpc documentation then commit and push those to the wiki repo. * fix gitlab ci lint * Only apply jsonrpc docs update on tags * Update gitlab-rpc-docs.sh * Copy correct parity repo to jsonrpc folder Copy correct parity repo to jsonrpc folder before attempting to build docs since the CI runner clones the repo as parity and not parity-ethereum. * Fix JSONRPC docs CI job Update remote config in wiki repo before pushing changes using a github token for authentication. Add message to wiki tag when pushing changes. Use project directory to correctly copy parity code base into the jsonrpc repo for doc generation. * Fix set_remote_wiki function call in CI * Prevent blockchain & miner racing when accessing pending block. (#9310) * Prevent blockchain & miner racing when accessing pending block. * Fix unavailability of pending block during reseal. * Prevent sync restart if import queue full (#9381) * Add POA Networks: Core and Sokol (#9413) * ethcore: add poa network and sokol chainspecs * rpc: simplify chain spec docs * cli: rearrange networks by main/test and size/range * parity: don't blacklist 0x00a328 on sokol testnet * parity: add sokol and poanet to params and clean up a bit, add tests * ethcore: add the poa networks and clean up a bit * ethcore: fix path to poacore chain spec * parity: rename poa networks to poacore and poasokol * parity: fix configuration tests * parity: fix parameter tests * ethcore: rename POA Core and POA Sokol * Update tobalaba.json (#9419) * Update hardcoded sync (#9421) - Update foundation hardcoded header to block 6219777 - Update ropsten hardcoded header to block 3917825 - Update kovan hardcoded header to block 8511489

Add update docs script to CI

8f9b64f

Added a script to CI that will use the jsonrpc tool to update rpc documentation then commit and push those to the wiki repo.

fix gitlab ci lint

a162087

5chdn requested a review from General-Beck July 26, 2018 20:27

5chdn added A0-pleasereview 🤓 Pull request needs code review. M1-ci 🙉 Continuous integration. M3-docs 📑 Documentation. labels Jul 26, 2018

5chdn modified the milestones: Patch, 2.1 Jul 26, 2018

5chdn added the B0-patchthis label Jul 26, 2018

5chdn reviewed Jul 26, 2018

View reviewed changes

5chdn mentioned this pull request Jul 26, 2018

beta backports for 2.0.3 #9229

Merged

28 tasks

cpurta and others added 2 commits July 26, 2018 13:42

Only apply jsonrpc docs update on tags

b970425

Update gitlab-rpc-docs.sh

10c4e0c

Copy correct parity repo to jsonrpc folder

4c5952d

Copy correct parity repo to jsonrpc folder before attempting to build docs since the CI runner clones the repo as parity and not parity-ethereum.

General-Beck approved these changes Jul 31, 2018

View reviewed changes

gitcoinbot mentioned this pull request Aug 1, 2018

Auto-Generate JSON-RPC API docs #8835

Closed

5chdn added A4-gotissues 💥 Pull request is reviewed and has significant issues which must be addressed. and removed A0-pleasereview 🤓 Pull request needs code review. labels Aug 21, 2018

5chdn mentioned this pull request Aug 21, 2018

Updating the CI system #8765

Merged

Fix JSONRPC docs CI job

a3be408

Update remote config in wiki repo before pushing changes using a github token for authentication. Add message to wiki tag when pushing changes. Use project directory to correctly copy parity code base into the jsonrpc repo for doc generation.

Fix set_remote_wiki function call in CI

596e5d0

5chdn added A8-looksgood 🦄 Pull request is reviewed well. and removed A4-gotissues 💥 Pull request is reviewed and has significant issues which must be addressed. labels Aug 23, 2018

5chdn approved these changes Aug 23, 2018

View reviewed changes

5chdn merged commit 7abe9ec into openethereum:master Aug 24, 2018

ordian mentioned this pull request Sep 10, 2018

ci: fix rpc docs generation #9515

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add update docs script to CI #9219

Add update docs script to CI #9219

cpurta commented Jul 25, 2018 •

edited

parity-cla-bot commented Jul 25, 2018

5chdn Jul 26, 2018

cpurta Jul 26, 2018

5chdn Jul 26, 2018

cpurta Jul 26, 2018

5chdn Jul 26, 2018

General-Beck Jul 27, 2018

5chdn commented Jul 30, 2018

5chdn commented Jul 31, 2018

cpurta commented Jul 31, 2018

General-Beck left a comment

debris commented Aug 1, 2018

cpurta commented Aug 8, 2018

5chdn commented Aug 20, 2018

5chdn commented Aug 21, 2018

cpurta commented Aug 21, 2018

cpurta commented Aug 21, 2018

5chdn left a comment

5chdn commented Aug 24, 2018

Add update docs script to CI #9219

Add update docs script to CI #9219

Conversation

cpurta commented Jul 25, 2018 • edited

parity-cla-bot commented Jul 25, 2018

5chdn Jul 26, 2018

Choose a reason for hiding this comment

cpurta Jul 26, 2018

Choose a reason for hiding this comment

5chdn Jul 26, 2018

Choose a reason for hiding this comment

cpurta Jul 26, 2018

Choose a reason for hiding this comment

5chdn Jul 26, 2018

Choose a reason for hiding this comment

General-Beck Jul 27, 2018

Choose a reason for hiding this comment

5chdn commented Jul 30, 2018

5chdn commented Jul 31, 2018

cpurta commented Jul 31, 2018

General-Beck left a comment

Choose a reason for hiding this comment

debris commented Aug 1, 2018

cpurta commented Aug 8, 2018

5chdn commented Aug 20, 2018

5chdn commented Aug 21, 2018

cpurta commented Aug 21, 2018

cpurta commented Aug 21, 2018

5chdn left a comment

Choose a reason for hiding this comment

5chdn commented Aug 24, 2018

cpurta commented Jul 25, 2018 •

edited