Skip to content
This repository has been archived by the owner on Aug 8, 2018. It is now read-only.

Pyeth Coordination Call: 06.01.17 Meeting Minutes

Jump to bottom
S. Matthew English edited this page Jun 1, 2017 · 2 revisions

Participants:

Matthew English, ConsenSys

Shahan Khatchadourian, ConsenSys

Bob Summerwill, ConsenSys

Herman Junge, ConsenSys

Konrad

Guiherme

cdetrio

Karl Floersch

alex@brainbot

Xin Wang

jan@cryptape

Ulrich Petri

Agenda:

Would it be worth making a -dev release of pyeth*/devp2p so that people can run it against the mainnet easily?

Mailing list creation for pyethapp/pyethereum dev discussions?

Shall we make this a recurring call?

Review Updating for mainnet to see what's been done and what's left to do

Previous Notes:

https://github.com/ethereum/pyethapp/wiki/Updating-for-mainnet

Meetings:

→ Last meeting, focus: client app not sync’ing with main app.

→ Pyeth Code: Tests are failing, things are broken: focus on stabilizing codebase so we can move on to other points covered in the previous meeting. - What particularly? i.e. Network, Storage, etc.

→ Working on Hive, adding support for Ethereum, to run RPC tests and Consensus tests

→ Syncopation of changes with MainNet

→ Not many people running PyEth at the moment, because it doesn’t sync with MainNet

→ Test Update (Still Failing):

→ MainNet Sync’ing Status:

Merged all changes for MainNet sync with development branch Some problems remain in development layer Attempting to sync with Parity nodes or Geth nodes. At first the sync appears to work, but after running for a while it breaks. Memory leakage problem has been addressed in the Network layer Pynode is stable- can run for a long time and remains stable, without using too much resources or consuming too much energy Sync with MainNet task is basically complete, only with small problems to resolve.

State Revamp- much refactoring, many files changed and modified- merge is labour intensive.

State Revamp branch vs. Development branch

Code in Pyeth app has been updated. Close to new release.

StateRevamp branch includes Metropolis milestone changes.

Most important next step: make the pynode more stable.

Guiherme making pynode pass the hive tests.

→ Creation of EVM wrapper for fast testing.

Clarify with Jan if the branch that sync’s with the main net now is the development branch or the state revamp now? -- the develop branch is the one the sync’s

Vitalik is actively working on the state revamp branch

Is there a particular checkpoint we’re targeting that we can say to the EEA “this release is now suitable for generating specifications, inter alia”

  • Developer announcement. Of interest? In works?

→ Big release planned for when Vitalik fixes tests in Develop state revamp branch.

Aim for improved modularity. After refactoring, changes to the consensus algorithm will be easier.

Consensus algorithm is quite easy to swap right now-- but refactoring still needs to pass all the tests.

-- Vitalik working actively on fixing tests that don’t pass.

→ Next few weeks, track progress of these developments.

→ Vitalik is fixing the pyethereum tests, after that he will merge the state revamp branch.

→ pyeth app: still needs to work, the full test suite still does not pass.

:: people working on py ethereum core component vs people working on pyeth app ::

Development efforts underway:

Xin Wang- big code change in pyethereum. In pyeth app there are also issues- how can test coverage be improved? Is it possible to work in parallel/cover issues concurrently?

→ Run pyeth app as long as possible and try to find existing bugs (feeling is that some exist)

→ when pynode is trying to connect with other implementations like geth and parity-... we need more inventigation- because this is currently problematic.

→ We have a snapshot on Vitaliks server (so we don’t need to sync from block zero)

Pynode supports snapshot import and export.

:: information sharing on how to access snapshot to be diseminated by mail*

-- Is there a particular branch on pyeth app that is trying to keep in sync with changes that are going on in Ethereum?

Develop with state revamp branch has code on both develop branch and state revamp branch.

We’re close to merge this branch back to develop branch.

BRANCHES ACROSS REPOS: For the pyehteum github repo- is there an equivalent branch name for the pyeth app repo? Yes- same branch and same name.

Guiherme “py release package, where people can use pip to install, if you try to clone the pyeth app branch locally and install it- it uses pip to install the latest release of Ethereum and dependencies incompatible with the latest development branch”

→ Possible solution: version fixed in the config file. Problem: no compatible version- requires manual installation of dependencies before pyeth app can be installed. Frequently people are asking for help with this- likely to encourage people running this software against the Mainnet. → Guiherme happy to take responsibility for creating a release that has increased stability and easy of use. → Perhaps automoation by a docker container?

Does the working client pass all the tests? Two pull requests fix a lot of tests, remaining tests are being worked on at the moment,

  • RC pre-release, -dev release, … how to indicate that something other than the “full version” is being worked on.

→ Does it help developers start with development? Or is it just for users to run the code? What is the time horizon for creating such a release? How much of the release process is automated? How many end users will actually deploy the software?

→ After build passes a release will be created- necessitates the passing of tests.

  • Currently no support for pre-released, but can be added in the next few days.

  • Regarding pluggable consensus: proof of authority testnet: competing protocols for proof of authority (POA).

→ Header based POA (Easier for liteclients) vs…

→ How easy to swap out the casper contract for the covan POA…

→ Currently adding in the hybrid casper consensus to the pyethereum.

→ Should be easy to add in whatever consensus rules one likes from a smart contract.

→ Smart contract-based consensus algorithm: does it require a pre-compile or components ancilary to the EVM: does it expect out of band packets, etc…

  • In casper messages are added into blocks, functions as normal transaction.

  • Able to penalize or reward validators.

  • consensus algorithms that rely only on smart contract may not work correctly if an alternative is used, such as raft, etc.

Consensus mechanisms that rely only on a smart contract- how would they function in case of alternative network consensus algorithms such as raft or pbft.

→ depends on implmentation; concensus algorithm logic not contained in smart contract, it will be used most to monitor participants, for penalize or reward.

  • What messages have I received? Which validators are online?

→ In smart contract can do as much or as little as you want.

If you want to combine raft or pbft with a smartcontract there are two ways to implement that- votes or messages exchanged in some out of band channel- or can use the blockchain to exchange those messages.

Brief pause to re-calibrate zoom

Consensus question: as part of development of casper- are people looking at how consensus algorithms are implemented in geth or partity? To implement this in pyeth would it be a lot of work? Can people specify those protocols, from the standpoint of one client, could they be re-implemnted easily in pyeth?

Vitaliik conception of how to implment consensus acorss clients- mangeable

Parity original proof of authpority was not contract-based, has since pivoted to contract based.

From reddit: documentation and full switch from covan testnet- no contract on the covan tesnet keeping track of the current set of authorities. They’re planning to craete such a contract, at which time they will switch to contract based POA.

In geth POA is header based, and set of authorities is hard-coded, not stored in a contract.

If casper contract is swapped out for poa contract- what role would be assumed by the casper demon (to have it function with poa)

*The pyeth app would \need modifications

→ does the casper demon use the pyeth app? How are they related?

The casper demon- being added to the actual pyethereum (equivalent to adding it to a full implementation of casoer) will serve as a reference implementation for geth, partity, etc. who are interested in implementing casper.

Casper deamon sits on the side- deploy casper contract on main net to some rnadom address, and once the contract is deployed people can fill in the correct block information and interact with that contract.

Eventually the casper deamoin won’t be needed, once the hard fork comes.

The POA testnet may or may not need a deamon like casper deamon, as it’s used only for casper.

If the covan testnet is a simple POA, not any POA + raft or pbft algorithm, we may not need such a deamon. Can suffice to only implement a consensus module in pyeth app

→ interesting challenges of integrating different consensus algirthms- since they touch multiple components.

Mailing list creation for pyeth discussion:

→ Gitter channels tend to get many users asking for support- asking many questions- the signal to noise ratio there is unfavourable- corrupted by noise- development discussion are drowned out by support requests- proposed creation of a mailing list for development only- free from fetters of support requests.

→ alternative to a mailing list would be- to create informative github issues- for coordination/informational purposes. Yet another communication channel might further dilute the space where the conversations are happening.

→ creation of a tab on github issues.

→ preference for using github issues initially due to centralization of info. Since the eip issue process is taking place on github.

→ improvement of signal to noise ration by creating an invite only dev channel that goes along with the pyethereum gitter.

→ creating a /dev channel- as well as a user facing one- support questions are redirected to public facing channel.

→ creation of a /dev room for discussion across repos, potentially a pyeth dev channel (higher level of abstraction).

→ Discussion about pyeth app, some support needed for the p2p related module- to test the stability of the current python node.

  • Hudson- talking about- more general- in terms of metrolpolis- overall testing.

  • Dimitry doing much of the testing- lot’s to be covered for metropolis

  • Related to the Hive testing.

→ tie into testing/dashboarding… trying to extend dashboarding, etc.

→ share tools between publoc ethereum and the eea- with a mind toward - two distinct databases in the end- but other things shared.

*tests on pyeth are failing: to what extent do we focus on client specific tests rather than state based tests- higher level tests- not clear what the individual py unit tests are looking for- are they redundent for things that are being evaluated by the multi-client tests?

  • Unit tests:: each client has it’s own set of unit tests, and unit test specific hooks…

  • cpp ethereum has it’s own unit tests, geth has it’s own unit tests, each client would have it’s own hooks… to correlate with things that are tested within hive.

  • usually the unit tests are useful when developing- since they run faster and can demonstrate what exactly is broken on the low level implementation that you are currently working on- more targeted. No need to drill down from the rpc to see what is actually broken: a valid reason to maintain the individual unit tests- or even to add more- since in the case of pyeth there are actually very few.

Regarding the py-ethereum the solidity module:

  • problem with the solidity features that are not supported- plans to isolate solidity development- no known plans to do so.

  • whether or not to invest more work in the current sol-c, if no bigger changes are planned might make sense to make small pull requests and bug fixes.

  • pyetherum parses a lot of render usable results, but is not up to date with latest language features of solidity, cause of known bugs- new way to interact with py-ethereum using solidity- could require substanitive refactoring.

  • Solidity itself, the JSON io is a relatively new feature to standardize how you pass in contracts and how you receive the compiled contracts- but has resulted in some backwards compatibility issues, particularly with the python client- at the moment no one is working on fixing this- but it could be reconciled…

Clone this wiki locally