New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open source zcash blockchain analysis platform (development and hosting) #39

Open
rmhere opened this Issue May 18, 2018 · 9 comments

Comments

Projects
None yet
3 participants
@rmhere

rmhere commented May 18, 2018

Blockchain Exploration Research Group (BERG)
Wrocław University of Science and Technology, Wrocław, Poland
https://www.ii.pwr.edu.pl/~michalski/berg/

The proposal covers creating and hosting an open source zcash blockchain analysis platform for research purposes.

In early 2018 we provisioned a tool for Bitcoin (authored by M. Pieczka, a member of BERG) that is providing the researchers with the ability to query and filter the block database in order to perform further analyses, such as:

  • the analysis of the structural changes in Bitcoin blockchain
  • the discovery of clusters in Bitcoin blockchain
  • the automated classification of nodes based on multiple attributes (in progress)

This tool is open source (https://github.com/MarcinPieczka/Mongo-BTC-Blocks-Database/) and it consists from two main components: the query engine for querying the blockchain and extensible API that is now being extended with new services to perform multiple analyses in-site instead of just sending the data outside the server for further research.

The goal of the project proposal is to extend and update the tool to support zcash too to enable the research on zcash blockchain. This tool will be also open source plus it will be made available as a free service for other researchers hosted by us for one year after the completion of the project (hopefully longer).

The project team will consist of three BERG members: PI Dr. Radosław Michalski, Marcin Pieczka, and one more programmer. The project is estimated to last for about three months (5 person months).

@tromer

This comment has been minimized.

Collaborator

tromer commented May 25, 2018

@rmhere, can you provide examples of use cases for your analysis tool?

Also, can you comment on how it compares/relates to the tool developed under ZcashFoundation/GrantProposals-2017Q4#24 (see their draft report, https://cryptolux.org/images/d/d9/Zcash.pdf)?

@tromer tromer added the info-needed label May 25, 2018

@rmhere

This comment has been minimized.

rmhere commented May 28, 2018

Thanks @tromer for your comment.

Firstly, the tool is intended to provide generic information about blocks mined so far. By doing so it provides information for researchers interested in studying different aspects of the cryptocurrency (economy, security, network science etc.) - the obtained results can be of interest of multiple stakeholders – researchers per se, blockchain developers, market analysts and so on. As we are a group that is mostly interested in looking at blockchain as at a complex network, nowadays we use the platform we developed for the purposes of analyses we provided information about in the grant proposal (we can describe the details of analyses we did so far if you want us to do so, just write). However, the idea is that the extensible API should also provide the ability to run on-site analyses, depending on the researchers’ goals in order to support reproducible research as well as let the researchers to focus on their key work, not on writing the same code multiple times.

The analyses we think one could perform using the platform migrated to zcash + with extended API:
• discovery of critical phenomena in the zcash blockchain
• multivariate study of the evolution of the network over time
• automatic classification of addresses’ types in order to find out whether the anonymity of the node can be preserved
• assets’ transfer analyses (trends in the volume of transactions)

Moreover, as we wrote, we would be happy to host the platform for the scientific and developers community for a year after finishing the project + it will be open source (as the current platform for Bitcoin blockchain is). In our group, we also have a number of researchers and the platform will be also extended by the services supporting our own research on blockchain.

Comparing to the platform mentioned in ZcashFoundation/GrantProposals-2017Q4#24, some of the goals are similar, yet we believe we could bring some more network science background with our work (as we focus on network science). Also maybe there is a possibility to merge the efforts (we are open for collaboration). Apart from that, we offer hosting of the platform as mentioned in the proposal.

If you have some more questions, please feel free to ask.

@tromer

This comment has been minimized.

Collaborator

tromer commented May 30, 2018

Can you help us understand how network science applies here, and give concrete examples of interesting analysis methods and API features that exceed those provided by ZcashFoundation/GrantProposals-2017Q4#24 (as well as standard block explorers API, the Insight software, etc.)?

@rmhere

This comment has been minimized.

rmhere commented May 30, 2018

@tromer, let me then add some more details

Firstly, as Insight or standard block explorers, the Bitcoin version of the tool does the same in terms of basic functionality (but please read further :). Namely, it parses the blocks into more suitable form for queries and further analyses. However, as we progress with the development of our Bitcoin version, we see that in order to perform more sophisticated analyses, transferring blocks' and transactions' information to end-users is absolutely not the way to go, since before running the analysis a lot of data has to be transferred. This is why we believe that this sort of tools should provide their extensive API to be available to run analyses on-site and just send the results. E.g., there is no need to send some blocks to compute shortest paths or send transactions information to calculate the flow of cryptocurrency. Our intention is to build a number of API services we already use in our research and invite others to suggest (or even develop) more. The goal is to make the platform a point you query for information instead of a tool you query for blocks/transactions.

Secondly, I read the description and report of ZcashFoundation/GrantProposals-2017Q4#24. How do I see the differences between this proposal and the one above? This proposal will go in the following directions:

  • zcash analysis tool as a service (+ source code for those willing to have their own instances, free for all researchers and interested parties)
  • the possibility to cache queries' results in order to speed up the processing
  • the possibility to submit the long-lasting analysis and be informed when it is done + the ability to use previous results by other researchers
  • implementation of selected generic network science measures and metrics (centrality measures, clustering algorithms, e.g., CPM, random walk and others),
  • implementation of different temporal models to analyse the dynamics of the zcash blockchain
  • quantitative analyses
  • node classification algorithms

We would like to start with extending our API with basic measures and clustering algorithms, as we already did some research on that, then we wish to go to towards node classification algorithms.

Trying to answer your question on network science. This is a discipline derived from graph theory you could think of as an applied graph theory. It is observed that in a number of applications network science methods improve the quality of results as they look at the network structure, network dynamics, nodes' positions in graphs etc., something that is often overlooked. By doing so, network science brings new quality into disciplines like biology, chemistry, sociology etc. When thinking of using the benefits of network analysis to blockchain, I believe this could lead to an identification of the trends in network evolution, network robustness, link prediction, node identification, cluster analyses and so on. Moreover, we could answer the questions on which models is the zcash network most similar to and others. I believe you already know or looked at some articles about network science, but if not, this is the easiest way: https://en.wikipedia.org/wiki/Network_science. Of course, we could not cover all in this project proposal, so we will focus on what was listed above.

@tromer tromer added the invited-full label Jun 1, 2018

@tromer

This comment has been minimized.

Collaborator

tromer commented Jun 1, 2018

The Zcash Foundation Grant Review committee has reviewed your pre-proposal, including the above discussion, to evaluate its potential and competitiveness relative to other proposals. Every pre-proposal was evaluated by at least 3 (and typically more than 4) committee members .

The committee's opinion is that your pre-proposal is a promising candidate funding in this round, and the committee therefore invites you to submit a full proposal.
Please submit a full proposal by June 15th, following the detailed structure described in the Call for Proposals. We encourage you to submit a draft as early as possible, to allow for community feedback.

@rmhere

This comment has been minimized.

rmhere commented Jun 15, 2018

Thank you for reviewing the pre-proposal and we were really happy to be invited to the full proposals stage. Our full proposal is therefore attached below. In case of any more questions, please do not hesitate to ask.

zcash_grant_proposal_39_upd.pdf

(updated with small corrections in the text, not influencing the merits of the proposal)

@tromer

This comment has been minimized.

Collaborator

tromer commented Sep 25, 2018

@rmhere, we are about to finalize the grant funding recommendations. Are there any updates in your status or plans that we should be aware of?

@rmhere

This comment has been minimized.

rmhere commented Sep 25, 2018

@tromer, I queried both planned collaborators and nothing changed, we await the decision

@sonyamann

This comment has been minimized.

Collaborator

sonyamann commented Nov 6, 2018

I'm thrilled to inform you that the Grant Review Committee and the Zcash Foundation Board of Directors have approved your proposal, pending a final compliance review. Congratulations, and thank you for the excellent submission!

Next steps: Please email josh@z.cash.foundation from an email address that will be a suitable point of contact going forward. We plan to proceed with disbursements following a final confirmation that your grant is within the strictures of our 501(c)(3) status, and that our payment to you will comply with the relevant United States regulations.

We also wish to remind you of the requirement for monthly progress updates to the Foundation’s general mailing list, as noted in the call for proposals.

Before the end of this week, the Zcash Foundation plans to publish a blog post announcing grant winners to the public at large, including a lightly edited version of the Grant Review Committee’s comments on your project. The verbatim original text of the comments can be found below.

Congratulations again!

Grant Review Committee comments:

Proposes an on-line platform for analyzing Zcash transactions, to help researchers investigate the Zcash on-chain data effectively and and reproducibly.

Even after t-addresses are deprecated, lots of statistical information can still be extracted from the blockchain. Having this tool available will enable researchers, for example, find parameters that are estimable and comparable with other currencies.

The following reservation was raised. There have already been 3 papers showing the flaws with users interacting with T-Z addresses (e.g., https://blog.z.cash/maintaining-privacy/). It is crucial for the proposers to focus on facilitiating novel discoveries, and to strike for useful information even with the eventual deprecation t-addresses. For example, consider how network-layer information can be captured and conveyed by the tool.

We recommend funding this proposal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment