New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snark worker delay #1533

Merged
merged 11 commits into from Jan 30, 2019

Conversation

Projects
None yet
2 participants
@deepthiskumar
Copy link
Contributor

deepthiskumar commented Jan 28, 2019

Changes:

1.added work-delay-factor to the scan state.
Work-delay-factor is the number of slots it takes for a considerable number of proofs to be done. Considerable because it depends on how many transactions we always want in a block. For instance, we always want a coinbase and so there has to be at least two proofs available. If it takes 2 slots to generate two proofs then work-delay-factor = 2.
Work delay factor essentially increases the tree size (therefore, we need to minimize the delay) so that transactions can be en-queued while snark workers are still generating the required proofs. (#1525)

  1. added work-capacity to the scan state
    In order to control this behaviour so as to not allow free transactions beyond a point, Work-capacity is introduced. Work-capacity is the maximum number of jobs there can be at any point in time given a specific transaction capacity and work-delay-factor. Thejob_count field of the scan state tracks the number of jobs currently on the tree. This value is updated using the following equation

    `job_count += job_count - amount-of-work-done-in-a-block + (no-of-txns-added-in-the-block * 2)`
    

We add (no-of-txns-added-in-the-block * 2) jobs which means that both job_count and work-capacity account for the jobs aren't there on the tree yet for snark workers to pick up. These are the future jobs that will be added when present jobs are completed.

Periodically job_count will be decremented since the amount of work required for t transactions is (2*t -1)
(#1525 )
3. perform checks when updating the scan state
Check work-capacity every time work is added and transactions are en-queued(#1525 )

  1. Work constraints have now changed. The amount of work and number of transactions included should not cause scan-state's job_count to exceed work-capacity.
    Also, a bunch of changes to include coinbase splitting (#1526 )

  2. Added a sequence number field to the nodes of the parallel scan tree. All the jobs new jobs created will have curr_job_seq_no. This field corresponds to the block number in which a particular Base or a Merge job was added. It was initially used to implement the delay-factor but now remains just for debugging purposes. (#1524 )

  3. added a cli option work-delay-factor (#1527 )

  • Tests were added for the new behaviour

deepthiskumar added some commits Jan 28, 2019

1.added work delay factor to the scan state. 2. added work capacity t…
…o the scan state 3. perform checks when updating the scan state (#1525)

Changes:

1.added work-delay-factor to the scan state. 
Work-delay-factor is the number of slots it takes for a considerable number of proofs to be done. Considerable because it depends on how many transactions we always want in a block. For instance, we always want a coinbase and so there has to be at least two proofs available. If it takes 2 slots to generate two proofs then work-delay-factor = 2.  
Work delay factor essentially increases the tree size (therefore, we need to minimize the delay) so that transactions can be en-queued while snark workers are still generating the required proofs.

2. added work-capacity to the scan state 
In order to control this behaviour so as to not allow free transactions beyond a point, `Work-capacity` is introduced. Work-capacity is the maximum number of jobs there can be at any point in time given a specific transaction capacity and work-delay-factor. The`job_count` field of the scan state tracks the number of jobs currently on the tree. This value is updated using the following equation

       `job_count += job_count - amount-of-work-done-in-a-block + (no-of-txns-added-in-the-block * 2)`

We add `(no-of-txns-added-in-the-block * 2)` jobs which means that  both `job_count` and `work-capacity` account for the jobs aren't there on the tree yet for snark workers to pick up. These are the future jobs that will be added when present jobs are completed. 

Periodically `job_count` will be decremented since the amount of work required for `t` transactions is `(2*t -1)`

3. perform checks when updating the scan state
 Check work-capacity every time work is added and  transactions are en-queued


Unit tests are in `Staged_ledger`
Snark work delay 3: diffs that obey new rules (#1526)
* 1.added work capacity and diffs are created according to the work capacity invariant

* merged changes from master

@deepthiskumar deepthiskumar requested a review from bkase Jan 28, 2019

deepthiskumar added some commits Jan 29, 2019

cli
cli
@bkase

bkase approved these changes Jan 29, 2019

@deepthiskumar deepthiskumar merged commit 992cc17 into master Jan 30, 2019

13 checks passed

Codacy/PR Quality Review Up to standards. A positive pull request.
Details
ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: build-macos Your tests passed on CircleCI!
Details
ci/circleci: build_public Your tests passed on CircleCI!
Details
ci/circleci: build_withsnark_sig Your tests passed on CircleCI!
Details
ci/circleci: build_withsnark_stake Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
ci/circleci: test-all_sig_integration_tests Your tests passed on CircleCI!
Details
ci/circleci: test-all_stake_integration_tests Your tests passed on CircleCI!
Details
ci/circleci: test-fake_hash_full_test Your tests passed on CircleCI!
Details
ci/circleci: test-unit-test Your tests passed on CircleCI!
Details
ci/circleci: test-withsnark Your tests passed on CircleCI!
Details
ci/circleci: tracetool Your tests passed on CircleCI!
Details

@deepthiskumar deepthiskumar deleted the fix/work-delay-master branch Jan 30, 2019

@deepthiskumar deepthiskumar referenced this pull request Jan 30, 2019

Merged

apply-diff checks (fixes #1448) #1549

1 of 3 tasks complete

deepthiskumar added a commit that referenced this pull request Jan 31, 2019

apply-diff checks (fixes #1448) (#1549)
1. Added Check for zero fee excess in the diff
2. Work-capacity check was added in #1533
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment