Snark worker delay (#1533)

Changes:

1.added work-delay-factor to the scan state. 
Work-delay-factor is the number of slots it takes for a considerable number of proofs to be done. Considerable because it depends on how many transactions we always want in a block. For instance, we always want a coinbase and so there has to be at least two proofs available. If it takes 2 slots to generate two proofs then work-delay-factor = 2.  
Work delay factor essentially increases the tree size (therefore, we need to minimize the delay) so that transactions can be en-queued while snark workers are still generating the required proofs. (#1525)

2. added work-capacity to the scan state 
In order to control this behaviour so as to not allow free transactions beyond a point, `Work-capacity` is introduced. Work-capacity is the maximum number of jobs there can be at any point in time given a specific transaction capacity and work-delay-factor. The`job_count` field of the scan state tracks the number of jobs currently on the tree. This value is updated using the following equation

       `job_count += job_count - amount-of-work-done-in-a-block + (no-of-txns-added-in-the-block * 2)`

We add `(no-of-txns-added-in-the-block * 2)` jobs which means that  both `job_count` and `work-capacity` account for the jobs aren't there on the tree yet for snark workers to pick up. These are the future jobs that will be added when present jobs are completed. 

Periodically `job_count` will be decremented since the amount of work required for `t` transactions is `(2*t -1)`
(#1525 )
3. perform checks when updating the scan state
 Check work-capacity every time work is added and  transactions are en-queued(#1525 )

4. Work constraints have now changed. The amount of work and number of transactions included should not cause scan-state's `job_count` to exceed work-capacity. 
Also, a bunch of changes to include coinbase splitting (#1526 )

5. Added a sequence number field to the nodes of the parallel scan tree. All the jobs new jobs created will have curr_job_seq_no. This field corresponds to the block number in which a particular Base or a Merge job was added. It was initially used to implement the delay-factor but now remains just for debugging purposes. (#1524 )

6. added a cli option `work-delay-factor` (#1527 )

docs/daemon.md

@@ -21,6 +21,7 @@ command-line flags. These flags are supported in the config file:
 - `client-port` int
 - `propose` bool
 - `txn-capacity` int
+- `work-delay-factor` int
 - `rest-port` int
 - `peers` string list. This does not get overridden by `-peer` arguments.
   Instead, `-peer` arguments are added to this list.

src/app/cli/src/coda.ml

@@ -87,7 +87,13 @@ let daemon log =
        flag "txn-capacity"
          ~doc:
            "CAPACITY_LOG_2 Log of capacity of transactions per transition \
-            (default: 4)"
+            (default: 8)"
+         (optional int)
+     and work_delay_factor =
+       flag "work-delay-factor"
+         ~doc:
+           "DELAY_LOG_2 Log of number of block-times snark workers take to \
+            produce atleast two proofs (default:2)"
          (optional int)
      and is_background =
        flag "background" no_arg ~doc:"Run process on the background"
@@ -169,6 +175,10 @@ let daemon log =
          or_from_config YJ.Util.to_int_option "txn-capacity" ~default:8
            transaction_capacity_log_2
        in
+       let work_delay_factor =
+         or_from_config YJ.Util.to_int_option "work-delay-factor" ~default:2
+           work_delay_factor
+       in
        let snark_work_fee_flag =
          Currency.Fee.of_int
            (or_from_config YJ.Util.to_int_option "snark-worker-fee" ~default:0
@@ -275,6 +285,8 @@ let daemon log =
 
          let transaction_capacity_log_2 = transaction_capacity_log_2
 
+         let work_delay_factor = work_delay_factor
+
          let commit_id = commit_id
 
          let work_selection = work_selection

src/app/cli/src/coda_main.ml

@@ -98,6 +98,8 @@ module type Config_intf = sig
   val transaction_capacity_log_2 : int
   (** Capacity of transactions per block *)
 
+  val work_delay_factor : int
+
   val commit_id : Daemon_rpcs.Types.Git_sha.t option
 
   val work_selection : Protocols.Coda_pow.Work_selection.t

src/app/cli/src/coda_worker.ml

@@ -204,6 +204,8 @@ module T = struct
 
         let transaction_capacity_log_2 = 3
 
+        let work_delay_factor = 2
+
         let commit_id = None
 
         let work_selection = work_selection

src/app/cli/src/full_test.ml

@@ -59,6 +59,8 @@ let run_test () : unit Deferred.t =
         (*this works because we don't have prover fees. Once we have that, the transaction_capacity_log_2 has to be at least 2 for transactions to be included*)
       else 2
 
+    let work_delay_factor = 2
+
     let commit_id = None
 
     let work_selection = Protocols.Coda_pow.Work_selection.Seq