Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txscript: Zero alloc optimization refactor. #1656

Merged
merged 122 commits into from Mar 26, 2019

Conversation

Projects
None yet
9 participants
@davecgh
Copy link
Member

davecgh commented Mar 13, 2019

Live profiling data pulled from an instance of dcrd v1.4.0 performing an initial sync shows that roughly 50% of all allocations, which equates to around a massive 295.12GB, are the result of parsing scripts into slices of parsed opcodes prior to performing any analysis on them and executing them. Further, almost 72% of the entire CPU time is spent in garbage collection which is a direct result of producing a ton of allocations.

CPU Profiling Data:

      flat  flat%   sum%        cum   cum%
 51.77mins 24.04% 24.04% 123.53mins 57.35%  runtime.scanobject
 38.17mins 17.72% 41.76%  38.20mins 17.74%  runtime.cgocall
 36.43mins 16.91% 58.67%  39.73mins 18.44%  runtime.findObject
 13.72mins  6.37% 65.04%  13.72mins  6.37%  runtime.markBits.isMarked
 10.03mins  4.66% 69.69%  27.96mins 12.98%  runtime.greyobject
  4.38mins  2.03% 71.73%   4.38mins  2.03%  cmpbody

Memory Profiling Data:

      flat  flat%   sum%        cum   cum%
  295.12GB 50.24% 50.24%   295.94GB 50.38%  github.com/decred/dcrd/txscript.parseScriptTemplate

Consequently, this completely refactors the txscript module to remove its current reliance on parsed opcodes by introducing a new zero-allocation script tokenizer, updating all script analysis functions to make use of a combination of the new tokenizer and raw (unparsed) script analysis, and finally completely reworking the script engine to work with raw scripts by also making use of a a combination of the new tokenizer and raw (unparsed) script analysis.

It is important to note that since this is consensus critical code, and, in many respects, one of the most error prone areas of consensus, significant effort has been put into making these changes easier to reason about and review by carefully crafting a series of individual commits such that every commit message thoroughly describes its purpose, maintains consensus, and therefore passes all tests.

As can be seen in the following graphs which show the overall effect of this entire set of changes on initial sync, the total number of allocations is significantly reduced, while the overall in-use memory remains around same, as expected, which results in a significantly faster sync time (the effect is more pronounced without the profiling active that was used to generate these graphs as well):

dcrd_initial_sync_total_bytes_used_comparison

dcrd_initial_sync_mem_usage_comparison

The following is a before and after comparison of the allocations of the vast majority of the optimized functions:

benchmark                                   old allocs   new allocs   delta
------------------------------------------------------------------------------
BenchmarkScriptParsing                      1            0            -100.00%
BenchmarkIsPayToScriptHash                  1            0            -100.00%
BenchmarkIsMultisigScriptLarge              1            0            -100.00%
BenchmarkIsMultisigScript                   1            0            -100.00%
BenchmarkIsMultisigSigScriptLarge           9            0            -100.00%
BenchmarkIsMultisigSigScript                3            0            -100.00%
BenchmarkGetSigOpCount                      1            0            -100.00%
BenchmarkIsAnyKindOfScriptHash              1            0            -100.00%
BenchmarkIsPayToScriptHash                  1            0            -100.00%
BenchmarkGetPreciseSigOpCount               3            0            -100.00%
BenchmarkIsPubKeyScript                     1            0            -100.00%
BenchmarkIsAltPubKeyScript                  1            0            -100.00%
BenchmarkIsPubKeyHashScript                 1            0            -100.00%
BenchmarkIsAltPubKeyHashScript              1            0            -100.00%
BenchmarkIsNullDataScript                   1            0            -100.00%
BenchmarkIsStakeSubmissionScript            1            0            -100.00%
BenchmarkIsStakeGenerationScript            1            0            -100.00%
BenchmarkIsStakeRevocationScript            1            0            -100.00%
BenchmarkIsStakeChangeScript                1            0            -100.00%
BenchmarkGetScriptClass                     1            0            -100.00%
BenchmarkContainsStakeOpCodes               1            0            -100.00%
BenchmarkExactCoinbaseNullData              1            0            -100.00%
BenchmarkCalcMultiSigStats                  1            0            -100.00%
BenchmarkMultisigRedeemScript               1            0            -100.00%
BenchmarkPushedData                         5            4            -20.00%
BenchmarkIsUnspendable                      1            0            -100.00%
BenchmarkExtractAtomicSwapDataPushes        2            1            -50.00%
BenchmarkExtractAtomicSwapDataPushesLarge   1            0            -100.00%
BenchmarkExtractPkScriptAddrsLarge          1            0            -100.00%
BenchmarkExtractPkScriptAddrs               5            2            -60.00%
BenchmarkExtractAltSigType                  1            0            -100.00%
BenchmarkCalcSigHash                        1691         1068         -36.84%

Finally, the following is a before and after comparison of the performance of the vast majority of the optimized functions:

benchmark                                   old ns/op    new ns/op    delta
------------------------------------------------------------------------------
BenchmarkScriptParsing                      153099       961          -99.37%
BenchmarkDisasmString                       288729       94157        -67.39%
BenchmarkIsPayToScriptHash                  139961       0.66         -100.00%
BenchmarkIsMultisigScriptLarge              121599       8.63         -99.99%
BenchmarkIsMultisigScript                   797          72.8         -90.87%
BenchmarkIsMultisigSigScriptLarge           158149       4            -100.00%
BenchmarkIsMultisigSigScript                3445         202          -94.14%
BenchmarkGetSigOpCount                      163896       1048         -99.36%
BenchmarkIsAnyKindOfScriptHash              101249       3.83         -100.00%
BenchmarkIsPayToScriptHash                  139961       0.66         -100.00%
BenchmarkGetPreciseSigOpCount               287939       1077         -99.63%
BenchmarkIsPubKeyScript                     124749       4.01         -100.00%
BenchmarkIsAltPubKeyScript                  143449       2.99         -100.00%
BenchmarkIsPubKeyHashScript                 165903       0.64         -100.00%
BenchmarkIsAltPubKeyHashScript              107100       2.63         -100.00%
BenchmarkIsNullDataScript                   120800       3.81         -100.00%
BenchmarkIsStakeSubmissionScript            140308       4.20         -100.00%
BenchmarkIsStakeGenerationScript            121043       4.26         -100.00%
BenchmarkIsStakeRevocationScript            117699       4.58         -100.00%
BenchmarkIsStakeChangeScript                133810       4.39         -100.00%
BenchmarkGetScriptClass                     145001       62.9         -99.96%
BenchmarkContainsStakeOpCodes               134599       968          -99.28%
BenchmarkExactCoinbaseNullData              227          31.0         -86.34%
BenchmarkCalcMultiSigStats                  972          79.5         -91.82%
BenchmarkMultisigRedeemScript               153623       1830         -98.81%
BenchmarkPushedData                         132400       1619         -98.78%
BenchmarkIsUnspendable                      149899       860          -99.43%
BenchmarkExtractAtomicSwapDataPushes        1330         410          -69.17%
BenchmarkExtractAtomicSwapDataPushesLarge   136819       69.3         -99.95%
BenchmarkExtractPkScriptAddrsLarge          132400       44.4         -99.97%
BenchmarkExtractPkScriptAddrs               1265         231          -81.74%
BenchmarkExtractAltSigType                  497          12.8         -97.42%
BenchmarkCalcSigHash                        2792057      2760042      -1.15%

@davecgh davecgh added this to the 1.5.0 milestone Mar 13, 2019

@dajohi

dajohi approved these changes Mar 14, 2019

Copy link
Member

dajohi left a comment

24+ hours says testnet3 miner approves

@davecgh davecgh force-pushed the decred:master branch from 2713aab to 4e7f080 Mar 19, 2019

@matheusd
Copy link
Member

matheusd left a comment

Commit message has a typo on line This introduces a new funciton.

Edit:

This commit: d7ec61e

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Mar 19, 2019

Thanks. I fixed the commit message locally, but I'll wait to push it until the end to avoid changing all the commit hashes during the review period.

@matheusd
Copy link
Member

matheusd left a comment

typo in commit message a83a43f

multiple input transaciton

Ran benchmarks before & after and verified numbers.

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Mar 19, 2019

Fixed commit message for txscript: Optimize CalcSignatureHash. locally. As previously mentioned, I'll wait for rebase until the end.

Show resolved Hide resolved txscript/script.go Outdated
Show resolved Hide resolved txscript/bench_test.go Outdated
Show resolved Hide resolved txscript/standard.go Outdated
@matheusd
Copy link
Member

matheusd left a comment

message for fae1186 refers to adding checkScriptParses, but that was added in an earlier commit.

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Mar 19, 2019

Fixed commit message for txscript: Make typeOfScript accept raw script. locally.

Nice catch. I reordered commits for clarity when splitting everything up for easier review and must have missed updating that.

@matheusd
Copy link
Member

matheusd left a comment

Reviewed up to commit 0543f6f so far.

This is called out explicitly in the commits but just to point out for those only looking at the PR, this changes the semantics for the exported function GetScriptClass for the following script types:

func TestMyZZZZ(t *testing.T) {
	script01 := mustParseShortForm("0 0 OP_CHECKMULTISIG")
	cls01 := GetScriptClass(0, script01)
	t.Logf("class of first script is: %v", cls01)

	script02 := mustParseShortForm("'b'{100} 1 OP_CHECKSIGALT")
	cls02 := GetScriptClass(0, script02)
	t.Logf("class of second script is: %v", cls02)
}

Executing this test (with go test -v) on current master shows that the first script is of class "nonstandard" and the second is "pubkeyalt". Executing the same test in commit 0543f6f outputs classes of "multisig" and "nonstandard".

As far as I can tell, these particular changes don't provoke a hard fork since GetScriptClass is only used in consensus code in instances where its result is compared via equality testing to a few pre-defined script classes which don't include the affected ones.

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Mar 19, 2019

That is correct regarding why it doesn't cause a hard fork. It is also worth noting that the same applies to the ExtractPkScriptAddrs function which has been changed in the same way (also called out in the commits).

I also have issues #1625 and #1626 open which are out that the consensus code needs to be updated to remove the dependence on these functions because they are not supposed to be used in consensus.

In the mean time, I was extremely careful to examine all call sites and ensure that any changed semantics do not affect consensus. I'm glad to see you independently verify.

Show resolved Hide resolved txscript/standard_test.go Outdated
@matheusd
Copy link
Member

matheusd left a comment

Typo on commit message for 6188092

"... each the dection of each script ..."

Show resolved Hide resolved txscript/script_test.go Outdated
Show resolved Hide resolved txscript/engine.go Outdated
@matheusd
Copy link
Member

matheusd left a comment

On commit message for 1be0973

"...by making using of..."
"... which describing..."

@davecgh

This comment has been minimized.

Copy link
Member Author

davecgh commented Mar 20, 2019

Fixed commit messages for txscript: Optimize ExtractPkScriptAddrs scripthash. and txscript: Refactor engine to use raw scripts. locally.

Show resolved Hide resolved txscript/standard_test.go Outdated
@matheusd

This comment has been minimized.

Copy link
Member

matheusd commented Mar 20, 2019

🏁 🏁 🏁

Got to the end of it, reviewing each individual commit. This took somewhat of a long time, but was at least possible (compared to reviewing a single big diff) given that each individual commit made sense on its own.

I'm still going to do a couple of functional tests, which I haven't done yet but otherwise looks pretty much good to go based on other people already running this, so if you want to go ahead and rebase with the fixed commit messages, feel free from my end.

Might be a bit understated, but nice work! 😄

@davecgh davecgh force-pushed the davecgh:txscript_zero_alloc_optimization_refactor branch from 44f3fd3 to 4b45582 Mar 20, 2019

@matheusd
Copy link
Member

matheusd left a comment

Performed a full sync, all good.

@vctt94
Copy link
Member

vctt94 left a comment

Made a full sync and an atomic swap against bitcoin core and everything worked just nice.

@raedah

This comment has been minimized.

Copy link
Member

raedah commented Mar 21, 2019

-mainnet-

2019-03-21 02:17:28.484 [INF] DCRD: Version 1.4.0+release (Go version go1.11.5 linux/amd64)
2019-03-21 03:08:59.358 [INF] BMGR: ... height 328519, 2019-03-21 03:08:39 +0000 UTC)           
51m31s

2019-03-21 00:10:43.809 [INF] DCRD: Version 1.5.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 00:57:30.858 [INF] BMGR: ... height 328501, 2019-03-21 00:56:55 +0000 UTC)
46m47s

9.2% faster


-testnet-

2019-03-21 03:50:49.917 [INF] DCRD: Version 1.4.0+release (Go version go1.11.5 linux/amd64)
2019-03-21 04:23:17.875 [INF] BMGR: ... height 155444, 2019-03-21 04:23:16 +0000 UTC)
32m28s

2019-03-21 04:36:26.947 [INF] DCRD: Version 1.5.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 05:03:30.154 [INF] BMGR: ... height 155430, 2019-03-21 03:20:00 +0000 UTC)
27m4s

16.6% faster

edit:
did another run syncing to a running dcrd instance on the same host, was able to see a 28% speed improvement that way.

@degeri

This comment has been minimized.

Copy link

degeri commented Mar 21, 2019


System Configs:
SSD:Yes
Output of "dd if=/dev/zero of=/tmp/basezap.img bs=512 count=1024 oflag=dsync"

1024+0 records in
1024+0 records out
524288 bytes (524 kB, 512 KiB) copied, 0.261686 s, 2.0 MB/s

Memory: 3 GB
CPU Cores:3 Cores

model name : Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz
model name : Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz
model name : Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz


1.5.0-pre+dev DCRD:

testnet:

2019-03-21 04:29:46.987 [INF] DCRD: Version 1.5.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 05:13:14.725 [INF] BMGR: Processed 344 blocks in the last 10s (1216 transactions, 1852 tickets, 1584 votes, 134 revocations, height 155270, 2019-03-20 11:23:11 -0400 EDT)
2019-03-21 05:14:40.285 [INF] BMGR: Processed 226 blocks in the last 1m25.56s (819 transactions, 1247 tickets, 1028 votes, 106 revocations, height 155507, 2019-03-21 05:14:39 -0400 EDT)

44 minutes and 54 seconds

mainnet:

2019-03-21 10:42:05.274 [INF] DCRD: Version 1.4.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 07:54:41.814 [INF] BMGR: Processed 276 blocks in the last 10.1s (1869 transactions, 1535 tickets, 1371 votes, 15 revocations, height 328499, 2019-03-20 20:41:33 -0400 EDT)
2019-03-21 08:02:27.539 [INF] BMGR: Processed 128 blocks in the last 7m45.72s (793 transactions, 581 tickets, 629 votes, 14 revocations, height 328636, 2019-03-21 08:02:23 -0400 EDT)

1 hour, 21 minutes and 24 seconds


1.4.0-pre+dev DCRD:

testnet:

2019-03-21 09:04:43.898 [INF] DCRD: Version 1.4.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 10:09:18.167 [INF] BMGR: Processed 268 blocks in the last 10s (982 transactions, 1476 tickets, 1211 votes, 128 revocations, height 155546, 2019-03-21 07:44:02 -0400 EDT)
2019-03-21 10:16:00.467 [INF] BMGR: Processed 49 blocks in the last 6m42.3s (163 transactions, 231 tickets, 225 votes, 16 revocations, height 155619, 2019-03-21 10:15:59 -0400 EDT)

1 hour, 11 minutes and 17 seconds

mainnet:

2019-03-21 12:13:17.492 [INF] DCRD: Version 1.4.0-pre+dev (Go version go1.12.1 linux/amd64)
2019-03-21 13:59:56.849 [INF] BMGR: Processed 202 blocks in the last 10.16s (1372 transactions, 1100 tickets, 999 votes, 18 revocations, height 328585, 2019-03-21 03:48:32 -0400 EDT)
2019-03-21 14:00:25.062 [INF] BMGR: Processed 146 blocks in the last 28.21s (868 transactions, 660 tickets, 710 votes, 24 revocations, height 328736, 2019-03-21 13:52:02 -0400 EDT)

1 hour, 47 minutes and 8 seconds


Calculations.

Testnet:
1.4.0 - 1.5.0:
4277 seconds - 2694 seconds = 1583 seconds.
37% faster

Mainnet:

1.4.0 - 1.5.0:
6428 seconds - 4884 seconds = 1544 seconds.

24 % faster

@jholdstock

This comment has been minimized.

Copy link
Member

jholdstock commented Mar 21, 2019

Initial sync times:

Testnet
SSD. Poor quality internet connection.
1.4.0: 29 mins
1656: 24 mins

Mainnet
SSD. High speed internet connection.
1.4.0: 54 mins
1656: 42 mins

@dnldd

dnldd approved these changes Mar 22, 2019

@davecgh davecgh requested a review from vctt94 Mar 22, 2019

davecgh added some commits Mar 13, 2019

txscript: mergeMultiSig function def order cleanup.
This moves the function definition for mergeMultiSig so it is more
consistent with the preferred order used through the codebase.  In
particular, the functions are defined before they're first used and
generally as close as possible to the first use when they're defined in
the same file.
txscript: Use raw scripts in RawTxInSignature.
This converts RawTxInSignature to make use of the recently converted
CalcSignatureHash function that works with raw scripts in order to
remove the reliance on parsed opcodes as a step towards utlimately
removing them altogether and updates the comment to explicitly call out
the script version semantics.

It is worth noting that this has the side effect of optimizing the
function as well, however, since this change is not focused on the
optimization aspects, no benchmarks are provided.
txscript: Use raw scripts in RawTxInSignatureAlt.
This converts RawTxInSignatureAlt to make use of the recently converted
CalcSignatureHash function that works with raw scripts in order to
remove the reliance on parsed opcodes as a step towards utlimately
removing them altogether and updates the comment to explicitly call out
the script version semantics.

It is worth noting that this has the side effect of optimizing the
function as well, however, since this change is not focused on the
optimization aspects, no benchmarks are provided.
txscript: Use raw scripts in SignTxOutput.
This converts SignTxOutput and supporting funcs, namely sign,
mergeScripts and mergeMultiSig, to make use of the new tokenizer as well
as some recently added funcs that deal with raw scripts in order to
remove the reliance on parsed opcodes as a step towards utlimately
removing them altogether and updates the comments to explicitly call out
the script version semantics.

It is worth noting that this has the side effect of optimizing the
function as well, however, since this change is not focused on the
optimization aspects, no benchmarks are provided.
txscript: Implement efficient opcode data removal.
This introduces a new function named removeOpcodeByDataRaw which accepts
the raw scripts and data to remove versus requiring the parsed opcodes
to both significantly optimize it as well as make it more flexible for
working with raw scripts.

There are several places in the rest of the code that currently only
have access to the parsed opcodes, so this only introduces the function
for use in the future and deprecates the existing one.

Note that, in practice, the script will never actually contain the data
that is intended to be removed since the function is only used during
signature verification to remove the signature itself which would
require some incredibly non-standard code to create.

Thus, as an optimization, it avoids allocating a new script unless there
is actually a match that needs to be removed.

Finally, it updates the tests to use the new function.
txscript: Make isDisabled accept raw opcode.
This converts the isDisabled function defined on a parsed opcode to a
standalone function which accepts an opcode as a byte instead in order
to make it more flexible for raw script analysis.

It also updates all callers accordingly.
txscript: Make alwaysIllegal accept raw opcode.
This converts the alwaysIllegal function defined on a parsed opcode to a
standalone function named isOpcodeAlwaysIllegal which accepts an opcode
as a byte instead in order to make it more flexible for raw script
analysis.

It also updates all callers accordingly.
txscript: Make isConditional accept raw opcode.
This converts the isConditional function defined on a parsed opcode to a
standalone function named isOpcodeConditional which accepts an opcode as
a byte instead in order to make it more flexible for raw script
analysis.

It also updates all callers accordingly.
txscript: Make min push accept raw opcode and data.
This converts the checkMinimalDataPush function defined on a parsed
opcode to a standalone function which accepts an opcode and data slice
instead in order to make it more flexible for raw script analysis.

It also updates all callers accordingly.
txscript: Convert to use non-parsed opcode disasm.
This converts the engine's current program counter disasembly to make
use of the standalone disassembly function to remove the dependency on
the parsed opcode struct.

It also updates the tests accordingly.
txscript: Refactor engine to use raw scripts.
This refactors the script engine to store and step through raw scripts
by making using of the new zero-allocation script tokenizer as opposed
to the less efficient method of storing and stepping through parsed
opcodes.  It also improves several aspects while refactoring such as
optimizing the disassembly trace, showing all scripts in the trace in
the case of execution failure, and providing additional comments
describing the purpose of each field in the engine.

It should be noted that this is a step towards removing the parsed
opcode struct and associated supporting code altogether, however, in
order to ease the review process, this retains the struct and all
function signatures for opcode execution which make use of an individual
parsed opcode.  Those will be updated in future commits.

The following is an overview of the changes:

- Modify internal engine scripts slice to use raw scripts instead of
  parsed opcodes
- Introduce a tokenizer to the engine to track the current script
- Remove no longer needed script offset parameter from the engine since
  that is tracked by the tokenizer
- Add an opcode index counter for disassembly purposes to the engine
- Update check for valid program counter to only consider the script
  index
  - Update tests for bad program counter accordingly
- Rework the NewEngine function
  - Store the raw scripts
  - Setup the initial tokenizer
  - Explicitly check against version 0 instead of DefaultScriptVersion
    which would break consensus if changed
  - Check the scripts parse according to version 0 semantics to retain
    current consensus rules
  - Improve comments throughout
- Rework the Step function
  - Use the tokenizer and raw scripts
  - Create a parsed opcode on the fly for now to retain existing
    opcode execution function signatures
  - Improve comments throughout
- Update the Execute function
  - Explicitly check against version 0 instead of DefaultScriptVersion
    which would break consensus if changed
  - Improve the disassembly tracing in the case of error
- Update the CheckErrorCondition function
  - Modify clean stack error message to make sense in all cases
  - Improve the comments
- Update the DisasmPC and DisasmScript functions on the engine
  - Use the tokenizer
  - Optimize construction via the use of strings.Builder
- Modify the subScript function to return the raw script bytes since the
  parsed opcodes are no longer stored
- Update the various signature checking opcodes to use the raw opcode
  data removal and signature hash calculation functions since the
  subscript is now a raw script
  - opcodeCheckSig
  - opcodeCheckMultiSig
  - opcodeCheckSigAlt
txscript: Rename removeOpcodeByDataRaw func.
This renames the removeOpcodeByDataRaw to removeOpcodeByData now that
the old version has been removed.
txscript: Rename calcSignatureHashRaw func.
This renames the calcSignatureHashRaw to calcSignatureHash now that the
old version has been removed.
txscript: Remove unused unparseScript func.
Also remove tests associated with unparsing opcodes accordingly.
txscript: Remove unused parseScriptTemplate func.
Also remove tests associated with the func accordingly.
txscript: Make executeOpcode take opcode and data.
This converts the executeOpcode function defined on the engine to accept
an opcode and data slice instead of a parsed opcode as a step towards
removing the parsed opcode struct and associated supporting code altogether.

It also updates all callers accordingly.
txscript: Make op callbacks take opcode and data.
This converts the callback function defined on the internal opcode
struct to accept the opcode and data slice instead of a parsed opcode as
the final step towards removing the parsed opcode struct and associated
supporting code altogether.

It also updates all of the callbacks and tests accordingly and finally
removes the now unused parsedOpcode struct.

@davecgh davecgh force-pushed the davecgh:txscript_zero_alloc_optimization_refactor branch from 4b45582 to 6adbaa6 Mar 26, 2019

@davecgh davecgh merged commit 6adbaa6 into decred:master Mar 26, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@davecgh davecgh deleted the davecgh:txscript_zero_alloc_optimization_refactor branch Mar 26, 2019

//
// NOTE: This function is only valid for version 0 scripts. Since the function
// does not accept a script version, the results are undefined for other script
// versions.
func ExtractCoinbaseNullData(pkScript []byte) ([]byte, error) {

This comment has been minimized.

Copy link
@nitronick600

nitronick600 Mar 26, 2019

Is it possible to make this function valid for versions > 0?

This comment has been minimized.

Copy link
@davecgh

davecgh Mar 26, 2019

Author Member

It's possible, but would require a v2 module since it requires the addition of a new version parameter which constitutes a major change according to semver. However, it's important to note that no script versions greater than 0 are currently valid and this is simply calling out those existing semantics.

That said, part of introducing new script versions will necessarily involve ensuring the new ScriptTokenizer which this PR introduces and does take a script version, is able to properly parse the newly introduced script version(s).

To that end, the function this performs is now possible and trivial to implement via the new aforementioned ScriptTokenizer this PR introduces:

package main

import (
	"encoding/hex"
	"fmt"

	"github.com/decred/dcrd/txscript"
)

func main() {
	// This would oridinarily come from the tx output.
	const scriptVersion = 0

	// This is from main chain block 330233 and would ordinarily come from
	// somehwere else.
	pkScript, err := hex.DecodeString("6a0cf9090500f1ee035d7e430274")
	if err != nil {
		fmt.Println(err)
		return
	}

	// Note that this is ignoring the max allowed length which could be added
	// too, but for most applications, it won't matter hence I omitted it here.
	var data []byte
	if len(pkScript) > 1 && pkScript[0] == txscript.OP_RETURN {
		tokenizer := txscript.MakeScriptTokenizer(scriptVersion, pkScript[1:])
		if tokenizer.Next() && tokenizer.Done() &&
			tokenizer.Opcode() <= txscript.OP_PUSHDATA4 {

			data = tokenizer.Data()
		}
	}

	fmt.Printf("Coinbase data: %x", data)
}

Output:

Coinbase data: f9090500f1ee035d7e430274
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.