Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Algod: Simulation PC exposure for App approval/clear-state transaction #5360

Merged
merged 83 commits into from
Jun 1, 2023
Merged
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
92187d9
init commit
ahangsu May 4, 2023
6daf24f
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 4, 2023
3900cb2
is this it
ahangsu May 4, 2023
7fe44ff
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 5, 2023
2d6f843
before program hook
ahangsu May 5, 2023
b64059f
attempted to tell lsig/approval/csp program before txn exec prog
ahangsu May 6, 2023
e594ff6
minor, oapi attempt to design exec trace incremental config
ahangsu May 6, 2023
0735f48
minor, change != to >
ahangsu May 6, 2023
c67d3a2
transaction trace rewrite logic
ahangsu May 8, 2023
e7eae8e
pc exposure from eval context
ahangsu May 8, 2023
8fbe1fd
I wonder if this exec trace stack thing adapt to inner txn?
ahangsu May 8, 2023
75d97c7
somehow I dont think we need before program
ahangsu May 8, 2023
5189e66
clean up a bit on logic
ahangsu May 8, 2023
086eb38
step to inner index util
ahangsu May 8, 2023
4aadd90
oops, return response generating
ahangsu May 9, 2023
b440932
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 9, 2023
6e48df8
minor, model description add required for txn exec trace
ahangsu May 9, 2023
86062bc
convert Trace to model.trace
ahangsu May 9, 2023
5a85641
try to add option to clerk for debugging new feature
ahangsu May 9, 2023
6612929
locally... it worked
ahangsu May 9, 2023
4965173
first testcase, with log limit lifting
ahangsu May 9, 2023
ad23922
add an option to disable simulation trace features
ahangsu May 10, 2023
02cc01a
pr response on clerk and oas2
ahangsu May 10, 2023
8ef5ac4
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 10, 2023
6b2b8a7
starting recursive max depth test
ahangsu May 10, 2023
5c943e5
unfinished test
ahangsu May 10, 2023
a93b573
test improved, trace get right
ahangsu May 10, 2023
103343a
minor refactoring convertTxnTrace
ahangsu May 10, 2023
d1b9356
minor refactoring on tracer.go BeforeTxn case separation for new elem…
ahangsu May 10, 2023
29af161
finish testcase with recursive calls
ahangsu May 11, 2023
e64d542
move the execTraceStack into tracer
ahangsu May 11, 2023
d5b8098
erroring on local disable trace earlier in simulate
ahangsu May 11, 2023
f75d5ef
fix logicsig implementation, I am not thinking thoroughly
ahangsu May 11, 2023
e26c5f2
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 11, 2023
bb42a1b
minor improving oas2 model file, make trace optional
ahangsu May 11, 2023
f777055
return txn trace for any txn
ahangsu May 12, 2023
7f7e181
oas2 trace step to inner map comment, change internal impl from map t…
ahangsu May 12, 2023
9523139
e2e test, including simulation trace option in response
ahangsu May 12, 2023
2bd88b2
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 13, 2023
1853a14
revert minor changes
ahangsu May 13, 2023
1395d5b
adapt to Jason's convension of local config + simulation impl
ahangsu May 15, 2023
288698d
update rest client test
ahangsu May 15, 2023
cd21658
minor, import format, check local config pc deactivated incur 400 err
ahangsu May 15, 2023
d8d211f
EnableSimulationDeveloperAPI local configuration
ahangsu May 15, 2023
f5e64cc
minor renaming other-transaction to non-app-call-transaction
ahangsu May 16, 2023
c635401
use before program, after program for logic sig verification
ahangsu May 16, 2023
7b0e0f1
minor, more comments and better formatting
ahangsu May 16, 2023
fa1047e
BeforeOpcode removing defer func
ahangsu May 16, 2023
18a2f7c
change to EnableDeveloperAPI
ahangsu May 16, 2023
705aac9
update REST API for exec trace config definition
ahangsu May 16, 2023
250b664
REST response drastically change: return AP/CSP/LOGIC trace + inner t…
ahangsu May 16, 2023
d47479a
helpers on enum
ahangsu May 17, 2023
7629952
remove enum, flatten result and request in simulation
ahangsu May 17, 2023
c4cf47c
update test case, add optin and clear out to ensure clear out trace c…
ahangsu May 17, 2023
de83e07
indenting program's itxn, better formatting
ahangsu May 17, 2023
829db2b
merge makeSimulateResult with makeSimulateResultWithVersion
ahangsu May 17, 2023
fc90ba8
daemon utils.go stop spamming same code everywhere
ahangsu May 17, 2023
8c62ca3
test file for trace.go for new things
ahangsu May 17, 2023
413493c
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 18, 2023
a37fb75
PR review response
ahangsu May 18, 2023
00570e9
trace cmdline rename
ahangsu May 18, 2023
2440ad2
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 19, 2023
e9025bb
update app failing test and logic sig failing test
ahangsu May 19, 2023
33352e9
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 19, 2023
c530a5a
readability
ahangsu May 22, 2023
337b107
Add spawned inners to opcode trace unit
jasonpaulos May 23, 2023
b831c18
vfuture consensus parameter
ahangsu May 23, 2023
ef711fc
testing teal code explanation
ahangsu May 23, 2023
6b6d01e
deep copy leading common steps
ahangsu May 23, 2023
a6e5aba
Merge pull request #2 from jasonpaulos/simulate-app-pc-expose-opcode-…
ahangsu May 23, 2023
244d78d
minor fix
ahangsu May 23, 2023
c482e05
remove SimulateProgramTrace object in oas2
ahangsu May 23, 2023
206b638
typo fix
ahangsu May 23, 2023
38d6823
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 24, 2023
f147040
Merge branch 'master' into simulate-app-pc-expose
ahangsu May 26, 2023
dcdac1f
clerk.go pr response
ahangsu May 26, 2023
f61d309
symmetric to convert to simulation trace model
ahangsu May 26, 2023
9640723
add logic for exec mode sensitive group index returning
ahangsu May 30, 2023
2610b29
remove exec trace config copying routines, use codec reflect marshalling
ahangsu May 30, 2023
c213a6b
avoid passing config.Local whole thing around
ahangsu May 30, 2023
4262da6
not ambiguous developer API boolean flag usage in testing for making …
ahangsu May 30, 2023
2f3144e
testcase change
ahangsu May 30, 2023
d392f99
use cx.GroupIndex
ahangsu May 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 10 additions & 0 deletions cmd/goal/clerk.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
simulateAllowMoreLogging bool
simulateAllowMoreOpcodeBudget bool
simulateExtraOpcodeBudget uint64
simulateEnableRequestTrace bool
)

func init() {
Expand Down Expand Up @@ -161,6 +162,7 @@
simulateCmd.Flags().BoolVar(&simulateAllowMoreLogging, "allow-more-logging", false, "Lift the limits on log opcode during simulation")
simulateCmd.Flags().BoolVar(&simulateAllowMoreOpcodeBudget, "allow-more-opcode-budget", false, "Apply max extra opcode budget for apps per transaction group (default 320000) during simulation")
simulateCmd.Flags().Uint64Var(&simulateExtraOpcodeBudget, "extra-opcode-budget", 0, "Apply extra opcode budget for apps per transaction group during simulation")
simulateCmd.Flags().BoolVar(&simulateEnableRequestTrace, "trace", false, "Enable simulation time execution trace of app calls")
}

var clerkCmd = &cobra.Command{
Expand Down Expand Up @@ -1276,6 +1278,7 @@
AllowEmptySignatures: simulateAllowEmptySignatures,
AllowMoreLogging: simulateAllowMoreLogging,
ExtraOpcodeBudget: simulateExtraOpcodeBudget,
ExecTraceConfig: traceCmdOptionToSimulateTraceConfigModel(),

Check warning on line 1281 in cmd/goal/clerk.go

View check run for this annotation

Codecov / codecov/patch

cmd/goal/clerk.go#L1281

Added line #L1281 was not covered by tests
}
err := writeFile(requestOutFilename, protocol.EncodeJSON(simulateRequest), 0600)
if err != nil {
Expand All @@ -1299,6 +1302,7 @@
AllowEmptySignatures: simulateAllowEmptySignatures,
AllowMoreLogging: simulateAllowMoreLogging,
ExtraOpcodeBudget: simulateExtraOpcodeBudget,
ExecTraceConfig: traceCmdOptionToSimulateTraceConfigModel(),

Check warning on line 1305 in cmd/goal/clerk.go

View check run for this annotation

Codecov / codecov/patch

cmd/goal/clerk.go#L1305

Added line #L1305 was not covered by tests
}
simulateResponse, responseErr = client.SimulateTransactions(simulateRequest)
} else {
Expand Down Expand Up @@ -1358,3 +1362,9 @@
}
return txgroup
}

func traceCmdOptionToSimulateTraceConfigModel() simulation.ExecTraceConfig {
return simulation.ExecTraceConfig{
Enable: simulateEnableRequestTrace,

Check warning on line 1368 in cmd/goal/clerk.go

View check run for this annotation

Codecov / codecov/patch

cmd/goal/clerk.go#L1366-L1368

Added lines #L1366 - L1368 were not covered by tests
}
}
73 changes: 73 additions & 0 deletions daemon/algod/api/algod.oas2.json
Original file line number Diff line number Diff line change
Expand Up @@ -3453,6 +3453,9 @@
"extra-opcode-budget": {
"description": "Applies extra opcode budget during simulation for each transaction group.",
"type": "integer"
},
"exec-trace-config": {
"$ref": "#/definitions/SimulateTraceConfig"
}
}
},
Expand All @@ -3475,6 +3478,16 @@
}
}
},
"SimulateTraceConfig": {
"description": "An object that configures simulation execution trace.",
"type": "object",
"properties": {
"enable": {
"description": "A boolean option for opting in execution trace features simulation endpoint.",
"type": "boolean"
}
}
},
"Box": {
"description": "Box name and its content.",
"type": "object",
Expand Down Expand Up @@ -3726,6 +3739,9 @@
"logic-sig-budget-consumed": {
"description": "Budget used during execution of a logic sig transaction.",
"type": "integer"
},
"exec-trace": {
"$ref": "#/definitions/SimulationTransactionExecTrace"
}
}
},
Expand Down Expand Up @@ -3830,6 +3846,60 @@
"type": "integer"
}
}
},
"SimulationOpcodeTraceUnit": {
"description": "The set of trace information and effect from evaluating a single opcode.",
"type": "object",
"required": [
"pc"
],
"properties": {
"pc": {
"description": "The program counter of the current opcode being evaluated.",
"type": "integer"
},
"spawned-inners": {
"description": "The indexes of the traces for inner transactions spawned by this opcode, if any.",
"type": "array",
"items": {
"type": "integer"
}
}
}
},
"SimulationTransactionExecTrace": {
"description": "The execution trace of calling an app or a logic sig, containing the inner app call trace in a recursive way.",
"type": "object",
"properties": {
"approval-program-trace": {
"description": "Program trace that contains a trace of opcode effects in an approval program.",
"type": "array",
"items": {
"$ref": "#/definitions/SimulationOpcodeTraceUnit"
}
},
"clear-state-program-trace": {
"description": "Program trace that contains a trace of opcode effects in a clear state program.",
"type": "array",
"items": {
"$ref": "#/definitions/SimulationOpcodeTraceUnit"
}
},
"logic-sig-trace": {
"description": "Program trace that contains a trace of opcode effects in a logic sig.",
"type": "array",
"items": {
"$ref": "#/definitions/SimulationOpcodeTraceUnit"
}
},
"inner-trace": {
"description": "An array of SimulationTransactionExecTrace representing the execution trace of any inner transactions executed.",
"type": "array",
"items": {
"$ref": "#/definitions/SimulationTransactionExecTrace"
}
}
}
}
},
"parameters": {
Expand Down Expand Up @@ -4459,6 +4529,9 @@
},
"eval-overrides": {
"$ref": "#/definitions/SimulationEvalOverrides"
},
"exec-trace-config": {
"$ref": "#/definitions/SimulateTraceConfig"
}
}
}
Expand Down
79 changes: 79 additions & 0 deletions daemon/algod/api/algod.oas3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -790,6 +790,9 @@
"eval-overrides": {
"$ref": "#/components/schemas/SimulationEvalOverrides"
},
"exec-trace-config": {
"$ref": "#/components/schemas/SimulateTraceConfig"
},
"last-round": {
"description": "The round immediately preceding this simulation. State changes through this round were used to run this simulation.",
"type": "integer"
Expand Down Expand Up @@ -1918,6 +1921,9 @@
"description": "Lifts limits on log opcode usage during simulation.",
"type": "boolean"
},
"exec-trace-config": {
"$ref": "#/components/schemas/SimulateTraceConfig"
},
"extra-opcode-budget": {
"description": "Applies extra opcode budget during simulation for each transaction group.",
"type": "integer"
Expand Down Expand Up @@ -1954,6 +1960,16 @@
],
"type": "object"
},
"SimulateTraceConfig": {
"description": "An object that configures simulation execution trace.",
"properties": {
"enable": {
"description": "A boolean option for opting in execution trace features simulation endpoint.",
"type": "boolean"
}
},
"type": "object"
},
"SimulateTransactionGroupResult": {
"description": "Simulation result for an atomic transaction group",
"properties": {
Expand Down Expand Up @@ -1996,6 +2012,9 @@
"description": "Budget used during execution of an app call transaction. This value includes budged used by inner app calls spawned by this transaction.",
"type": "integer"
},
"exec-trace": {
"$ref": "#/components/schemas/SimulationTransactionExecTrace"
},
"logic-sig-budget-consumed": {
"description": "Budget used during execution of a logic sig transaction.",
"type": "integer"
Expand Down Expand Up @@ -2031,6 +2050,60 @@
},
"type": "object"
},
"SimulationOpcodeTraceUnit": {
"description": "The set of trace information and effect from evaluating a single opcode.",
"properties": {
"pc": {
"description": "The program counter of the current opcode being evaluated.",
"type": "integer"
},
"spawned-inners": {
"description": "The indexes of the traces for inner transactions spawned by this opcode, if any.",
"items": {
"type": "integer"
},
"type": "array"
}
},
"required": [
"pc"
],
"type": "object"
},
"SimulationTransactionExecTrace": {
"description": "The execution trace of calling an app or a logic sig, containing the inner app call trace in a recursive way.",
"properties": {
"approval-program-trace": {
"description": "Program trace that contains a trace of opcode effects in an approval program.",
"items": {
"$ref": "#/components/schemas/SimulationOpcodeTraceUnit"
},
"type": "array"
},
"clear-state-program-trace": {
"description": "Program trace that contains a trace of opcode effects in a clear state program.",
"items": {
"$ref": "#/components/schemas/SimulationOpcodeTraceUnit"
},
"type": "array"
},
"inner-trace": {
"description": "An array of SimulationTransactionExecTrace representing the execution trace of any inner transactions executed.",
"items": {
"$ref": "#/components/schemas/SimulationTransactionExecTrace"
},
"type": "array"
},
"logic-sig-trace": {
"description": "Program trace that contains a trace of opcode effects in a logic sig.",
"items": {
"$ref": "#/components/schemas/SimulationOpcodeTraceUnit"
},
"type": "array"
}
},
"type": "object"
},
"StateDelta": {
"description": "Application state delta.",
"items": {
Expand Down Expand Up @@ -6289,6 +6362,9 @@
"eval-overrides": {
"$ref": "#/components/schemas/SimulationEvalOverrides"
},
"exec-trace-config": {
"$ref": "#/components/schemas/SimulateTraceConfig"
},
"last-round": {
"description": "The round immediately preceding this simulation. State changes through this round were used to run this simulation.",
"type": "integer"
Expand Down Expand Up @@ -6319,6 +6395,9 @@
"eval-overrides": {
"$ref": "#/components/schemas/SimulationEvalOverrides"
},
"exec-trace-config": {
"$ref": "#/components/schemas/SimulateTraceConfig"
},
"last-round": {
"description": "The round immediately preceding this simulation. State changes through this round were used to run this simulation.",
"type": "integer"
Expand Down