Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(starlark): make execution step limit configurable #947

Merged
merged 11 commits into from
Aug 31, 2023

Conversation

ecrupper
Copy link
Contributor

Ref: #943

Since we don't have a good idea on how to potentially calculate a limit, it's a good idea to make this value configurable on start up, rather than incrementing it whenever we run into problems.

Default is set to 7500, though I'm open to other ideas on that 👍 .

colindean and others added 2 commits August 28, 2023 16:20
5,000 was too few to enable the example added to the testdata to work, so was 6,000. I chose 7,500 arbitrarily after a test at 10,000 and both worked.

In the long term, this should probably be configurable so as not to require recompilation. For now, this kicks the can down the road while allowing this build matrix use case to exist.
@ecrupper ecrupper requested a review from a team as a code owner August 29, 2023 17:14
@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Merging #947 (afc99b7) into main (7c23391) will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #947      +/-   ##
==========================================
+ Coverage   70.79%   70.85%   +0.05%     
==========================================
  Files         311      311              
  Lines       12852    12858       +6     
==========================================
+ Hits         9099     9110      +11     
+ Misses       3288     3283       -5     
  Partials      465      465              
Files Changed Coverage Δ
compiler/native/expand.go 71.23% <100.00%> (ø)
compiler/native/native.go 86.91% <100.00%> (+0.64%) ⬆️
compiler/native/parse.go 84.61% <100.00%> (ø)
compiler/template/starlark/render.go 49.36% <100.00%> (+1.89%) ⬆️
scm/github/webhook.go 85.91% <100.00%> (+0.03%) ⬆️

... and 1 file with indirect coverage changes

Copy link
Contributor

@colindean colindean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to have a "Starlark execution environment" config struct where this limit and inevitably other stuff gets shoved. It would perhaps include the environment name, e.g.

thread := &starlark.Thread{Name: "templated-base"}

Maybe something like

type StarklarkEnv struct {
executionStepLimit     uint64
name string
}

Also, there's still a hardcoded step limit in the Render method on around line 41-42.

plyr4
plyr4 previously approved these changes Aug 29, 2023
wass3r
wass3r previously approved these changes Aug 30, 2023
@wass3r wass3r dismissed their stale review August 30, 2023 05:21

we should honor this request #943 (comment) for attribution

Copy link
Contributor

@colindean colindean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lookin' good.

I would like to see that StarlarkExecutionConfig struct idea, but maybe I'll do that in a follow up PR.

@wass3r
Copy link
Collaborator

wass3r commented Aug 30, 2023

@colindean what is the use case for controlling the thread name? or was that just arbitrarily chosen as an example for making it more extensible?

@colindean
Copy link
Contributor

Arbitrarily chosen. Wouldn't be surprised if there's other things in there that end up needing to be tuned

@colindean
Copy link
Contributor

For that config could also include things for controlling the inevitable subprocess, like a timeout, memory limits, etc

Copy link
Contributor

@KellyMerrick KellyMerrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ecrupper ecrupper merged commit 17af770 into main Aug 31, 2023
11 of 12 checks passed
@ecrupper ecrupper deleted the enhance/configurable-starlark-exec-limit branch August 31, 2023 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants