Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

js: Cache scripts by absolute not relative paths #2903

Merged
merged 2 commits into from
Feb 7, 2023
Merged

Conversation

mstoykov
Copy link
Contributor

@mstoykov mstoykov commented Feb 6, 2023

This fixes a regression introduced in 2753196 where in all imports are cached on the specifier that is provided for them. This is fine for k6 modules which happen to always have the same absolute names.

But source modules can have relative paths that resolve to different absolute ones depending on where they are imported from.

For example: import { cool } from "./somepath/cool.js"` - was cached based on the name/key "./somepath/cool.js" and it will be returned if anywhere in the code that same import specifier is used.

But this is a bug if this is done from two different directories.

As in one called from /A/A.js and from /B/B.js should get two different files /A/somepath/cool.js and /B/somepath/cool.js. Instead, in v0.42.0 the first to load the relative path will be wrongly cached with the relative path, and then it will be used for the other import as well.

Now k6 again caches source modules based on their absolute paths.

Fixes #2902

This fixes a regression introduced in 2753196 where in all imports are
cached on the specifier that is provided for them. This is fine for k6
modules which happen to always have the same absolute names.

But source modules can have relative paths that resolve to different
absolute ones depending on where they are imported from.

Fixes #2902
@mstoykov mstoykov added high prio breaking change for PRs that need to be mentioned in the breaking changes section of the release notes labels Feb 6, 2023
@mstoykov mstoykov added this to the v0.43.0 milestone Feb 6, 2023
programs map[string]programWithSource
exportsCache map[string]goja.Value
programs map[string]programWithSource
// mege this and the above
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// mege this and the above
// merge this and the above

Can you document when it is expected to be done and what is the issue at the moment that blocks it, please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The blockign part is mostly that thsi will be quite a big refactor which in practice is done in #2881

if export, ok = i.k6ModulesCache[arg]; ok {
return export
}
defer func() { i.k6ModulesCache[arg] = export }()
Copy link
Contributor

@codebien codebien Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify why the code in defer is required, does it expected to be not nil in some cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code sets the cache to whatever is returned.

I can rewrite this more I guess moving the whole reference to k6ModulesCache to requireModule.

It was just a lot more easier and faster to just move teh code that did this irregardless of source or go module to be just a few lines below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want me to rewrite if we are going to be getting #2881 next release?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nop, fine for me then

olegbespalov
olegbespalov previously approved these changes Feb 6, 2023
Co-authored-by: Ivan <2103732+codebien@users.noreply.github.com>
@codecov-commenter
Copy link

codecov-commenter commented Feb 6, 2023

Codecov Report

Merging #2903 (fab5be7) into master (b750e20) will decrease coverage by 0.16%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #2903      +/-   ##
==========================================
- Coverage   76.88%   76.73%   -0.16%     
==========================================
  Files         225      223       -2     
  Lines       16867    16865       -2     
==========================================
- Hits        12969    12941      -28     
- Misses       3066     3086      +20     
- Partials      832      838       +6     
Flag Coverage Δ
ubuntu ?
windows 76.73% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
js/initcontext.go 88.18% <100.00%> (ø)
lib/netext/tls.go 27.08% <0.00%> (-22.92%) ⬇️
loader/readsource.go 75.86% <0.00%> (-10.35%) ⬇️
lib/netext/httpext/transport.go 94.17% <0.00%> (-2.92%) ⬇️
cmd/ui.go 66.83% <0.00%> (-2.05%) ⬇️
lib/testutils/httpmultibin/httpmultibin.go 88.35% <0.00%> (-1.59%) ⬇️
lib/executor/vu_handle.go 92.52% <0.00%> (-0.94%) ⬇️
js/modules/k6/ws/ws.go 85.50% <0.00%> (-0.58%) ⬇️
execution/scheduler.go 92.72% <0.00%> (-0.37%) ⬇️
cmd/ui_unix.go
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Contributor

@codebien codebien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, can you check that the failed test is just one more flaky, please?

@mstoykov mstoykov merged commit 85571ca into master Feb 7, 2023
@mstoykov mstoykov deleted the fixModuleCaching branch February 7, 2023 08:55
@imiric imiric mentioned this pull request Feb 10, 2023
imiric pushed a commit that referenced this pull request Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change for PRs that need to be mentioned in the breaking changes section of the release notes high prio
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Current file path is not taken into account when caching duplicate relative import paths
4 participants