Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate Sobek and k6 logic from the module logic #271

Open
inancgumus opened this issue Mar 10, 2022 · 4 comments
Open

Separate Sobek and k6 logic from the module logic #271

inancgumus opened this issue Mar 10, 2022 · 4 comments
Labels
mapping k6 browser to Goja mapping related. productivity Issues and PRs that improve our productivity refactor

Comments

@inancgumus
Copy link
Member

inancgumus commented Mar 10, 2022

This can also mitigate some of the known bugs caused by the concurrent Sobek usage throughout the extension 🤞

Runtime is most needed because of Sobek conversions. If we can provide a translation layer between Sobek and the extension code, we can greatly simplify it. For example:

func (b *BrowserContext) WaitForEvent(event string, optsOrPredicate sobek.Value) interface{} {
	rt := k6common.GetRuntime(b.ctx)

	if optsOrPredicate != nil && !sobek.IsUndefined(optsOrPredicate) && !sobek.IsNull(optsOrPredicate) {
		switch optsOrPredicate.ExportType() {
		case reflect.TypeOf(sobek.Object{}):
			opts := optsOrPredicate.ToObject(rt)
			for _, k := range opts.Keys() {
				switch k {
				case "predicate":
					predicateFn, isCallable = sobek.AssertFunction(opts.Get(k))
					if !isCallable {
						k6Throw(b.ctx, "expected callable predicate")
					}
				case "timeout":
					timeout = time.Duration(opts.Get(k).ToInteger()) * time.Millisecond
				}
			}

This kind of cluttering happens because of internally exposing Sobek, whereas we could do:

// Extension mechanics package (actual logic here)

package browser

func (c *Context) WaitForEvent(event string, predicate func()) (Event, error) {
	...
	return predicate(c.page)
}

// Translation layer (Sobek -> extension) — It's now named as the mapping layer.

type BrowserContext struct {
	...
	bc browser.Context
	...
}

func (b *BrowserContext) WaitForEvent(event string, optsOrPredicate sobek.Value) interface{} {
	// sobek stuff here
	r, err := b.bc.WaitForEvent(event predicate)
	if err != nil {
		k6Throw(...)
	}   
}

                      Test script
                           │
                           │
┌──────────────────────────▼────────────────────────────┐
│            Exposed methods to the test script         │
│          Extension translation layer: JS -> Sobek     │
└──────────────────────────┬────────────────────────────┘
                           │
┌──────────────────────────┴────────────────────────────┐
│                          ▼                            │
│                                                       │
│               Extension logic (Sobek-free)            │
│                                                       │
│                          │                            │
└──────────────────────────┼────────────────────────────┘
                           │
                           │
             ┌─────────────┼─────────────┐
             │             ▼             │
             │                           │
             │          Browser          │
             │                           │
             │                           │
             └───────────────────────────┘

See #268 (comment).

@imiric
Copy link
Contributor

imiric commented Mar 10, 2022

This is also important to avoid data races from sharing goja.Runtime. See #254 (comment) and #263 (review). The quickest relief would be to remove it from ElementHandle.eval().

@inancgumus
Copy link
Member Author

quickest relief would be to remove it from ElementHandle.eval()

What's on your mind doing so? :)

@imiric
Copy link
Contributor

imiric commented Mar 10, 2022

Well, we currently pass goja.Values to this function, and to ExecutionContext.eval(), and do the conversion inside it:

expression = pageFunc.ToString().String()

func convertArgument(ctx context.Context, execCtx *ExecutionContext, arg goja.Value) (*cdpruntime.CallArgument, error) {

Instead, this should happen much higher up in the stack, as close as possible or inside the methods we already expose to JS, via this JS->Go translation layer. This way we should(?) ensure a single goroutine has access to the goja.Runtime, and then when used internally we can worry about our own data races.

@inancgumus
Copy link
Member Author

inancgumus commented Mar 10, 2022

+1 That is exactly what I had in my mind: Goja should not enter into the extension logic realm.

imiric pushed a commit that referenced this issue Mar 22, 2022
The previous way of waiting for navigation events had a couple of issues:

- There was a race condition where the Page.frameNavigated CDP event was
  received before we had a chance to setup the waiter for it, causing
  timeouts (#272).

- The implementation was repeated two times in FrameManager
  (NavigateFrame and WaitForNavigation).

This change tries to setup the waiters concurrently with the
FrameSession.navigateFrame call, which minimizes--but doesn't completely
eliminate--the chance of a missed event and timeout. It also moves the
implementation to Frame and reuses it for both Goto and WaitForNavigation.
There's another version of this in Page.Reload which I didn't bother
changing here. It also does the JS option parsing upfront in the
functions exported to the script, so that internally we don't depend on
goja.Runtime (part of #271).

It's currently not possible to completely avoid the race, since there's
no way to synchronize when the waiters are actually started (i.e. when
the goroutine in createWaitForEventHandler is started). There's another
issue with Page.Goto when it sometimes returns a nil Response. I tried
fixing this by adding another waiter for EventPageResponse, but that
turned out to be racy as well (e.g. on internal pages like about:blank
or data URIs).

Fixing these issues will require a major change in how events are
handled internally.

Closes #272
imiric pushed a commit that referenced this issue Mar 23, 2022
The previous way of waiting for navigation events had a couple of issues:

- There was a race condition where the Page.frameNavigated CDP event was
  received before we had a chance to setup the waiter for it, causing
  timeouts (#272).

- The implementation was repeated two times in FrameManager
  (NavigateFrame and WaitForNavigation).

This change tries to setup the waiters concurrently with the
FrameSession.navigateFrame call, which minimizes--but doesn't completely
eliminate--the chance of a missed event and timeout. It also moves the
implementation to Frame and reuses it for both Goto and WaitForNavigation.
There's another version of this in Page.Reload which I didn't bother
changing here. It also does the JS option parsing upfront in the
functions exported to the script, so that internally we don't depend on
goja.Runtime (part of #271).

It's currently not possible to completely avoid the race, since there's
no way to synchronize when the waiters are actually started (i.e. when
the goroutine in createWaitForEventHandler is started). There's another
issue with Page.Goto when it sometimes returns a nil Response. I tried
fixing this by adding another waiter for EventPageResponse, but that
turned out to be racy as well (e.g. on internal pages like about:blank
or data URIs).

Fixing these issues will require a major change in how events are
handled internally.

Closes #272
imiric pushed a commit that referenced this issue Apr 13, 2022
This was causing data races if the runtime was also being used by the
event loop goroutine, since goja is not thread-safe. It also moves
towards removing the dependency on goja from internal methods, which is
what we're aiming for. A good rule of thumb to follow from now on is to
never use goja on unexported methods, and only on those that are
directly exposed to JS.

Part of #271
@inancgumus inancgumus changed the title Separate JS layer from logic Increase developer productivity by clearly separating the k6 browser code and Goja Jan 19, 2023
@inancgumus inancgumus added the dx developer experience label Jan 19, 2023
@inancgumus inancgumus self-assigned this Jan 19, 2023
@inancgumus inancgumus added this to the v0.9.0 milestone Feb 2, 2023
@inancgumus inancgumus added the mapping k6 browser to Goja mapping related. label Feb 28, 2023
@inancgumus inancgumus changed the title Increase developer productivity by clearly separating the k6 browser code and Goja Separate Goja to reduce bugs and increase dx Feb 28, 2023
This was referenced Feb 28, 2023
@inancgumus inancgumus removed their assignment Mar 2, 2023
@inancgumus inancgumus removed this from the v0.9.0 milestone Mar 31, 2023
@inancgumus inancgumus changed the title Separate Goja to reduce bugs and increase dx Separate Goja and k6 logic from the module logic Aug 24, 2023
@inancgumus inancgumus mentioned this issue Aug 25, 2023
3 tasks
@inancgumus inancgumus added productivity Issues and PRs that improve our productivity and removed dx developer experience labels Oct 11, 2023
@inancgumus inancgumus changed the title Separate Goja and k6 logic from the module logic Separate Sobek and k6 logic from the module logic Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mapping k6 browser to Goja mapping related. productivity Issues and PRs that improve our productivity refactor
Projects
None yet
Development

No branches or pull requests

2 participants