Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add web vital metrics #836

Merged
merged 19 commits into from
Mar 29, 2023
Merged

Add web vital metrics #836

merged 19 commits into from
Mar 29, 2023

Commits on Mar 23, 2023

  1. Add the downloaded web vital js library

    We're going to bundle the web vital js library from the internet
    instead of downloading them on each test run. This will ensure that
    users who run tests in a locked down environment can still work with
    web vitals, and it will avoid any CORS related issues. The downside
    is that we will need to manually update this library, and users who
    do not update to latest k6 binary will be working with an outdated
    version of the library.
    ankur22 committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    9372a00 View commit details
    Browse the repository at this point in the history
  2. Refactor to use helper gojaValueExists

    This will reduce the complexity of the method.
    ankur22 committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    840cc51 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ebce769 View commit details
    Browse the repository at this point in the history
  4. Add implementation to page.evaluateOnNewDocument

    This will evaluate the script on the current page. BrowserContext
    relies on page.evaluateOnNewDocument when a new init script is added
    so that they are applied to all existing pages.
    
    For now, the preference is not to work with the worldName param in the
    AddScriptToEvaluateOnNewDocument CDP command. For now we don't want
    the script to be isolated to a particular named world.
    
    The script identifier that is returned from a successful CDP command
    to add the init script, is ignored. When we need it (e.g. for removing
    init scripts) we should use it then.
    
    Co-authored-by: Daniel Jiménez <daniel.jimenez@grafana.com>
    ankur22 and ka3de committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    519211a View commit details
    Browse the repository at this point in the history
  5. Add method on bCtx so page can apply init scripts

    When a new page is created, it needs to add the init scripts that the
    browser context has had applied to it.
    
    Instead of retrieving the list of scripts to apply in the page, we're
    sending the current page to the browser context, and it will loop over
    the scripts it has in evaluateOnNewDocumentSources.
    ankur22 committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    206f1fb View commit details
    Browse the repository at this point in the history
  6. Add binding which will be used for web vitals

    Bindings are a CDP runtime feature, where we can bind the current
    frame with a method on the window object in the browser. When the
    method is called with a payload, the listener in frame_session.go will
    currently read from the event and logs the output.
    ankur22 committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    e06b4d1 View commit details
    Browse the repository at this point in the history
  7. Refactor NewBrowserContext to return error

    By refactoring this one method, we need to follow the chain up so that
    other methods that work with NewBrowserContext also return an error.
    We need NewBrowserContext to return an error so that we can validate
    the web vital init script ,which is currently being added to
    evaluateOnNewDocumentSources without any validation (change to come in
    the next commit).
    ankur22 committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    af0baf4 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2023

  1. Update to add web vital script via AddInitScript

    Now that we can return errors from NewBrowserContext, we can also
    validate the web vital script. We might as well reuse the
    AddInitScript to help us validate and add the web vital script to
    evaluateOnNewDocumentSources and ready for use by the new pages.
    ankur22 committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    ff51716 View commit details
    Browse the repository at this point in the history
  2. Add web vital init script

    The Web Vital init script glues together the existing change. We first
    import the web vital lib and web vital init scripts into the website
    when a new page is created. The init script uses the web vital lib
    script which has been predownloaded, and the binding setup earlier to
    send the web vital metric payload to xk6-browser. At the moment the
    metric payload is logged (debug only).
    ankur22 committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    688616e View commit details
    Browse the repository at this point in the history
  3. Add the web vital metrics to the metrics registry

    This adds the six web vitals, as well as the three ratings for each.
    When a web vital metric is send from the JS code to k6-browser via
    the binding, it will need to extract the metric name and the rating.
    
    The rating can be good, needs-improvement or poor.
    ankur22 committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    90f9a53 View commit details
    Browse the repository at this point in the history
  4. Add parse & emit of web vital metrics

    This unmarshalls the web vital metric from the JS script; does some
    error checking (ensuring that the metric is recognized and is
    registered), and finally emits the metric.
    ankur22 committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    a6bfe60 View commit details
    Browse the repository at this point in the history
  5. Remove first contentful paint cdp metric

    We're now getting web vital metrics which are calculated by the JS
    web vital script. It also contains first contentful paint FCP metric.
    The metric coming from the JS lib also comes back with a rating. We
    can't correlate that with the current CDP FCP. To keep web vital
    metrics source consistent, we will remove the existing CDP FCP and opt
    to work with the one coming from the JS lib.
    ankur22 committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    bd370fa View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2023

  1. Refactor error formatting

    Co-authored-by: İnanç Gümüş <inanc.gumus@grafana.com>
    Co-authored-by: Daniel Jiménez <daniel.jimenez@grafana.com>
    3 people committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    2ade4fb View commit details
    Browse the repository at this point in the history
  2. Refactor the binding function name

    This will hopefully help highlight that it is an internal k6-browser
    function that was defined by us, and also not be used by anyone else.
    
    Resolves: #836 (comment)
    ankur22 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    3e7471c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    09a4050 View commit details
    Browse the repository at this point in the history
  4. Remove the import of the wv module

    It was using POC code to import the module directly from the internet
    instead of using the library that was downloaded and embedded in the
    k6-browser binary and initialised when the browser context was
    created.
    
    Resolves: #836 (comment)
    ankur22 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    54c094d View commit details
    Browse the repository at this point in the history
  5. Add test to check web vital scripts added

    This test ensures that the embedded web vital scripts have been added
    to the browser contexts internal evaluateOnNewDocumentSources slice.
    It does not check whether this has then been applied to a page.
    ankur22 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    e4d42e3 View commit details
    Browse the repository at this point in the history
  6. Add custom metric sample chan in tests

    This allows us to read off the metric sample channel when it gets
    populated with metrics during a test run. This will be useful to
    assert that certain metrics have been emitted during a test.
    ankur22 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    65ab80a View commit details
    Browse the repository at this point in the history
  7. Add a web vital integration test

    This tests ensures that web vitals are measured and emitted to the
    metric samples channel. We read off the channel and assert that
    expected web vitals metrics have been emitted during the test run.
    ankur22 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    2eb7085 View commit details
    Browse the repository at this point in the history