Add support for error logging only

[doteric]

Hello 👋

We're running Memlab in AWS Lambdas, the amount of produced logs is very high and we've decided to set muteConsole to true, this however means that we do not receive error logs and only receive critical error logs. A nice thing to have would be to decide which type of logs to receive. This way only errors could be logged.

Cheers,
Eric

[JacksonGL]

I can add an additional config.muteConfig for fine tuning the console mute configuration.

[JacksonGL]

The new config.muteConfig option is now available in memlab@1.1.52. Here's an example code snippet for your reference:

const {run} = require('@memlab/api');
const {config} = require('@memlab/core');

config.muteConfig = {
  muteError: false,
  muteWarning: false,
  muteInfo: true,
  muteSuccess: true,
  muteLog: true,
  muteTable: true,
  muteTrace: true,
  muteTopLevel: true,
  muteHighLevel: true,
  muteMidLevel: true,
  muteLowLevel: true,
};

(async function () {
  const scenario = {
    url: () => 'https://www.facebook.com',
    action: async () => {
      throw new Error('test');
    },
  };
  const {leaks} = await run({scenario, skipWarmup: true});
})();

[doteric]

That was fast, big thanks @JacksonGL 🙇

[doteric]

Hey @JacksonGL , sorry to bother you, just one thing to note, it might be worth changing up the error and warn.

With the following config:

memlabConfig.muteConfig = {
      muteError: false,
      muteWarning: false,
      muteInfo: true,
      muteSuccess: true,
      muteLog: true,
      muteTable: true,
      muteTrace: true,
      muteTopLevel: true,
      muteHighLevel: true,
      muteMidLevel: true,
      muteLowLevel: true,
    };

I get the following log when an interaction fails:

2024-08-26T21:06:56.068ZSTART RequestId: a8ad0c51-3bd6-403f-9065-c8afa88c9a35 Version: $LATEST
  | 2024-08-26T21:08:15.427Z | start is not defined
  | 2024-08-26T21:08:15.428Z | addRow is not defined
  | 2024-08-26T21:08:15.429Z | addRow is not defined
  | 2024-08-26T21:08:15.429Z | addRow is not defined
... (more addRow errors)
  | 2024-08-26T21:08:47.067Z | �[K�[F�[K
  | 2024-08-26T21:09:22.716Z | �[K�[F�[K
  | 2024-08-26T21:09:57.444Z | interaction fail, making 1 more attempt(s)...
  | 2024-08-26T21:10:30.485Z | interaction fail
  | 2024-08-26T21:10:34.401Z | END RequestId: a8ad0c51-3bd6-403f-9065-c8afa88c9a35

That means that while the error logs show they aren't really useful as they do not state what exactly has failed. What I would suggest is error logging what interaction (and on what element) exactly has failed.
What do you think about that?

Cheers and really appreciate the work 💪

[JacksonGL]

@doteric You might want to add the --verbose flag when running memlab, which should provide a stack trace that pinpoints the line in your test scenario file where the interaction failed.

[JacksonGL]

Setting memlabConfig.verbose = true should get the same result.

[doteric]

Hey @JacksonGL ,
big thanks for the suggestions, however even with the verbose mode set to true the logs still don't really point out what exactly has failed, but just that an interaction has failed.

  | 2024-09-16T09:32:56.694Z | �[K�[F�[K
-- | -- | --
  | 2024-09-16T09:33:26.704Z | interaction fail

It would be nice to point out what interaction has failed, it would help in debugging a lot, thoughts? ;D

[JacksonGL]

@doteric Did a stack trace appear when you used the --verbose flag? The stack trace should point to the line that invokes puppeteer for interaction (e.g., page.click or something else) in your test scenario file where the exception happens.

[doteric]

@JacksonGL thanks for the the quick reply 🙇
With

memlabConfig.muteConfig = {
      muteError: false,
      muteWarning: false,
      muteInfo: true,
      muteSuccess: true,
      muteLog: true,
      muteTable: true,
      muteTrace: true,
      muteTopLevel: true,
      muteHighLevel: true,
      muteMidLevel: true,
      muteLowLevel: true,
};
// https://github.com/facebook/memlab/issues/124#issuecomment-2313668271
memlabConfig.verbose = true;

used I do not see any stack trace logs. I wonder if it's the matter of the logs being in AWS Lambda or the verbose mode not enabling correctly? 🤔

[JacksonGL]

@doteric I see, I wasn't aware that you had set the muteConfig in the last question. The stack traces are displayed at the lowLevel, and since you set muteLowLevel: true, MemLab didn't output the stack trace. I can create a patch to ensure the stack trace is shown in this scenario. Alternatively, you can set muteLowLevel: false to enable the output.

[doteric]

Thanks again @JacksonGL
enabling low level logs with verbose enabled produces a bit of unnecessary logs in our case, but does indeed output the stack trace which is very useful.
Logs:

2024-09-19T09:50:26.066Z | Xvfb: true
-- | --
  | 2024-09-19T09:50:26.066Z | Setting env variable PUPPETEER_DISABLE_HEADLESS_WARNING=1 to mute puppeteer warnings.
  | 2024-09-19T09:50:26.265Z | [1/6]: warming up web server (page-load)...
  | 2024-09-19T09:50:26.265Z | url: https://vvv
  | 2024-09-19T09:50:39.488Z | [2/6]: warming up web server (action-on-page)...
  | 2024-09-19T09:50:39.488Z | url: https://vvv
  | 2024-09-19T09:50:51.740Z | [3/6]: warming up web server (revert)...
  | 2024-09-19T09:50:51.740Z | url: https://vvv
  | 2024-09-19T09:51:04.092Z | [4/6]: warming up web server (page-load)...
  | 2024-09-19T09:51:04.093Z | url: https://vvv
  | 2024-09-19T09:51:16.141Z | [5/6]: warming up web server (action-on-page)...
  | 2024-09-19T09:51:16.141Z | url: https://vvv
  | 2024-09-19T09:51:28.420Z | [6/6]: warming up web server (revert)...
  | 2024-09-19T09:51:28.420Z | url: https://vvv
  | 2024-09-19T09:51:41.747Z | �[K�[K�[K�[K�[K�[K�[K�[K�[K�[K�[K�[KXvfb: true
  | 2024-09-19T09:51:41.980Z | {
  | 2024-09-19T09:51:41.980Z | "_browserVersion": "HeadlessChrome/123.0.6312.0",
  | 2024-09-19T09:51:41.980Z | "_puppeteerConfig": {
  | 2024-09-19T09:51:41.980Z | "headless": true,
  | 2024-09-19T09:51:41.980Z | "devtools": false,
  | 2024-09-19T09:51:41.980Z | "protocolTimeout": 300000,
  | 2024-09-19T09:51:41.980Z | "ignoreHTTPSErrors": true,
  | 2024-09-19T09:51:41.980Z | "ignoreDefaultArgs": [
  | 2024-09-19T09:51:41.980Z | "--disable-extensions"
  | 2024-09-19T09:51:41.980Z | ],
  | 2024-09-19T09:51:41.980Z | "userDataDir": "/tmp/memlab-2-1726739423880-1/data/profile",
  | 2024-09-19T09:51:41.980Z | "args": [
  | 2024-09-19T09:51:41.980Z | "--no-sandbox",
  | 2024-09-19T09:51:41.980Z | "--disable-notifications",
  | 2024-09-19T09:51:41.980Z | "--use-fake-ui-for-media-stream",
  | 2024-09-19T09:51:41.980Z | "--use-fake-device-for-media-stream",
  | 2024-09-19T09:51:41.980Z | "--js-flags=\"--no-move-object-start\"",
  | 2024-09-19T09:51:41.980Z | "--enable-precise-memory-info",
  | 2024-09-19T09:51:41.980Z | "--enable-webgl",
  | 2024-09-19T09:51:41.980Z | "browser-test",
  | 2024-09-19T09:51:41.980Z | "--allow-pre-commit-input",
  | 2024-09-19T09:51:41.980Z | "--disable-background-networking",
  | 2024-09-19T09:51:41.980Z | "--disable-background-timer-throttling",
  | 2024-09-19T09:51:41.980Z | "--disable-backgrounding-occluded-windows",
  | 2024-09-19T09:51:41.980Z | "--disable-breakpad",
  | 2024-09-19T09:51:41.980Z | "--disable-client-side-phishing-detection",
  | 2024-09-19T09:51:41.980Z | "--disable-component-extensions-with-background-pages",
  | 2024-09-19T09:51:41.980Z | "--disable-component-update",
  | 2024-09-19T09:51:41.980Z | "--disable-default-apps",
  | 2024-09-19T09:51:41.980Z | "--disable-dev-shm-usage",
  | 2024-09-19T09:51:41.980Z | "--disable-extensions",
  | 2024-09-19T09:51:41.980Z | "--disable-hang-monitor",
  | 2024-09-19T09:51:41.980Z | "--disable-ipc-flooding-protection",
  | 2024-09-19T09:51:41.980Z | "--disable-popup-blocking",
  | 2024-09-19T09:51:41.980Z | "--disable-prompt-on-repost",
  | 2024-09-19T09:51:41.980Z | "--disable-renderer-backgrounding",
  | 2024-09-19T09:51:41.980Z | "--disable-sync",
  | 2024-09-19T09:51:41.980Z | "--enable-automation",
  | 2024-09-19T09:51:41.980Z | "--enable-blink-features=IdleDetection",
  | 2024-09-19T09:51:41.980Z | "--export-tagged-pdf",
  | 2024-09-19T09:51:41.980Z | "--force-color-profile=srgb",
  | 2024-09-19T09:51:41.980Z | "--metrics-recording-only",
  | 2024-09-19T09:51:41.980Z | "--no-first-run",
  | 2024-09-19T09:51:41.980Z | "--password-store=basic",
  | 2024-09-19T09:51:41.980Z | "--use-mock-keychain",
  | 2024-09-19T09:51:41.980Z | "--disable-domain-reliability",
  | 2024-09-19T09:51:41.980Z | "--disable-print-preview",
  | 2024-09-19T09:51:41.980Z | "--disable-speech-api",
  | 2024-09-19T09:51:41.980Z | "--disk-cache-size=33554432",
  | 2024-09-19T09:51:41.980Z | "--mute-audio",
  | 2024-09-19T09:51:41.980Z | "--no-default-browser-check",
  | 2024-09-19T09:51:41.980Z | "--no-pings",
  | 2024-09-19T09:51:41.980Z | "--single-process",
  | 2024-09-19T09:51:41.980Z | "--font-render-hinting=none",
  | 2024-09-19T09:51:41.980Z | "--hide-scrollbars",
  | 2024-09-19T09:51:41.980Z | "--ignore-gpu-blocklist",
  | 2024-09-19T09:51:41.980Z | "--in-process-gpu",
  | 2024-09-19T09:51:41.980Z | "--window-size=1920,1080",
  | 2024-09-19T09:51:41.980Z | "--use-gl=angle",
  | 2024-09-19T09:51:41.980Z | "--use-angle=swiftshader",
  | 2024-09-19T09:51:41.980Z | "--allow-running-insecure-content",
  | 2024-09-19T09:51:41.980Z | "--disable-setuid-sandbox",
  | 2024-09-19T09:51:41.980Z | "--disable-site-isolation-trials",
  | 2024-09-19T09:51:41.980Z | "--disable-web-security",
  | 2024-09-19T09:51:41.980Z | "--no-sandbox",
  | 2024-09-19T09:51:41.980Z | "--no-zygote",
  | 2024-09-19T09:51:41.980Z | "--headless"
  | 2024-09-19T09:51:41.980Z | ],
  | 2024-09-19T09:51:41.980Z | "defaultViewport": null,
  | 2024-09-19T09:51:41.980Z | "executablePath": "/tmp/chromium"
  | 2024-09-19T09:51:41.980Z | },
  | 2024-09-19T09:51:41.980Z | "_consoleMessages": []
  | 2024-09-19T09:51:41.980Z | }
  | 2024-09-19T09:51:42.021Z | start is not defined
  | 2024-09-19T09:51:42.022Z | addRow is not defined
  | 2024-09-19T09:51:42.023Z | addRow is not defined
  | 2024-09-19T09:51:42.026Z | addRow is not defined
  | 2024-09-19T09:51:42.031Z | addRow is not defined
  | 2024-09-19T09:51:42.038Z | addRow is not defined
  | 2024-09-19T09:51:42.040Z | addRow is not defined
  | 2024-09-19T09:51:42.041Z | addRow is not defined
  | 2024-09-19T09:51:42.041Z | addRow is not defined
  | 2024-09-19T09:51:42.042Z | addRow is not defined
  | 2024-09-19T09:51:42.042Z | addRow is not defined
  | 2024-09-19T09:51:42.047Z | addRow is not defined
  | 2024-09-19T09:51:42.060Z | addRow is not defined
  | 2024-09-19T09:51:42.063Z | addRow is not defined
  | 2024-09-19T09:51:42.064Z | addRow is not defined
  | 2024-09-19T09:51:42.066Z | addRow is not defined
  | 2024-09-19T09:51:42.066Z | addRow is not defined
  | 2024-09-19T09:51:42.066Z | addRow is not defined
  | 2024-09-19T09:51:42.069Z | addRow is not defined
  | 2024-09-19T09:51:42.078Z | addRow is not defined
  | 2024-09-19T09:51:42.079Z | addRow is not defined
  | 2024-09-19T09:51:42.079Z | addRow is not defined
  | 2024-09-19T09:51:42.079Z | addRow is not defined
  | 2024-09-19T09:51:42.084Z | Browser version: HeadlessChrome/123.0.6312.0
  | 2024-09-19T09:51:42.084Z | Node.js version: v20.17.0
  | 2024-09-19T09:51:42.089Z | Start tracking JS heap
  | 2024-09-19T09:51:42.099Z | Thu Sep 19 2024 09:51:42 GMT+0000 (Coordinated Universal Time)
  | 2024-09-19T09:51:42.100Z | url: https://vvv
  | 2024-09-19T09:51:42.100Z | loading: https://vvv
  | 2024-09-19T09:51:42.101Z | Connecting to web server...
  | 2024-09-19T09:52:00.130Z | �[KCheck timeout '/betting/en-gb/football' set wait: 2000ms
  | 2024-09-19T09:52:00.634Z | �[Krunning a full GC...
  | 2024-09-19T09:52:06.099Z | �[Ktaking screenshot...
  | 2024-09-19T09:52:08.774Z | �[Kheap snapshot 0% complete
  | 2024-09-19T09:52:08.789Z | �[Kheap snapshot 1% complete
  | 2024-09-19T09:52:08.831Z | �[Kheap snapshot 3% complete
  | 2024-09-19T09:52:08.866Z | �[Kheap snapshot 5% complete
  | 2024-09-19T09:52:08.930Z | �[Kheap snapshot 7% complete
  | 2024-09-19T09:52:08.948Z | �[Kheap snapshot 9% complete
  | 2024-09-19T09:52:08.980Z | �[Kheap snapshot 11% complete
  | 2024-09-19T09:52:09.010Z | �[Kheap snapshot 13% complete
  | 2024-09-19T09:52:09.048Z | �[Kheap snapshot 15% complete
  | 2024-09-19T09:52:09.074Z | �[Kheap snapshot 17% complete
  | 2024-09-19T09:52:09.100Z | �[Kheap snapshot 18% complete
  | 2024-09-19T09:52:09.176Z | �[Kheap snapshot 20% complete
  | 2024-09-19T09:52:09.215Z | �[Kheap snapshot 22% complete
  | 2024-09-19T09:52:09.257Z | �[Kheap snapshot 24% complete
  | 2024-09-19T09:52:09.313Z | �[Kheap snapshot 26% complete
  | 2024-09-19T09:52:09.341Z | �[Kheap snapshot 28% complete
  | 2024-09-19T09:52:09.368Z | �[Kheap snapshot 30% complete
  | 2024-09-19T09:52:09.399Z | �[Kheap snapshot 32% complete
  | 2024-09-19T09:52:09.430Z | �[Kheap snapshot 34% complete
  | 2024-09-19T09:52:09.469Z | �[Kheap snapshot 35% complete
  | 2024-09-19T09:52:09.487Z | �[Kheap snapshot 37% complete
  | 2024-09-19T09:52:09.529Z | �[Kheap snapshot 39% complete
  | 2024-09-19T09:52:09.564Z | �[Kheap snapshot 41% complete
  | 2024-09-19T09:52:09.596Z | �[Kheap snapshot 43% complete
  | 2024-09-19T09:52:09.629Z | �[Kheap snapshot 45% complete
  | 2024-09-19T09:52:09.660Z | �[Kheap snapshot 47% complete
  | 2024-09-19T09:52:09.693Z | �[Kheap snapshot 49% complete
  | 2024-09-19T09:52:09.720Z | �[Kheap snapshot 51% complete
  | 2024-09-19T09:52:09.818Z | �[Kheap snapshot 52% complete
  | 2024-09-19T09:52:09.894Z | �[Kheap snapshot 54% complete
  | 2024-09-19T09:52:09.913Z | �[Kheap snapshot 56% complete
  | 2024-09-19T09:52:09.946Z | �[Kheap snapshot 58% complete
  | 2024-09-19T09:52:09.979Z | �[Kheap snapshot 60% complete
  | 2024-09-19T09:52:10.009Z | �[Kheap snapshot 62% complete
  | 2024-09-19T09:52:10.030Z | �[Kheap snapshot 64% complete
  | 2024-09-19T09:52:10.055Z | �[Kheap snapshot 66% complete
  | 2024-09-19T09:52:10.075Z | �[Kheap snapshot 68% complete
  | 2024-09-19T09:52:10.111Z | �[Kheap snapshot 69% complete
  | 2024-09-19T09:52:10.131Z | �[Kheap snapshot 71% complete
  | 2024-09-19T09:52:10.160Z | �[Kheap snapshot 73% complete
  | 2024-09-19T09:52:10.179Z | �[Kheap snapshot 75% complete
  | 2024-09-19T09:52:10.197Z | �[Kheap snapshot 77% complete
  | 2024-09-19T09:52:10.216Z | �[Kheap snapshot 79% complete
  | 2024-09-19T09:52:10.233Z | �[Kheap snapshot 81% complete
  | 2024-09-19T09:52:10.285Z | �[Kheap snapshot 83% complete
  | 2024-09-19T09:52:10.303Z | �[Kheap snapshot 85% complete
  | 2024-09-19T09:52:10.324Z | �[Kheap snapshot 86% complete
  | 2024-09-19T09:52:10.364Z | �[Kheap snapshot 88% complete
  | 2024-09-19T09:52:10.389Z | �[Kheap snapshot 90% complete
  | 2024-09-19T09:52:10.405Z | �[Kheap snapshot 92% complete
  | 2024-09-19T09:52:10.421Z | �[Kheap snapshot 94% complete
  | 2024-09-19T09:52:10.457Z | �[Kheap snapshot 96% complete
  | 2024-09-19T09:52:10.478Z | �[Kheap snapshot 98% complete
  | 2024-09-19T09:52:10.786Z | �[Kheap snapshot 99% complete
  | 2024-09-19T09:52:11.230Z | �[Kheap snapshot 100% complete
  | 2024-09-19T09:52:11.230Z | �[Kheap snapshot 100% complete
  | 2024-09-19T09:52:13.689Z | �[Kduration: 5.7s
  | 2024-09-19T09:52:13.689Z | snapshot saved to disk
  | 2024-09-19T09:52:13.690Z | �[K�[K
  | 2024-09-19T09:52:13.690Z | �[K�[F�[K�[K�[K�[KThu Sep 19 2024 09:52:13 GMT+0000 (Coordinated Universal Time)
  | 2024-09-19T09:52:13.690Z | url: https://vvv
  | 2024-09-19T09:52:43.711Z | TimeoutError: Waiting for selector `div[data-test-id="test"]` failed: Waiting failed: 30000ms exceeded
  | 2024-09-19T09:52:43.711Z | at new WaitTask (/opt/node_modules/puppeteer-core/lib/cjs/puppeteer/common/WaitTask.js:50:34)
  | 2024-09-19T09:52:43.711Z | at IsolatedWorld.waitForFunction (/opt/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Realm.js:25:26)
  | 2024-09-19T09:52:43.711Z | at CSSQueryHandler.waitFor (/opt/node_modules/puppeteer-core/lib/cjs/puppeteer/common/QueryHandler.js:172:95)
  | 2024-09-19T09:52:43.711Z | at async CdpFrame.waitForSelector (/opt/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Frame.js:522:21)
  | 2024-09-19T09:52:43.711Z | at async CdpPage.waitForSelector (/opt/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:1305:20)
  | 2024-09-19T09:52:43.711Z | at async openTest (/var/task/index.js:88:3)
  | 2024-09-19T09:52:43.711Z | at async openExample (/var/task/index.js:98:3)
  | 2024-09-19T09:52:43.711Z | at async action (/var/task/index.js:117:5)
  | 2024-09-19T09:52:43.711Z | interaction fail

Might be nice to have the stack trace logs without low level logs enabled, what do you think? 🤔
At worse it's not the end of the world for us if we have some unnecessary logs, so let me know if you plan to change something or keep as is 🙇

[JacksonGL]

@doteric The patch is released in memlab@1.1.55. With the following setting, it will print the stack traces without the other lowLevel messages:

config.muteConfig = {
  muteError: false,
  muteWarning: false,
  muteInfo: true,
  muteSuccess: true,
  muteLog: true,
  muteTable: true,
  muteTrace: true,
  muteTopLevel: true,
  muteHighLevel: true,
  muteMidLevel: true,
  muteLowLevel: true,
};

config.verbose = true;