Page.waitForSelector timeout issue with parallel page tests in a single browser instance

[quicksnap]

The following code is giving me issues: (Explanation below)

// My tests contain about 30 pages I want to test in parallel
const aBunchOfUrls = [
  {
    desc: 'Name of test #1',
    url: SOME_URL,
  },
  {
    desc: 'Name of test #2',
    url: ANOTHER_URL,
  },
  // ... snip ...
];

const browserPromise = puppeteer.launch();

// These test pass! And rather quickly. Slowest link is the backend server.
// They're running concurrently, generating a new page within the same browser instance
describe('Generate about 20 parallel page tests', () => {
  aBunchOfUrls.forEach((testObj, idx) => {
    it.concurrent(testObj.desc, async () => {
      const browser = await browserPromise;
      const page = await browser.newPage();

      await page.goto(testObj.url, { waitUntil: 'networkidle' });
      await page.waitForSelector('#content');

      // assert things..
    });
  });
});

// This test will cause `waitForSelector` to time out, but, as the logging shows, the classnames are
// present in the document at the time of timeout
describe('Another test that times out =(', () => {
  it('Shouldnt time out', async () => {
    const browser = await browserPromise;
    const page = await browser.newPage();

    await page.goto(
      ROOT_URL,
      { waitUntil: 'networkidle' },
    );

    await page.waitForSelector('#content');

    try {
      const classNameBefore = await page.evaluate(() => {
        return document.getElementById('global-loading-spinner').className;
      });
      console.log(`${classNameBefore}`); // Logs: "global-loading-spinner"

      await page.waitForSelector('.hidden', { timeout: 10000 });
    } catch (e) {
      const classNameAfter = await page.evaluate(() => {
        return document.getElementById('global-loading-spinner').className;
      });

      // The log statement below contains the classname we were waiting for!
      console.log(`${classNameAfter}`); // Logs: "global-loading-spinner hidden"
      throw e;
    }

    // Assert things..
  });
});

The second test only times out (sporadically) after hammering the browser with concurrent page tests. Every time it times out, the evaluate/log combo shows the classname we're waiting for is present in the document.

Before I go and try to set up a replication app or anything, I wanted to see if there was any ideas on why this was happening.

Thanks much! I'll try to replicate in a sandbox if it helps.

[aslushnikov]

Quick question: would it timeout for you is you pass in visible: true?

await page.waitForSelector('.hidden', { timeout: 10000, visible: true });

[quicksnap]

@aslushnikov it passes when I pass that option, but only due to CSS visibility transition being enabled =) When I disable the CSS transition, it fails again.

Essentially I'm trying to do the opposite of visible: true. When hidden happens, for all intents and purposes, we should assume there's no css transition and the element has disappeared.

Is this a problem where waitForSelector isn't expecting that case? Should we add a notVisible option?

Lastly, would a small test repo for this help out?

[quicksnap]

I also just tried the original code, but with CSS transitions disabled for my element. Same failure.

[quicksnap]

I think the problem may be here: https://github.com/GoogleChrome/puppeteer/blob/151d512ae2d652d8bc91db1ebcb635fd24a45d1a/lib/FrameManager.js#L498-L501

MutationObserver isn't firing when I directly modify the className on the DOM in my code:
document.getElementById('global-loading-spinner').className = 'global-loading-spinner hidden';

Should the attributes option be added to the observer so it fires?

[quicksnap]

Adding this to my local copy seems to fix it.

[quicksnap]

Created PR @ #499

[stevenlafl]

What is this using? it.concurrent is not a function