Skip to content

Context canceled for JS loading site #1214

@suntong

Description

@suntong

And here's an example for chromedp:

// The CSS selector for the element you must wait for
const selectorToWaitFor = "a" 

var htmlContent string
const targetURL = "https://example.com"

log.Printf("Navigating to %s, waiting for '%s', and retrieving source...", targetURL, selectorToWaitFor)

err := chromedp.Run(ctx,
    // 1. Navigate to the target URL
    chromedp.Navigate(targetURL),
    
    // 2. Add the waiting condition here
    // This pauses execution until the element is visible in the DOM
    chromedp.WaitVisible(selectorToWaitFor, chromedp.ByQuery),
    
    // 3. Retrieve the OuterHTML of the entire 'html' element
    chromedp.OuterHTML(`html`, &htmlContent, chromedp.ByQuery),
)

It works for example.com, but not another site (with the same code) which uses JS to load its content (targetURL = "https://so.gitee.com/?q=Vue%20go", selectorToWaitFor = "a[aria-label=Next]") --

I'm using the latest github.com/chromedp/chromedp (v0.14.2), and

I set a generous overall timeout using context.WithTimeout (300 seconds) for the entire operation. However, during execution of specific actions (chromedp.WaitVisible), the program consistently fails with a "context canceled" error after approximately 7 seconds. The actual body returned is (for sure the element is not found):

 <div class="d-flex justify-content-center align-middle" style="height: 80vh;"><div class="m-auto"><div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div> <div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div> <div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div></div></div></body></html>

The site is heavy JS based, the initial load takes 6~7 seconds. then the browser should run JS to finish loading.

Using go-rod, and waiting for the same element, it can retrieve the whole JS loaded content.

The issue might be a lightpanda cloud browser issue, as I am using Google Chrome browser using browser=chrome parameter in the url for go-rod, yet when adding browser=chrome parameter in the url for chromedp, it failed with the following error (so I'm not sure where exactly the problem is):

Visiting 'https://example.com' (& wait for 'a') ...
2025/11/14 09:26:50 -> {"id":1,"method":"Target.createTarget","params":{"url":"about:blank","enableBeginFrameControl":false,"newWindow":false,"background":false,"forTab":false,"hidden":false}}
2025/11/14 09:26:51 <- {"id":1,"error":{"code":-32000,"message":"Failed to open new tab - no browser is open"}}
...
exit status 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions