Url navigation change

[clanstyles]

I'm using the task system to run a chunk of tasks, then check for "completion" in another function. I'm trying to wait for the page to change and data to exist on page or check to see if the Url has changed.

What's the best pattern for this? Even after you click a button, the time to wait for a new page is hard. The page has no unique elements so the current page's (WaitVisible) wont work.

[kenshaw]

To be honest, the best way to accomplish a lot of this would be to monitor the Network domain events. The model we have with chromedp though, doesn't monitor Network events, but uses the Page and DOM events. There's "yet another dimension" though, for monitoring above the "Page" level that can only be surfaced through monitoring Network events.

Timing is definitely difficult; you'll see this problem exists more or less in all the various mechanisms for testing/driving browsers. It's difficult to give a general answer that you would always do XYZ, as every page/site is so different, that there's no real catchall. Typically, however, we use WaitNotVisible and then WaitVisible to essentially wait for page-level changes. Since in the ways that we use chromedp, we're not monitoring the low-level Network events.

[clanstyles]

So in this exact instance, there's no direct element that identifies this unique page. There is unique content on the page, but "WaitVisible" for specific text isn't accessable.

Maybe make the following externally accessable? Or wrap it in some Action we can then chain to WaitVisible?

	textJS = `(function(a) {
		var s = '';
		for (var i = 0; i < a.length; i++) {
			if (a[i].offsetParent !== null) {
				s += a[i].textContent;
			}
		}
		return s;
	})($x('%s/node()'))`

Right now I'm trying to do something like this

	to := time.Second * 5
	found := false

	for {
		// err := chromedp.ActionFunc(func(ctx context.Context, h cdp.Handler) error {
		fn := chromedp.QueryAfter(".content .contentblock .subline span", func(ctx context.Context, h cdp.Handler, nodes ...*cdp.Node) error {
			if len(nodes) == 0 {
				return nil
			}

			for _, node := range nodes {
				var txt string
				if err := chromedp.EvaluateAsDevTools(fmt.Sprintf(textJS, node.FullXPath()), &txt).Do(ctx, h); err != nil {
					return err
				}

				if strings.ContainsAny(txt, "Thank you") {
					found = true
				}
			}

			return nil
		})

		if err := m.chrome.Run(context.Background(), fn); err != nil {
			panic(err)
		}

		if found {
			return true
		}

		time.Sleep(to)
	}

This code obviously has a bunch of flaws and possible dead locks.

How would I monitor the nextwork events?

[clanstyles]

The real problem is, even if I run this code, it can happen so fast that knowing if ti SHOULD run becomes the issue. The page needs to have been loaded or some javascript needs to have been executed.

For example some background ajax that says that an input is invalid.

[clanstyles]

Can an action retry a whole task list? IE: this failed, retry?

[kenshaw]

Can an action retry a whole task list? IE: this failed, retry?

No, but there's an interesting idea there in terms of a high level action that might make sense. I'll think about it, and let me see about putting something like that in.

So in this exact instance, there's no direct element that identifies this unique page. There is unique content on the page, but "WaitVisible" for specific text isn't accessable.

As far as Chrome is concerned, there's no difference between text / or some other node. You just need to know how to write your selectors. You can do it with either CSS or XPath.

It would be something like this: .//*[contains(text(),"search text")]. If you use this with WaitVisible, then you would end up waiting for the node, as the visibility check that is performed is almost identical.

If you need to run javascript, btw, just use the Eval actions.

[mvdan]

Looks like the discussion here ended long ago, so I'm closing.