How to drag with rod?

[NormalPunch]

I tried to solve a drag captcha with rod:

The captcha will be loaded in an iframe, I can click buttons in the iframe but cannot drag.

	el:=page.Element(`#iframe`)
	iframe := el.WaitVisible().Frame()

        drag := frame.Element(`#drag`).WaitVisible()

        // copy from the Click 
	box, err := drag.ScrollIntoView().BoxE()
	if err != nil {
		panic(err)
	}

	x := box.Left + box.Width/2
	y := box.Top + box.Height/2

	err = iframe.Mouse.MoveE(x, y, 1)
	if err != nil {
		panic(err)
	}

       iframe.Mouse.Down(proto.InputMouseButtonLeft)
       iframe.Mouse.Move(1, 0)
       iframe.Mouse.Move(1, 0)
       iframe.Mouse.Move(2, 0)
       ...
       iframe.Mouse.Up(proto.InputMouseButtonLeft)

       

It seemed that only the Down and Up success, but the Move do nothing with no error.

Any idea? Thanks!

[NormalPunch]

I thought Swipe = Down - Moves - Up , but looked into rod a little, really couldn't move, Down - Moves - Up was more like Click...

But Move worked fine when used alone...I mean, without holding the InputMouseButtonLeft

[NormalPunch]

Chrome: 83

[ysmood]

Sorry, this is a known issue:

https://github.com/ysmood/rod/blob/efd99d27e081c6a62603153074a46c3f01759233/page_test.go#L293-L304

[ysmood]

I still don't know why it doesn't work. Feels like a bug of Chrome

[ysmood]

If you look at the puppeteer.js, they have the same issue:

puppeteer/puppeteer#4562

puppeteer/puppeteer#1376

[NormalPunch]

I still don't know why it doesn't work. Feels like a bug of Chrome

Shocked at this fact...

[ysmood]

But if you use headless mode, it should work as expected. This issue is only for head mode which means you can't watch it on live. This test shows the events will happen correctly:

https://github.com/ysmood/rod/blob/f59cc63696ec5beef2bc5ec0541bebd3f50330f1/page_test.go#L293-L315

I think the reason is that in head mode chrome will use the position of your real mouse, not the mocked one. A typical race condition bug which is really hard to resolve (since you still want to interact with the controlled page with the real mouse to debug), that's why this bug is there for so many years.

[ysmood]

You can use the monitor helper to watch what's happening when it runs in the background:

https://github.com/ysmood/rod/blob/f59cc63696ec5beef2bc5ec0541bebd3f50330f1/examples_test.go#L69-L70

[NormalPunch]

But if you use headless mode, it should work as expected. This issue is only for head mode which means you can't watch it on live. This test shows the events will happen correctly:

https://github.com/ysmood/rod/blob/f59cc63696ec5beef2bc5ec0541bebd3f50330f1/page_test.go#L293-L315

I think the reason is that in head mode chrome will use the position of your real mouse, not the mocked one. A typical race condition bug which is really hard to resolve (since you still want to interact with the controlled page with the real mouse to debug), that's why this bug is there for so many years.

Thanks @ysmood , but it was still stucked. 😥

[ysmood]

Here's the example, it works on headless mode:

func main() {
	browser := rod.New().Connect()
	defer browser.Close()

	pageURL := "your url"

	page.Navigate(pageURL)

	wait := page.WaitRequestIdle()
	wait()

	box := page.Element(".slider").Box()
	kit.Dump(box)

	mouse := page.Mouse

	mouse.Move(box.Left, box.Top)
	mouse.Down("left")

	mouse.MoveE(box.Left+100, box.Top, 5) // drag 100px to the right

	page.Screenshot("")

	mouse.Up("left")

	page.Screenshot("")
}

[NormalPunch]

Ok, I get it now, thanks @ysmood !

[ysmood]

The latest chrome has fixed this issue, now it works for both headless and headful.

[heemike]

page.Element("#pager").Box() ,出现错误，没有这个方法Box()。

[ysmood]

The code is outdated, check the new doc:

rod/input_test.go

Lines 184 to 197 in b7f9c86

	func TestMouseDrag(t *testing.T) {
	g := setup(t)
	page := g.newPage().MustNavigate(g.srcFile("fixtures/drag.html")).MustWaitLoad()
	mouse := page.Mouse
	mouse.MustMoveTo(3, 3)
	mouse.MustDown("left")
	g.E(mouse.MoveLinear(proto.NewPoint(60, 80), 3))
	mouse.MustUp("left")
	utils.Sleep(0.3)
	g.Eq(page.MustEval(`() => dragTrack`).Str(), " move 3 3 down 3 3 move 22 28 move 41 54 move 60 80 up 60 80")
	}