How to implement a double click or right click on selector

[terrygoldman]

I would like to perform a mouse double click on the table row.

If I do a:
click some_row_selector
click some_row_selector

It won't work as the webpage see it as 2 single clicks.

I can do a hover some_row_selector, which works

Any ideas?

BR,
Terry

[kensoh]

Hi Terry, thanks for raising this. This is not added as part of the standard steps as double click is not a usual action for web applications. In the meantime, if you are running it in default mode using PhantomJS browser, you can try the following -

this.mouse.doubleclick(tx(identifier));

This uses the CasperJS doubleclick function to click on the element, while still using TagUI smart identifier by using the tx(). Looks like adding a new dclick step will be useful to make double-clicking easier for Chrome browser and visual automation using Sikuli.

[terrygoldman]

Great, thanks for the prompt response, that work as expected.

[kensoh]

Besides dblick step, rclick step would also be good, to support more expressive UI interactions.

[kensoh]

Implemented with thanks to @Aussiroth and @lohvht - usable right away from cutting edge version.

[lookang]

dblick

dclick is the correct command
but i tried

/ click on first email
// take picture
snap editedaway@editedaway.com
dclick editedaway@editedaway.com

but it didn't double click on the text to launch a new page, any idea how to fix?

not dbclick

[kensoh]

Hi Lawrence, inside tagui/src/tagui_header.js, can you search for this function chrome.mouse.doubleclick and replace with following block of codes to see if either block works for this scenario?

If both does not work it could be how the web page somehow isn't automatable directly using this best practice approach from Puppeteer. In that case, can use the computer vision method to do the dclick to see if that works (either dclick directly visually or dclick base on an xy offset from an anchor element).

block 1 - moving mouse cursor to element before double-clicking

chrome.mouse.doubleclick = function(selector,y) { // double press and release on center of selector or point
if (!y) {chrome.scrollIntoViewIfNeeded(selector); var xy = chrome.mouse.getXY(selector);}
else var xy = {x: selector, y: y}; // get coordinates accordingly
chrome.mouse.action('mouseMoved',xy.x,xy.y,'none',0);
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',2); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',2);};

block 2 - separating the clicks into individual mouse down and up

chrome.mouse.doubleclick = function(selector,y) { // double press and release on center of selector or point
if (!y) {chrome.scrollIntoViewIfNeeded(selector); var xy = chrome.mouse.getXY(selector);}
else var xy = {x: selector, y: y}; // get coordinates accordingly
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);};

[lookang]

chrome.mouse.doubleclick = function(selector,y) { // double press and release on center of selector or point
if (!y) {chrome.scrollIntoViewIfNeeded(selector); var xy = chrome.mouse.getXY(selector);}
else var xy = {x: selector, y: y}; // get coordinates accordingly
chrome.mouse.action('mouseMoved',xy.x,xy.y,'none',0);
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',2); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',2);};

I tested with after re-opening a new cmd prompt (else it will not reload the new function in TagUI files)
code 2 works! Awesome work @kensoh
//#78 (comment)
// second chunk of code works
chrome.mouse.doubleclick = function(selector,y) { // double press and release on center of selector or point
if (!y) {chrome.scrollIntoViewIfNeeded(selector); var xy = chrome.mouse.getXY(selector);}
else var xy = {x: selector, y: y}; // get coordinates accordingly
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);};

[kensoh]

Thanks Lawrence! I see.. Do you have other websites where you need to use double click?

By right, this change should be made so future releases of TagUI will behave this way. But I'm cautious to change this because, it may mean whatever has worked for other users so far may suddenly break when this change is made. I think this approach is inherited from Puppeteer, I was using that a lot for reference when improving the clicks action for TagUI.

[kensoh]

In the meantime, can you create a tagui_global.js and put it in tagui/src folder? Just put below inside. This will override TagUI default behaviour with global custom functions and applies to all workflows on your laptop. If you put a tagui_local.js in your flow folder, it will apply the custom JS functions to only workflows in that folder.

chrome.mouse.doubleclick = function(selector,y) { // double press and release on center of selector or point
if (!y) {chrome.scrollIntoViewIfNeeded(selector); var xy = chrome.mouse.getXY(selector);}
else var xy = {x: selector, y: y}; // get coordinates accordingly
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);
chrome.mouse.action('mousePressed',xy.x,xy.y,'left',1); chrome.mouse.action('mouseReleased',xy.x,xy.y,'left',1);};

[ativis]

Hi @kensoh, thanks for this wonderful resource! I am facing the similar issues with double clicking using the visual image recognition. I tried to open up the tagui/src/tagui_header.js folder but I face this popup. Any suggestions to implement a successful dclick?

[kensoh]

I assume that you want to edit tagui_header.js manually. You will need an editor to do that (for eg Notepad++, VS Code, Notepad etc). Double-clicking will let Windows try to run the .js file, which is not what you want. CC @ruthtxh

[ruthtxh]

Hey @ativis perhaps you can right click the file then click Edit, to open it in an editor.