-
Notifications
You must be signed in to change notification settings - Fork 663
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
snap() captures whole page instead of element - can't replicate in Mac/Ubuntu #513
Comments
Hi @yucongshub thanks for raising this! The backend API only returns what Chrome returns. So my initial hunch is there might be some change or bug in Chrome in Linux that could be the root cause. And this is the first report of such issue. I've tried replicating on Mac and it is also working fine -
I have a Linux VPS but it is an old CentOS so I can't run Chrome there. Can you share with me the log file after running the snap() step from your Linux machine? I think you can directly attach the file here. It can be found by That is the transaction log between the TagUI engine and Chrome browser through websocket connection. So it contains the exact and entire low-level information returned from Chrome. |
Adding on @yucongshub - I've tried using the Google Colab notebook example (which runs on Ubuntu) and it appears to be working correctly. The notebook example link is the following and found on README.md page of this repo - https://colab.research.google.com/drive/1or8DtXZP8ZxJYK52me0dA6O9A1dXKKOE?usp=sharing Using the following lines, I have downloaded the log file and attaching it below for comparison later. log_file = r.tagui_location() + '/.tagui/src/tagui_chrome.log'
from google.colab import files; files.download(log_file) |
Hi @kensoh , I tested again and provided the previous and this time log files. I saw that the difference between the log files you provided is that there are no pixel coordinates in tagui_chrome.log.2024.1.15.log |
@kensoh I did some tests, using the coordinates returned in the tagui log, I used the following code to screenshot and it works fine on both Windows and Centos.
How to continue troubleshooting this problem? |
Hi @yucongshub can you tell me more what do you mean? Can you share more on what is the difference between on your Linux and the example using Google Colab Ubuntu? Is it your Linux doesn't return coordinates and the Google Colab Ubuntu returns? If on your Linux there is no coordinates returned, then you can't use workaround to capture the screenshot. Unless you use another system to first collect the coordinates and hard code the solution on your Linux system. Otherwise directly on Linux you don't have those coordinates to use the workaround. What is your Linux and version? I suspect it might be some Linux/Chrome edge case issue that is hard to replicate on other Linux/Chrome. The solution might be to use the one you shared above, collect the coordinates on another machine and use them to do the screen shot on your Linux/Chrome. |
Hi @kensoh , The Linux operating environment I use is centos7.9. I tested it in Windows, google colab, and centos7.9. Windows and google colab returned the correct element screenshots, but centos7.9 was wrong. Its coordinates and screenshot codes The execution order seems different from windows and google colab. How can I debug the order of obtaining element coordinates and taking screenshots? I want to compare and see how windows and centos7.9 run. I have provided the log files of the three, you can check from the logs. |
Hi @kensoh , I recently changed my laptop system to ubuntu 24.04, I tested this problem again and it also recurred. I checked the logs and found something strange. The base64 encoding of the image is first output in the log, and then the coordinates are printed. This order is opposite to the windows and google-colab I provided before, and is the same as centos7 that also has the same problem. Is this the cause of this problem? ? In addition, my centos7 and ubuntu24 have very few programs installed, and it is relatively clean, so I don’t think this is an accident, related to a specific third-party program or something like that. |
Hi @yucongshub I think you have narrowed down and found the issue. I suspect the most likely reason is somehow certain Linux version, the way the websocket calls are made are somehow not in order. Below is the code which explicitly makes a call to get the bounding rectangle ( Is this still an issue for you? I can't explain above and can't replicate on my macOS. I'm trying to see if there is another way to getting to the outcome you want. This seems like some low-level OS-level middleware-level issue.
|
If your goal is to capture the password maybe you can move the mouse using visual automation with r.hover() to the textbox on the left of the generated 4-digit number. Then add x,y offset to capture that spot containing the number using r.snap() which can take x,y coordinates. |
Hi @kensoh , thank you for your reply and solution. Because we plan to run in headless mode, we are currently using the method January 15 to temporarily solve the problem. However, we have encountered other issues during its implementation:
Nevertheless, in Figure 2, when the mouse is positioned slightly to the left of the center point, between two lines of text, it can click successfully. The method to adjust the coordinate point offset is unknown. As a temporary solution, I adjusted the Chrome startup resolution to 1920*1080, resulting in the text being displayed in a single line, as depicted in Figure 3. However, it remains uncertain if longer text that spans two lines would be unclickable under the 1920*1080 resolution. Is there a more effective approach to address this issue? |
Hi @yucongshub,
|
(closing issue for now, but will look out for your reply if any) |
Hi! I am using RPA to capture the verification code image. It has always been normal when running on Windows. However, when I migrated the code to Linux for execution, I found that it did not capture the verification code image, but the entire page. Below is the test situation, Xpath matches it correctly, and I use r.click() it refreshes a new verification code, but if I use r.snap(), it does not screenshot the element, but the entire page.
![P)XC_JEAP 4E52AJ SZ4K6C](https://private-user-images.githubusercontent.com/62985119/295864110-8ecc8f5f-0980-43f5-9512-6c9892ecf2c9.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU5Mzc4ODgsIm5iZiI6MTcyNTkzNzU4OCwicGF0aCI6Ii82Mjk4NTExOS8yOTU4NjQxMTAtOGVjYzhmNWYtMDk4MC00M2Y1LTk1MTItNmM5ODkyZWNmMmM5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEwVDAzMDYyOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFhYzI1YjU0ZjFjZmNiZGQ1MGFmMGJhZTA0Njg3NjU5MzU5MzY5MjBhYTk4ZjQ4ZTNjNGIyODU4M2VlM2Q3NTcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.gnCzXswTZPEzlK-cx4Z0rHEzAmuRI4OK-_ptRk-yCOA)
![image](https://private-user-images.githubusercontent.com/62985119/295864590-4b000c64-dd77-47fc-968d-929648d9d852.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU5Mzc4ODgsIm5iZiI6MTcyNTkzNzU4OCwicGF0aCI6Ii82Mjk4NTExOS8yOTU4NjQ1OTAtNGIwMDBjNjQtZGQ3Ny00N2ZjLTk2OGQtOTI5NjQ4ZDlkODUyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEwVDAzMDYyOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTFjOTQ4MTg5NTA1ZTMwMzgwNjE2OTA1NWFjZTJiM2Y3YjU4MjNmNzcwZGU0YzA2YWE5MTI4NTBkZjE5NzVkN2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.5N9f5glIsEkSIIoxSJFiPhcx1FQ9j4ojJUq8XOaqIuY)
View image 111.png, it is the entire page
My rpa and python versions are:
`
r.version
'1.50.0'
[root@wc-wuh-13-1-25-new zfw]# python3 -V
Python 3.10.10
`
I tried changing the chrome versions to 120, 108 and 73, but the problem still didn't solve. What could be the reason? How can I debug this?
The text was updated successfully, but these errors were encountered: