update `atspi_test` for improving CIs #1330

junkmd · 2023-09-04T08:19:25Z

please see #1329

codecov · 2023-09-04T10:02:59Z

Codecov Report

Merging #1330 (fc675c1) into atspi (bf7f789) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##            atspi    #1330      +/-   ##
==========================================
- Coverage   94.15%   94.14%   -0.01%     
==========================================
  Files          60       60              
  Lines       23036    23034       -2     
==========================================
- Hits        21689    21686       -3     
- Misses       1347     1348       +1

junkmd · 2023-09-05T01:51:24Z

Changing the yml under workflows/ does not trigger the GHA.
Is this due to permissions set on the repository?

Not only this PR, by not running the tests on Linux, the coverage is reduced and codecov results in failure.

If possible, the GHA would be triggered automatically by a PR.

vasily-v-ryabov · 2023-09-07T14:21:16Z

Hmm... All the tasks are in "queued" status. Only CodeQL checks passed. I'm not sure why they are all queued.

vasily-v-ryabov · 2023-09-07T14:22:31Z

Regarding approvals I've sent an invitation to the org with triage access to the repository. Hope CI can run for your PRs automatically after accepting the invitation.

junkmd · 2023-09-07T23:40:36Z

Thank you!

First, I'd like to investigate those problems by checking the GHA configuration and manually re-running CI for other PRs.

junkmd · 2023-09-08T00:06:11Z

I found that Ubuntu 18.04 is already marked as unsupported in GHA.

The Ubuntu 18.04 Actions runner image will begin deprecation on 2022/08/08 and will be fully unsupported by 2023/04/03 actions/runner-images#6002

This might be the reason it's stuck in the 'queued' state.

I will update the atspi_test.yml to use the currently supported Ubuntu versions, which are 20.04 and 22.04.

junkmd · 2023-09-08T04:23:39Z

@vasily-v-ryabov

I've found some problems with CIs, tests, and dependency packages.

Python 2.7 is not supported with `actions/setup-python@>v1`

Log

This is proposed in actions/setup-python#672.

related PRs

Error reports and workarounds have been proposed.

related comments

However, whether or not to introduce them into the CI requires further discussion.

`python-xlib` installation is failed with Ubuntu-22.04

Log

Do we need to update python-xlib?

`pywinauto/unittests/test_application_linux.py` does not terminate whether it passes or fails

Log

I observed tests timing out and failing, so I added a commit to extend the timeout-minutes to 45 minutes. After this change, some jobs passed in less time than the timeout, while others still timed out.

When looking at the jobs that timed out, I noticed that they all had completed up to test_keyboard.py but were not completing in test_application_linux.py.

It's unclear whether the timeout condition in test_application_linux.py is too long or if there is a possibility of it getting stuck in an infinite loop, but I believe some changes in the test code are necessary.

with the latest actions.

junkmd · 2023-09-08T13:31:41Z

Looking at the production codebase, the Linux Application doesn't have as many intricate branches or operations related to timeouts and idles as the Windows Application has.

I think that is the reason tests using assertRaises don't work well.

`start`

pywinauto/pywinauto/linux/application.py

Lines 67 to 80 in bf7f789

    
           def start(self, cmd_line, timeout=None, retry_interval=None, 
        
                     create_new_console=False, wait_for_idle=True, work_dir=None): 
        
               """Start the application as specified by cmd_line""" 
        
               command_line = shlex.split(cmd_line) 
        
               try: 
        
                   process = subprocess.Popen(command_line, shell=create_new_console) 
        
               except Exception as exc: 
        
                   # if it failed for some reason 
        
                   message = ('Could not create the process "%s"\n' 
        
                              'Error returned by CreateProcess: %s') % (cmd_line, str(exc)) 
        
                   raise AppStartError(message) 
        
               self._proc_descriptor = process 
        
               self.process = process.pid 
        
               return self

pywinauto/pywinauto/windows/application.py

Lines 394 to 460 in bf7f789

    
           def start(self, cmd_line, timeout=None, retry_interval=None, 
        
                     create_new_console=False, wait_for_idle=True, work_dir=None): 
        
               """Start the application as specified by cmd_line""" 
        
               # try to parse executable name and check it has correct bitness 
        
               if '.exe' in cmd_line and self.backend.name == 'win32': 
        
                   exe_name = cmd_line.split('.exe')[0] + '.exe' 
        
                   _warn_incorrect_binary_bitness(exe_name) 
        
               if timeout is None: 
        
                   timeout = Timings.app_start_timeout 
        
               if retry_interval is None: 
        
                   retry_interval = Timings.app_start_retry 
        
               start_info = win32process.STARTUPINFO() 
        
               # we need to wrap the command line as it can be modified 
        
               # by the function 
        
               command_line = cmd_line 
        
               # Actually create the process 
        
               dw_creation_flags = 0 
        
               if create_new_console: 
        
                   dw_creation_flags = win32con.CREATE_NEW_CONSOLE 
        
               try: 
        
                   (h_process, _, dw_process_id, _) = win32process.CreateProcess( 
        
                       None, 					# module name 
        
                       command_line,			# command line 
        
                       None, 					# Process handle not inheritable. 
        
                       None, 					# Thread handle not inheritable. 
        
                       0, 						# Set handle inheritance to FALSE. 
        
                       dw_creation_flags,		# Creation flags. 
        
                       None, 					# Use parent's environment block. 
        
                       work_dir,				# If None - use parent's starting directory. 
        
                       start_info)				# STARTUPINFO structure. 
        
               except Exception as exc: 
        
                   # if it failed for some reason 
        
                   message = ('Could not create the process "%s"\n' 
        
                              'Error returned by CreateProcess: %s') % (cmd_line, str(exc)) 
        
                   raise AppStartError(message) 
        
               self.process = dw_process_id 
        
               if self.backend.name == 'win32': 
        
                   self.__warn_incorrect_bitness() 
        
               def app_idle(): 
        
                   """Return true when the application is ready to start""" 
        
                   result = win32event.WaitForInputIdle( 
        
                       h_process, int(timeout * 1000)) 
        
                   # wait completed successfully 
        
                   if result == 0: 
        
                       return True 
        
                   # the wait returned because it timed out 
        
                   if result == win32con.WAIT_TIMEOUT: 
        
                       return False 
        
                   return bool(self.windows()) 
        
               # Wait until the application is ready after starting it 
        
               if wait_for_idle and not app_idle(): 
        
                   warnings.warn('Application is not loaded correctly (WaitForInputIdle failed)', RuntimeWarning) 
        
               self.actions.log("Started " + cmd_line + " application.") 
        
               return self

`connect`

pywinauto/pywinauto/linux/application.py

Lines 82 to 117 in bf7f789

    
               def connect(self, **kwargs): 
        
                   """Connect to an already running process 
        
                   The action is performed according to only one of parameters 
        
                   :param pid: a process ID of the target 
        
                   :param path: a path used to launch the target 
        
                   .. seealso:: 
        
                      :func:`pywinauto.findwindows.find_elements` - the keyword arguments that 
        
                      are also can be used instead of **pid** or **path** 
        
                   """ 
        
                   connected = False 
        
                   if 'pid' in kwargs: 
        
                       self.process = kwargs['pid'] 
        
                       assert_valid_process(self.process) 
        
                       connected = True 
        
                   elif 'path' in kwargs: 
        
                       for proc_id in os.listdir('/proc'): 
        
                           if proc_id == 'curproc': 
        
                               continue 
        
                           try: 
        
                               with open('/proc/{}/cmdline'.format(proc_id), mode='rb') as fd: 
        
                                   content = fd.read().decode().split('\x00') 
        
                           except IOError: 
        
                               continue 
        
                           if kwargs['path'] in " ".join(content): 
        
                               self.process = int(proc_id) 
        
                               connected = True 
        
                               break 
        
                   if not connected: 
        
                       raise RuntimeError( 
        
                           "You must specify process or handle")

pywinauto/pywinauto/windows/application.py

Lines 299 to 392 in bf7f789

    
               def connect(self, **kwargs): 
        
                   """Connect to an already running process 
        
                   The action is performed according to only one of parameters 
        
                   :param pid: a process ID of the target 
        
                   :param handle: a window handle of the target 
        
                   :param path: a path used to launch the target 
        
                   :param timeout: a timeout for process start (relevant if path is specified) 
        
                   .. seealso:: 
        
                      :func:`pywinauto.findwindows.find_elements` - the keyword arguments that 
        
                      are also can be used instead of **pid**, **handle** or **path** 
        
                   """ 
        
                   timeout = Timings.app_connect_timeout 
        
                   retry_interval = Timings.app_connect_retry 
        
                   if 'timeout' in kwargs and kwargs['timeout'] is not None: 
        
                       timeout = kwargs['timeout'] 
        
                   if 'retry_interval' in kwargs and kwargs['retry_interval'] is not None: 
        
                       retry_interval = kwargs['retry_interval'] 
        
                   connected = False 
        
                   if 'pid' in kwargs: 
        
                       self.process = kwargs['pid'] 
        
                       try: 
        
                           wait_until(timeout, retry_interval, self.is_process_running, value=True) 
        
                       except TimeoutError: 
        
                           raise ProcessNotFoundError('Process with PID={} not found!'.format(self.process)) 
        
                       connected = True 
        
                   elif 'process' in kwargs: 
        
                       warnings.warn("'process' keyword is deprecated, use 'pid' instead", DeprecationWarning) 
        
                       kwargs['pid'] = self.process = kwargs['process'] 
        
                       del kwargs['process'] 
        
                       try: 
        
                           wait_until(timeout, retry_interval, self.is_process_running, value=True) 
        
                       except TimeoutError: 
        
                           raise ProcessNotFoundError('Process with PID={} not found!'.format(self.process)) 
        
                       connected = True 
        
                   elif 'handle' in kwargs: 
        
                       if not handleprops.iswindow(kwargs['handle']): 
        
                           message = "Invalid handle 0x%x passed to connect()" % ( 
        
                               kwargs['handle']) 
        
                           raise RuntimeError(message) 
        
                       self.process = handleprops.processid(kwargs['handle']) 
        
                       connected = True 
        
                   elif 'path' in kwargs: 
        
                       try: 
        
                           self.process = wait_until_passes( 
        
                                   timeout, retry_interval, process_from_module, 
        
                                   ProcessNotFoundError, kwargs['path'], 
        
                               ) 
        
                       except TimeoutError: 
        
                           raise ProcessNotFoundError('Process "{}" not found!'.format(kwargs['path'])) 
        
                       connected = True 
        
                   elif kwargs: 
        
                       kwargs['backend'] = self.backend.name 
        
                       # XXX: vryabov 
        
                       # if 'found_index' not in kwargs: 
        
                       #     kwargs['found_index'] = 0 
        
                       #if 'visible_only' not in kwargs: 
        
                       #    kwargs['visible_only'] = False 
        
                       if 'timeout' in kwargs: 
        
                           del kwargs['timeout'] 
        
                           self.process = wait_until_passes( 
        
                                   timeout, retry_interval, findwindows.find_element, 
        
                                   exceptions=(findwindows.ElementNotFoundError, findbestmatch.MatchError, 
        
                                               controls.InvalidWindowHandle, controls.InvalidElement), 
        
                                   *(), **kwargs 
        
                               ).process_id 
        
                       else: 
        
                           self.process = findwindows.find_element(**kwargs).process_id 
        
                       connected = True 
        
                   if not connected: 
        
                       raise RuntimeError( 
        
                           "You must specify some of process, handle, path or window search criteria.") 
        
                   if self.backend.name == 'win32': 
        
                       self.__warn_incorrect_bitness() 
        
                       if not handleprops.has_enough_privileges(self.process): 
        
                           warning_text = "Python process has no rights to make changes " \ 
        
                               "in the target GUI (run the script as Administrator)" 
        
                           warnings.warn(warning_text, UserWarning) 
        
                   return self

To stabilize the CI, we need changes not only in the tests but also in the production codebase.
I'm not well-versed in Linux, so I hope someone else with Linux expertise can come up with a good implementation.

In the scope of this PR, I plan to make changes only to atspi_test.yml.

junkmd · 2023-09-08T23:06:46Z

I specified python:2.7.18-buster for container, but the Install dependencies isn't working properly.

https://github.com/pywinauto/pywinauto/actions/runs/6123447605/workflow

Install dependencies failure log

Run python -m pip install --upgrade pip
  python -m pip install --upgrade pip
  sudo apt update -y
  sudo apt upgrade
  sudo apt install -y --no-install-recommends python-setuptools tk-dev python-tk
  sudo apt install -y --no-install-recommends python3 python3-pip python3-setuptools
  sudo apt install -y --no-install-recommends python-xlib
  sudo apt install -y --no-install-recommends xsel
  sudo apt install -y libjavascriptcoregtk-4.0-18
  pip install coverage codecov
  pip install python-xlib --upgrade
  pip install mock --upgrade
  python -m pip install pytest
  sudo apt install -y build-essential libssl-dev
  sudo apt install -y qt5-default
  sudo apt install -y libxkbcommon-x11-0
  sudo apt install -y python3-gi gobject-introspection gir1.[2](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:2)-gtk-[3](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:3).0
  sudo apt install -y at-spi2-core
  sudo apt install -y python3-pyatspi
  sudo apt install -y xvfb x11-utils libxkbcommon-x11-0 \
      libxcb-icccm[4](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:4) libxcb-image0 libxcb-keysyms1 libxcb-randr0 \
      libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 xdotool
  #pip3 install scikit-build # required for cmake>=3.23.0
  pip3 install cmake==3.22.6
  shell: sh -e {0}
  env:
    DISPLAY: :0
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
  Downloading pip-20.3.4-py2.py3-none-any.whl (1.[5](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:5) MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-20.3.4
/__w/_temp/e840ddf7-74b5-4ca3-8a4[6](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:6)-3de336bec61d.sh: 2: /__w/_temp/e840ddf[7](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:7)-74b5-4ca3-[8](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:8)a46-3de336bec61d.sh: sudo: not found
Error: Process completed with exit code [12](https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664#step:5:12)7.

https://github.com/pywinauto/pywinauto/actions/runs/6123447605/job/16621351664

The cost of continuing to support Python 2.7 has increased compared to a few years ago.

Any opinions would be appreciated.

- add `Install sudo` - Remove redundant `with/without sudo` branches

junkmd · 2023-09-10T08:17:43Z

I have been considering a way to support Python 2.7 in an Ubuntu environment using containers and committed experimental changes for this purpose. However, the CI has failed.

Here is a part of the message when using the container:

debconf: unable to initialize frontend: Dialog

https://github.com/pywinauto/pywinauto/actions/runs/6135067536/job/16648479694?pr=1330#step:9:2653

I've realized that containers were not intended for working with dialog interactions, so they are unsuitable for the GUI tests for this package.

junkmd · 2023-09-10T08:35:29Z

I feel that there isn't much benefit in supporting backend=atspi in Python 2.7.

The last release version(https://github.com/pywinauto/pywinauto/tree/0.6.8) does not include backend=atspi (, AtspiWrapper and AtspiElementInfo).
Python 2.7 reached EOL on January 1, 2020, and it has been more than three years since then.

Therefore, nowadays, I think there are very few developers eagerly anticipating support for backend=atspi in Python 2.7.

junkmd · 2023-09-12T02:35:01Z

@vasily-v-ryabov

Hope CI can run for your PRs automatically after accepting the invitation.

I confirmed that submitting a new PR or temporarily closing and reopening a previous PR causes the GHA to run.

However, with my current permissions, I cannot execute "Cancel workflow", "Re-run failed jobs", or "Re-run all jobs".

As mentioned earlier, tests for Application(backend="atspi") are unstable and sometimes do not terminate.

Without "Cancel workflow", I cannot cancel unterminated tests, resulting in a significant amount of time spent running tests for all conditions in the matrix.
Additionally, without "Re-run failed jobs", I cannot re-run those timed-out tests.

junkmd · 2023-09-13T06:15:08Z

container: python:2.7.18-buster(bf7f789...d5dc55b) did not work.

So I removed Python 2.7 from matrix(bf7f789...fc675c1).
This has passed tests for at least one Python version.

update atspi_test for improving CIs

20311fe

junkmd closed this Sep 4, 2023

junkmd reopened this Sep 4, 2023

junkmd added 2 commits September 8, 2023 00:42

remove unsupported ubuntu-18.04

4d330ed

extend timeout-minutes

66092c6

junkmd added 2 commits September 8, 2023 04:30

Experimental: removing configs that didn't work

a31afe0

with the latest actions.

specify the container for py2.7

5a7426e

junkmd closed this Sep 8, 2023

junkmd reopened this Sep 8, 2023

junkmd closed this Sep 8, 2023

junkmd added 2 commits September 8, 2023 08:44

update GHA configs

2c08e46

add comments

6865810

junkmd reopened this Sep 8, 2023

junkmd and others added 6 commits September 8, 2023 08:54

fix include

cc98a33

fix include

149ef7f

add adjusting the Timings

d7242f7

remove comments

d602b8f

add container

568df86

remove Timings from test_application_linux.py

13ae33b

junkmd added 3 commits September 9, 2023 00:13

add a conditional branch

5f64e68

revert checkout and setup-python versions

4d05f70

update matrix.os

8002b9f

junkmd added 2 commits September 9, 2023 07:34

update checkout and setup-python versions

b385fb9

revert container

0eb5480

junkmd added 3 commits September 9, 2023 12:20

add conditional branches

7ff55cd

add Remote display start with/witout sudo

493e906

some improvements;

d5dc55b

- add `Install sudo` - Remove redundant `with/without sudo` branches

remove Python 2.7 container settings

fc675c1

junkmd requested a review from vasily-v-ryabov September 13, 2023 06:20

junkmd added the atspi label Sep 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update `atspi_test` for improving CIs #1330

update `atspi_test` for improving CIs #1330

junkmd commented Sep 4, 2023

codecov bot commented Sep 4, 2023 •

edited

junkmd commented Sep 5, 2023

vasily-v-ryabov commented Sep 7, 2023

vasily-v-ryabov commented Sep 7, 2023

junkmd commented Sep 7, 2023

junkmd commented Sep 8, 2023

junkmd commented Sep 8, 2023

junkmd commented Sep 8, 2023 •

edited

junkmd commented Sep 8, 2023 •

edited

junkmd commented Sep 10, 2023

junkmd commented Sep 10, 2023

junkmd commented Sep 12, 2023

junkmd commented Sep 13, 2023

update atspi_test for improving CIs #1330

Are you sure you want to change the base?

update atspi_test for improving CIs #1330

Conversation

junkmd commented Sep 4, 2023

codecov bot commented Sep 4, 2023 • edited

Codecov Report

junkmd commented Sep 5, 2023

vasily-v-ryabov commented Sep 7, 2023

vasily-v-ryabov commented Sep 7, 2023

junkmd commented Sep 7, 2023

junkmd commented Sep 8, 2023

junkmd commented Sep 8, 2023

Python 2.7 is not supported with actions/setup-python@>v1

python-xlib installation is failed with Ubuntu-22.04

pywinauto/unittests/test_application_linux.py does not terminate whether it passes or fails

junkmd commented Sep 8, 2023 • edited

start

connect

junkmd commented Sep 8, 2023 • edited

junkmd commented Sep 10, 2023

junkmd commented Sep 10, 2023

junkmd commented Sep 12, 2023

junkmd commented Sep 13, 2023

update `atspi_test` for improving CIs #1330

update `atspi_test` for improving CIs #1330

codecov bot commented Sep 4, 2023 •

edited

Python 2.7 is not supported with `actions/setup-python@>v1`

`python-xlib` installation is failed with Ubuntu-22.04

`pywinauto/unittests/test_application_linux.py` does not terminate whether it passes or fails

junkmd commented Sep 8, 2023 •

edited

`start`

`connect`

junkmd commented Sep 8, 2023 •

edited