Appium-Python: find_element_by_id intermittently produces 'Connection aborted.' error

[ShortCipher42]

The problem

This problem began 48 hours ago. I have been running mobile application automated tests on these particular devices in order to induce a server load. After a number of days of running it continuously, the following error has begun happening on the devices. The error is highly intermittent and I have not found any way to mitigate the error.

Helpers\AccountBalances.py:225: in CheckDetails
    self.detailsbutton = self.driver.find_element_by_id('productDetailsContainer')
c:\python\lib\site-packages\selenium\webdriver\remote\webdriver.py:360: in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
c:\python\lib\site-packages\appium\webdriver\webdriver.py:276: in find_element
    'value': value})['value']
c:\python\lib\site-packages\selenium\webdriver\remote\webdriver.py:319: in execute
    response = self.command_executor.execute(driver_command, params)
c:\python\lib\site-packages\selenium\webdriver\remote\remote_connection.py:374: in execute
    return self._request(command_info[0], url, body=data)
c:\python\lib\site-packages\selenium\webdriver\remote\remote_connection.py:397: in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
c:\python\lib\site-packages\urllib3\request.py:72: in request
    **urlopen_kw)
c:\python\lib\site-packages\urllib3\request.py:150: in request_encode_body
    return self.urlopen(method, url, **extra_kw)
c:\python\lib\site-packages\urllib3\poolmanager.py:326: in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
c:\python\lib\site-packages\urllib3\connectionpool.py:641: in urlopen
    _stacktrace=sys.exc_info()[2])
c:\python\lib\site-packages\urllib3\util\retry.py:368: in increment
    raise six.reraise(type(error), error, _stacktrace)
c:\python\lib\site-packages\urllib3\packages\six.py:685: in reraise
    raise value.with_traceback(tb)
c:\python\lib\site-packages\urllib3\connectionpool.py:603: in urlopen
    chunked=chunked)
c:\python\lib\site-packages\urllib3\connectionpool.py:387: in _make_request
    six.raise_from(e, None)
<string>:2: in raise_from
    ???
c:\python\lib\site-packages\urllib3\connectionpool.py:383: in _make_request
    httplib_response = conn.getresponse()
c:\python\lib\http\client.py:1336: in getresponse
    response.begin()
c:\python\lib\http\client.py:306: in begin
    version, status, reason = self._read_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <http.client.HTTPResponse object at 0x04378250>

    def _read_status(self):
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
        if len(line) > _MAXLINE:
            raise LineTooLong("status line")
        if self.debuglevel > 0:
            print("reply:", repr(line))
        if not line:
            # Presumably, the server closed the connection before
            # sending a valid response.
>           raise RemoteDisconnected("Remote end closed connection without"
                                     " response")
E           urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

c:\python\lib\http\client.py:275: ProtocolError

Environment

Windows 10
Appium Python client
6 android mobile devices

Desired Capabilities for all devices:

    if(deviceName == 'Galaxy S9+'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '9',
            'deviceName': 'Galaxy S9+',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': '31554941384a3098',
            'systemPort': '8200',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]
    elif(deviceName == 'Galaxy J7 Prime'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '8.1.0',
            'deviceName': 'Galaxy J7 Prime',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': '52008b7a5bf4c4c5',
            'systemPort': '8201',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]
    elif(deviceName == 'SAMSUNG-SM-J727V'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '7.0',
            'deviceName': 'SAMSUNG-SM-J727V',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': '7e09ce03',
            'systemPort': '8202',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]
    elif(deviceName == 'SAMSUNG-SM-G935V'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '7.0',
            'deviceName': 'SAMSUNG-SM-G935V',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': '49880284',
            'systemPort': '8203',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]
    elif(deviceName == 'Pixel 2'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '8.1.0',
            'deviceName': 'Pixel 2',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': 'HT81S1A01567',
            'systemPort': '8204',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]
    elif(deviceName == 'Moto E (4)'):
        desired_caps = ['http://localhost:4723/wd/hub',{
            'platformName': 'Android',
            'platformVersion': '7.1.1',
            'deviceName': 'Moto E (4)',
            'app': PATH('{}'.format(app)),
            'newCommandTimeout': 500,
            'automationName': 'UIAutomator2',
            'noReset': 'false',
            'udid': 'ZY224TLHDF',
            'systemPort': '8206',
            'printPageSourceOnFindFailure': 'true',
            'adbExecTimeout' : '50000'
        }]

Details

I have six python scripts running simultaneously. All devices are utilizing the same appium server and different android devices. All devices will randomly fail on find_element_by_id.

Link to Appium logs

The following gist is Appium logs for a particular session where all devices except the Pixel 2 failed by the above 'Connection Aborted' error.
Gist

Code To Reproduce Issue

This is a highly intermittent issue but seems to be kicked off by find_element_by_id every time.
I do not see any errors in Appium's logging.

[mykola-mokhnach]

To me it looks like an internal networking issue.

[ShortCipher42]

An internal networking issue using localhost?

[ciobania]

I have the same intermittent issue, when testing with multiple devices, through USB. This is a disconnect between the devices, and the host. I don't have my details at hand, now. What I had to do, in order to make some of the devices work together was to downgrade adb, and appium-client libraries to a version release from mid April. The reason for that is because that's the last time I was able to run all those devices, together, for 24hrs, without issues. Lots of things got introduced, such as new capabilities and I think some stuff got taken out, which means there's no backwards compatibility. At the time it was hard to pinpoint exactly what the root cause of it. I hope this helps. Thank you

…

On Thu, 12 Sep 2019, 18:34 ShortCipher42, ***@***.***> wrote: An internal networking issue using localhost? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#437?email_source=notifications&email_token=ABRBXCEEKOU35E5DTK5NQALQJJ4T5A5CNFSM4IWHKD2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6SVDKI#issuecomment-530928041>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABRBXCCD4WCXFENKLVGX4FLQJJ4T5ANCNFSM4IWHKD2A> .

[ki4070ma]

I'm not sure this issue is due to client side.

@ShortCipher42
Could you fill in below template?
And could you have a try #437 (comment) ?

Environment

• Appium version (or git revision) that exhibits the issue:
• Last Appium version that did not exhibit the issue (if applicable):
• Desktop OS/version used to run Appium:
• Node.js version (unless using Appium.app|exe):
• Mobile platform/version under test:
• Real device or emulator/simulator:
• Appium CLI or Appium.app|exe:

I found below log in server logs. (It might be related, but I'm not sure how for now.)

https://gist.github.com/ShortCipher42/a46f5874c9534684fede1492507e2442

• No obsolete sessions have been detected (Error: socket hang up)

[ShortCipher42]

I believe we've found the issue.

The issue appears to be the particular version of the app being fused with Antibot by Appdome and attempts to block debugging attempts. I am pursuing a solution internally.