Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep polling after computer went to sleep #561

Open
EmileSonneveld opened this issue Apr 26, 2024 · 0 comments
Open

Keep polling after computer went to sleep #561

EmileSonneveld opened this issue Apr 26, 2024 · 0 comments

Comments

@EmileSonneveld
Copy link
Contributor

When running a batch job and the computer goes to sleep, polling could stop in an error.
It would be handy if after an 110 error, polling resumes. Especially handy for overnight jobs.

...
0:16:36 Job 'vito-j-240425f55e6940e3bb0c37fa61153068': running (progress N/A)
0:17:38 Job 'vito-j-240425f55e6940e3bb0c37fa61153068': running (progress N/A)
0:18:40 Job 'vito-j-240425f55e6940e3bb0c37fa61153068': running (progress N/A)
0:19:40 Job 'vito-j-240425f55e6940e3bb0c37fa61153068': running (progress N/A)
custom_execute_batch end time: 2024-04-26 10:28:48.428607
Traceback (most recent call last):
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 462, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1132, in read
    return self._sslobj.read(len, buffer)
TimeoutError: [Errno 110] Connection timed out
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 469, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 375, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='openeocloud.vito.be', port=443): Read timed out. (read timeout=1200)
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "/home/emile/openeo/VITO/VITO2024/resample_spatial_high_res.py", line 47, in <module>
    ERA5_shift_dc = ERA5_shift_dc.resample_spatial(resolution=resolution, projection=4326)  # comment out to avoid OOM
  File "/home/emile/openeo/VITO/openeo_utils.py", line 187, in custom_execute_batch
    job.start_and_wait()
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/job.py", line 292, in start_and_wait
    job_info = self.describe()
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/job.py", line 74, in describe
    return self.connection.get(f"/jobs/{self.job_id}", expected_status=200).json()
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/connection.py", line 221, in get
    return self.request("get", path=path, stream=stream, auth=auth, **kwargs)
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/connection.py", line 770, in request
    return _request()
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/connection.py", line 763, in _request
    return super(Connection, self).request(
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/openeo/rest/connection.py", line 148, in request
    resp = self.session.request(
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/emile/openeo/venv/lib/python3.8/site-packages/requests/adapters.py", line 532, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='openeocloud.vito.be', port=443): Read timed out. (read timeout=1200)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant