VisualWebArena errors after multiple restart #274
Description
4x : Exception uncaught by agent or environment in task <task_name>.
KeyboardInterrupt:
Early termination?
- visualwebarena.317 seed: 31
- visualwebarena.317 seed: 31
- visualwebarena.319 seed: 0
- visualwebarena.319 seed: 0
Showing Max 3 stack traces:
2024-11-21 19:08:38,102 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:10:11,279 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:11:35,868 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:13:03,259 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:14:27,317 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:15:53,792 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:17:17,529 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:18:42,602 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:20:09,985 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:21:43,526 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:23:08,351 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:24:32,692 - 831364 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2x : Exception uncaught by agent or environment in task <task_name>.
TimeoutError:
Timeout 10000ms exceeded.
========================
- visualwebarena.305 seed: 18
- visualwebarena.305 seed: 18
Showing Max 3 stack traces:
2024-11-21 17:03:57,885 - 763432 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.305.
TimeoutError:
Timeout 10000ms exceeded.
=========================== logs ===========================
waiting for locator(":scope")
============================================================
Traceback (most recent call last):
File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 276, in run
step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 411, in from_step
self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
observation, reward, terminated, truncated, info = self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
return self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 442, in step
obs = self._get_obs()
^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 559, in _get_obs
raise e
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 538, in _get_obs
focused_element_bid = extract_focused_element_bid(self.page)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/observation.py", line 564, in extract_focused_element_bid
focused_bid = focused_element.get_attribute(BID_ATTR)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 1795, in get_attribute
self._sync(self._impl_obj.get_attribute(name=name))
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
return task.result()
^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_element_handle.py", line 67, in get_attribute
return await self._channel.send("getAttribute", dict(name=name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
return await self._connection.wrap_api_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
return await cb()
^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
result = next(iter(done)).result()
^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 10000ms exceeded.
=========================== logs ===========================
waiting for locator(":scope")
============================================================
2024-11-21 19:08:38,153 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:08:51,634 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:09:04,234 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:09:17,198 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:09:29,595 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:09:43,249 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:09:58,817 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:10:12,877 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:10:24,868 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
2024-11-21 19:10:39,890 - 831361 - root - WARNING - The content of the message has images, which are not displayed in the string representation.
...
...truncated middle of the log
...
step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 411, in from_step
self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
observation, reward, terminated, truncated, info = self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
return self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 433, in step
reward, done, user_message, task_info = self._task_validate()
^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 458, in _task_validate
reward, done, user_message, info = self.task.validate(self.page, self.chat.messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 283, in validate
score = self.evaluator(
^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 636, in __call__
cur_score = evaluator(trajectory, config_file, page)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<@beartype(visualwebarena.evaluation_harness.evaluators.HTMLContentExactEvaluator.__call__) at 0x7fd0005cc220>", line 92, in __call__
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 371, in __call__
page.goto(target_url)
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 9303, in goto
self._sync(
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
return task.result()
^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_page.py", line 473, in goto
return await self._main_frame.goto(**locals_to_params(locals()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 138, in goto
await self._channel.send("goto", locals_to_params(locals()))
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
return await self._connection.wrap_api_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
return await cb()
^^^^^^^^^^
File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
result = next(iter(done)).result()
^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 10000ms exceeded.
=========================== logs ===========================
navigating to "http://visualwebarenafast.eastus.cloudapp.azure.com:8080/f/memes/127908", waiting until "load"
============================================================