v0.4.1: VisualWebArena is here! 🎉

browsergym-visualwebarena

• New benchmark available!! #100 Bigs thanks to @ljang0

  import browsergym.visualwebarena
  
  env = gym.make("browsergym/visualwebarena.423")
  ...

  

browsergym-core

• New features
  • Tasks can now return a goal with both a message and images (in the form of URLs)

    class MyTask(AbstractBrowserTask):
      def setup(self, page: playwright.sync_api.Page):
        goal = {
          "message": "Where can I buy this car?",
          "image_urls": ["https://en.wikipedia.org/wiki/Electric_car#/media/File:Nissan_Leaf_2_(45992539055).jpg"]
        }
        info = {}
        return goal, info

  • Observations returned by BrowserEnv (reset(), step()) now have a new field for goal images

    obs, info = env.reset()
    # textual goal as before
    print(obs["goal"])
    # list of goal images (as urls, possibly using data:image/png;base64,... encoding)
    print(obs["goal_image_urls"])

  • Goal images are also sent to the chat, with a new user_image role

browsergym-webarena

• Breaking changes
  • Setup environment variables have been renamed to avoid clash with VisualWebArena #100

    export WA_SHOPPING=...
    export WA_SHOPPING_ADMIN=...
    export WA_REDDIT=...
    export WA_GITLAB=...
    export WA_WIKIPEDIA=...
    export WA_MAP=...
    export WA_HOMEPAGE=...

browsergym-experiments

• New features
  • new class AgentInfo #102
  • new AbstractAgentArgs methods prepare() and close() #106