MVP of ACS plus initial port of CLI to new ACS abstractions by nickjalbert · Pull Request #91 · agentos-project/agentos

nickjalbert · 2021-02-13T08:22:34Z

Initial (rough) port of agentos {init, train, test} command-line tools to the new abstractions. Issuing this PR in case you all want to start writing code against these changes.

See design discussion here.

Demo

# Create new agent dir
mkdir andy_agent
cd andy_agent/

# Initialize your agent
agentos init . -n AndyAgent

# Train your agent in a loop for 5 iterations
agentos learn 5

# Run your agent against default corridor environment
agentos run

# Install 2048 as your environment
agentos install 2048

# Run your agent against 2048
agentos run

Limitations

This is a very rough first stab, some limitations:

~~I assume you will be running these commands within the agent directory (and don't support specifying custom agent.ini yet)~~ train and run now support a -f flag to specify the agent.ini to use.
I skipped all but ~~one~~ two of the tests (😬). They'll come back online as we port example agents and run_agent to the new abstractions.
~~I added agentos test (instead of modifying agentos run)~~
General API instability

Next step

~~I'd like to port run_agent (and the dependent tests) to the new abstractions. Thoughts?~~
Reinstate the skipped tests
Porting over agents to use the new abstractions
Making a plan for action_space and observation_space in Policy

- agentos {init, train, test} - agent.ini file - functional basic agent

nickjalbert · 2021-02-13T23:38:05Z

Updated this PR to remove the Trainer abstraction.

nickjalbert · 2021-02-14T02:39:11Z

Made some more progress on porting the core and CLI to the new abstractions.

andyk · 2021-02-14T16:54:59Z

Can you link to the design discussion thread in the first comment of this PR conversation?

nickjalbert · 2021-02-17T21:51:46Z

Added a first cut at agentos install. Demo has been updated in this PR description.

andyk

This looks like a good first step towards a 0.0.8. Do we want to cut 0.0.7 before we merge this into master?

andyk · 2021-02-14T16:33:15Z

@@ -1,16 +1,31 @@
 """Implementation of agents available in AgentOS. See core.py for Agent API."""


Should we get rid of the agents module entirely and instead have a them available via the registry?

Yeah, this makes sense. Maybe we punt on it for now because the tests use this at the moment?

andyk · 2021-02-15T20:29:52Z

+        raise NotImplementedError
+
+
+def run_agent(agent, hz=40, max_iters=None, as_thread=False):


Maybe we should write down some user stories for the agentos.run_agent() helper function now that we have shifted towards an agent being defined by an agent.ini?

I think we talked about this before, but I forget where we came down on it: are we thinking we'll keep agentos.run_agent()?

Either way seems reasonable to me. An initial stab at a user story:

Agent developer incorporating RL into their tech stack
As a developer incorporating RL into my tech stack, I need to run experiments with different agent configurations to determine what flavor of RL is best suited to the business problem my work is addressing. Being able to programmatically interact with agents allows me to run these experiments as well as train and deploy my agent in a reproducible way.

Yeah, I like sticking to a guiding philosophy where anything you can do from the CLI you can also do via Python API and vice versa.

Good user story, should we add it to the design doc w/ the others?

Yeah, I like sticking to a guiding philosophy where anything you can do from the CLI you can also do via Python API and vice versa.

SGTM!

I'll add it to the design doc.

port some example agents, fix bug in core

nickjalbert · 2021-02-19T23:56:11Z

@andyk

Do we want to cut 0.0.7 before we merge this into master?

Just checking that I understand this proposal:

Update master so we're on version 0.0.7 (instead of 0.0.7-alpha)
Merge this PR
Call master 0.0.8-alpha (or whatever is after 0.0.7)

is that right? If so, sounds good to me!

andyk · 2021-02-20T00:42:54Z

@andyk

Do we want to cut 0.0.7 before we merge this into master?

Just checking that I understand this proposal:
* Update master so we're on version 0.0.7 (instead of 0.0.7-alpha)

* Merge this PR

* Call master 0.0.8-alpha (or whatever is after 0.0.7)
is that right? If so, sounds good to me!

Yeah, this is what I was thinking with one more step before these that you listed being: @trush to review and [if/when it's ready] merge #93

andyk · 2021-02-20T00:45:18Z

Call master 0.0.8-alpha (or whatever is after 0.0.7)

I propose we update the version to 0.1.0-alpha because it will change core abstractions quite a bit

nickjalbert · 2021-02-20T00:57:17Z

@andyk branch and versioning plan sounds good to me!

nickjalbert · 2021-02-23T01:11:02Z

97014da adds the ability to install 2048 via the agentos CLI. Demo updated.

andyk · 2021-02-23T17:49:29Z

@andyk branch and versioning plan sounds good to me!

Moving ahead with this plan in #102

Initial port of CLI to new abstractions

0d4c7ce

- agentos {init, train, test} - agent.ini file - functional basic agent

nickjalbert requested review from andyk and trush February 13, 2021 08:22

nickjalbert added 2 commits February 13, 2021 15:32

Fold Trainer into Policy.improve

0e263c2

Fix the CLI test

9e1329c

nickjalbert added 2 commits February 13, 2021 18:37

Start port of run_agent, CLI cleanup

13bd996

Unskip one test

87bef54

andyk and others added 5 commits February 15, 2021 15:51

port two rl agents, fix bug in core

2ea36cb

make random_nn_policy a bit more general

4ded67f

porting reinforce example

fcf343c

port free energy example agent

92db00b

Initial verision of CLI "install" command

b84660d

andyk reviewed Feb 19, 2021

View reviewed changes

nickjalbert added 2 commits February 19, 2021 15:31

Merge pull request #3 from andyk/0.1-dev

8f7b277

port some example agents, fix bug in core

Fix lint complaints

ec11e75

Components: add 2048 to the registry

97014da

andyk closed this Jun 14, 2021

andyk reopened this Jun 14, 2021

andyk merged commit 288e868 into agentos-project:master Jun 14, 2021

andyk changed the title ~~Initial port of CLI to new abstractions~~ MVP of ACS plus initial port of CLI to new ACS abstractions Sep 14, 2021

		@@ -1,16 +1,31 @@
		"""Implementation of agents available in AgentOS. See core.py for Agent API."""

		raise NotImplementedError


		def run_agent(agent, hz=40, max_iters=None, as_thread=False):

Conversation

nickjalbert commented Feb 13, 2021 • edited by andyk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Demo

Limitations

Next step

Uh oh!

nickjalbert commented Feb 13, 2021

Uh oh!

nickjalbert commented Feb 14, 2021

Uh oh!

andyk commented Feb 14, 2021

Uh oh!

nickjalbert commented Feb 17, 2021

Uh oh!

andyk left a comment

Choose a reason for hiding this comment

Uh oh!

andyk Feb 14, 2021

Choose a reason for hiding this comment

Uh oh!

nickjalbert Feb 19, 2021

Choose a reason for hiding this comment

Uh oh!

andyk Feb 15, 2021

Choose a reason for hiding this comment

Uh oh!

nickjalbert Feb 20, 2021

Choose a reason for hiding this comment

Uh oh!

andyk Feb 20, 2021

Choose a reason for hiding this comment

Uh oh!

nickjalbert Feb 20, 2021

Choose a reason for hiding this comment

Uh oh!

nickjalbert commented Feb 19, 2021

Uh oh!

andyk commented Feb 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andyk commented Feb 20, 2021

Uh oh!

nickjalbert commented Feb 20, 2021

Uh oh!

nickjalbert commented Feb 23, 2021

Uh oh!

andyk commented Feb 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nickjalbert commented Feb 13, 2021 •

edited by andyk

Loading

andyk commented Feb 20, 2021 •

edited

Loading