Generic interface #106

waydegg · 2023-09-23T03:22:32Z

Summary

This PR covers:

updating all blocking code to be async
creating a standard message format between the core library and client libraries
moving all terminal-specific logic and dependencies under mentat/terminal

Things that will be covered in a later PR:

A JSON RPC server with an LSP-compantible API that any editors with language server support can connect to
A SessionManager to coordinate multiple, concurrent chats

To-do

merge with main and resolve merge conflicts
validate cprint colors received from core
check/test for race conditions b/w core and terminal client (specifically w/ inputs and interrupts)
fix test cases
create existing test cases
- core library
- terminal client
fix misc typing errors

granawkins

Generally it looks great - seems to check all the boxes we identified with the concepts and what not. Nicely done 👏

granawkins · 2023-09-27T03:55:34Z

mentat/session_stream.py

+
+logger = logging.getLogger()
+
+_SESSION_STREAM: ContextVar[SessionStream] = ContextVar("mentat:session_stream")


Like I said on the call - I really like this approach to 'instancing'. I think it will make sense to do the same thing with CodeContext eventually.

Yeah we might want to do this with the entire session instance at some point

granawkins · 2023-09-27T04:07:51Z

mentat/session_stream.py

+@dataclass
+class StreamMessage:
+    id: UUID
+    channel: str


Looks like this could be 'default', 'input_request', a UUID, 'input_request:{UUID}' or 'interrupt'. It'd be nice to have these as an enum or something, but I'm not sure the best way to handle the uuids..

granawkins · 2023-09-27T04:40:11Z

mentat/session_input.py

+    # response = await stream.recv(f"default:{message.id}")
+
+    message = await stream.send("", channel="input_request")
+    response = await stream.recv(f"input_request:{message.id}")


I could imagine async tasks in the background could use this approach to. e.g. there could be an EmbeddingProvider listening on "embed_request".

Should there be an optional expire somewhere, maybe in recv, to handle network failures and the like?

Totally. Adding more keys/values to the StreamMessage in the future won't break anything with how it's setup atm

granawkins · 2023-09-27T04:42:05Z

mentat/session_stream.py

+    channel: str
+    source: StreamMessageSource
+    data: Any
+    extra: Dict[str, Any] | None


What if instead of "color" we used sth like debug level - "info", "warning", "error" - which terminal.output can interpret to colors.

biobootloader · 2023-09-27T18:12:29Z

I'm confused about why Github is showing a bunch of changes that have already been merged as part of the diff for this PR. Like my change here for PyPI going from mentat-ai to mentat is showing up in this PR:

Maybe it's due to the conflicts, it's not able to show a diff against main for some reason?

Edit: Is it from merging with other branches prior to them being merged to main? So the changes come from different commits (since we squash when merging with main)? Though I still don't understand

PCSwingle

Really love this! I'll take a closer look once it gets caught up to main, but looks good to me so far! Left a few questions on a couple things I wasn't sure about.

PCSwingle · 2023-09-27T22:51:41Z

mentat/broadcast.py

+
+
+class Event:
+    def __init__(self, channel: str, message: typing.Any) -> None:


nit: Why is this typing.Any instead of Any?

PCSwingle · 2023-09-27T22:55:25Z

mentat/code_change_display.py



-def get_previous_lines(code_change, num=2):
+async def print_previous_lines(code_change: CodeChange, num: int = 2):


Why did we change these to printing functions instead of just returning the string?

PCSwingle · 2023-09-27T22:58:27Z

mentat/session.py

+            logger.debug("Task has stopped")
+
+        self._stop_task = asyncio.create_task(run_stop())
+        self._stop_task.add_done_callback(cleanup_stop)


Is there a reason we need to have a done_callback instead of just doing this at the end of cleanup_stop?

jakethekoenig · 2023-09-28T16:50:12Z

I added ipdb to requirements.txt

jakethekoenig · 2023-09-28T17:07:42Z

There was a git merge conflict token in the logging_config I removed. It's now mostly working for me. I notice a few things though:

ctrl-c while the model is streaming closes mentat completely instead of returning to prompt.
Colors aren't working for me in streamed output removal. e.g. see the following conversation with coloring symbols instead of colors:

(.venv) Development/mentat % mentat README.md
Files included in context:
mentat
└── README.md

Including CodeMap (filepaths only)
File and prompt token count: 5961 / 8192
Type 'q' or use Ctrl-C to quit at any time.
What can I do for you?
>>> Please delete the fifth through seventh lines of the readme

Including CodeMap (filepaths only)
Total token count: 5971
Streaming(.venv) Development/mentat % mentat README.md
Files included in context:
mentat
└── README.md

Including CodeMap (filepaths only)
File and prompt token count: 5961 / 8192
Type 'q' or use Ctrl-C to quit at any time.
What can I do for you?
>>> Please delete the fifth through seventh lines of the readme

Including CodeMap (filepaths only)
Total token count: 5971
Streaming.e control-c to interrupt the model atu any point
I will delete lines 5 through 7 in the README.md file.

Steps:
1. Delete lines 5-7 in README.md

?[94m
README.md?[0m
============================================================
2:  [?[94m![Discord Follow?[39;49;00m](?[96mhttps://dcbadge.vercel.app/api/server/XbPdxAMJte?style=flat?[39;49;00m)](https://discord.gg/zbvd9qx9Pb)?[90m?[39;49;00m
3:  [?[94m![Stable Version?[39;49;00m](?[96mhttps://img.shields.io/pypi/v/mentat?color=blue?[39;49;00m)](https://pypi.org/project/mentat/)?[90m?[39;49;00m
4:  [?[94m![License?[39;49;00m](?[96mhttps://img.shields.io/pypi/l/mentat.svg?[39;49;00m)](https://github.com/AbanteAI/mentat/blob/main/LICENSE)
?[31m-
-   # 🧙‍♂️ Mentat ⚡
-   ?[0m
8:  > _It is by will alone I set my mind in motion_?[90m?[39;49;00m
9:  >
============================================================

Colors do seem to be working for inserts and system messages.
3. When mentat encounters an error e.g. the model doesn't correctly respond with line numbers then mentat starts streaming all the logs to stdout. Not sure if that's the intended behavior.

PCSwingle · 2023-09-28T20:44:19Z

mentat/terminal/client.py

+            await self._main()
+            await self._shutdown()
+        # NOTE: if an exception is caught here, the main process will likely still run
+        # due to background ascynio Tasks that are still running


nit: asyncio

granawkins · 2023-09-28T23:32:24Z

It looks like we lost the call to setup_api_key(), so I added it back in.

ctrl-c while the model is streaming closes mentat completely instead of returning to prompt.

Colors aren't working for me in streamed output removal. e.g. see the following conversation with coloring symbols instead of colors:

When mentat encounters an error e.g. the model doesn't correctly respond with line numbers then mentat starts streaming all the logs to stdout. Not sure if that's the intended behavior.

I'm setting the same errors as @jakethekoenig. I also got some logging to the terminal when running a diff:

granthawkins@Grants-MacBook-Air mentat % mentat --diff HEAD~1                                     
Diff annotations:
 ─•─ HEAD~1: a3063419: Fix logging config git conflict token | 2 files | 1 lines

DEBUG:root:Skipping /Users/granthawkins/mentat/.gitignore. Reason: lexer not found
Files included in context:
mentat
├── * .gitignore
└── mentat
    └── session.py
INFO:root:StreamMessage(id=UUID('71ba8857-3bb6-466f-9411-b3dce0a2d980'), channel='default', source=<StreamMessageSource.SERVER: 'server'>, data='\nIncluding CodeMap (filepaths only)', extra={'color': 'green'}, created_at=datetime.datetime(2023, 9, 28, 23, 31, 2, 618374))
INFO:root:StreamMessage(id=UUID('9651a3bf-e483-48da-8ca7-23c6a84470be'), channel='default', source=<StreamMessageSource.SERVER: 'server'>, data='File and prompt token count: 5979 / 8192', extra={'color': 'cyan'}, created_at=datetime.datetime(2023, 9, 28, 23, 31, 2, 621870))
INFO:root:StreamMessage(id=UUID('470117d2-e6c3-43e5-a705-672a525a62b0'), channel='default', source=<StreamMessageSource.SERVER: 'server'>, data="Type 'q' or use Ctrl-C to quit at any time.", extra={'color': 'cyan'}, created_at=datetime.datetime(2023, 9, 28, 23, 31, 2, 621913))
INFO:root:StreamMessage(id=UUID('d3598fd7-8925-4043-a805-c6fcc927b09d'), channel='default', source=<StreamMessageSource.SERVER: 'server'>, data='What can I do for you?', extra={'color': 'light_blue'}, created_at=datetime.datetime(2023, 9, 28, 23, 31, 2, 621932))
INFO:root:StreamMessage(id=UUID('a3f188bb-aabb-437b-96e2-7e58550f218f'), channel='input_request', source=<StreamMessageSource.SERVER: 'server'>, data='', extra={}, created_at=datetime.datetime(2023, 9, 28, 23, 31, 2, 621953))

Including CodeMap (filepaths only)
File and prompt token count: 5979 / 8192
Type 'q' or use Ctrl-C to quit at any time.
What can I do for you?

waydegg · 2023-09-29T18:26:18Z

Thanks for all of the comments everyone! I'm going to close and re-open this PR in an effort to get Github to show the proper git history (this might not be a github thing but we'll quickly find out lol).

waydegg added 21 commits September 14, 2023 01:40

inita commit

13d90d9

refactor sync code

00993bf

stream messages to terminal client, rename conversation

8367505

update message schema

a84954a

fix ansi color redering, asyncifying stuff

4ab19be

fix user input

de0daf9

add remote interrupt (wip)

e0e74f2

working remote interrupt

533e4b0

fix end formatting

aa13e22

fix completions, remove old prompt toolkit code

f6a974e

made ctags subprocess async, store session conv as a contextvar

f64f7dd

fix code change rendering

5599053

simplify stream api, remove skeleton code

d63ba71

refactor streaming (wip)

735447f

refactor streaming, fix terminal client

2861c37

remove streaming printer

fba647a

made code change feedback async, fixed terminal lifecycle

dee9c87

remove app.py

5a5a094

fix files not being added to context

4ed6bc7

remove engine, update cli

be5b32e

remove aiofiles (will add later)

99fe14a

granawkins mentioned this pull request Sep 24, 2023

Context tree #107

Closed

waydegg added 8 commits September 25, 2023 15:41

fix broadcast typing

bde1a43

bug fixes, add pytest-asyncio

60f3717

merge tests (need to update still)

9caabfa

merge project/packaging files

fb6055b

fix broadcast typing

7c3e77b

fix typing for SessionStream

3a66374

fix merge conflicts for ConfigManager

79eea21

merge code map, fix merge conflicts

90426c8

granawkins reviewed Sep 27, 2023

View reviewed changes

PCSwingle reviewed Sep 27, 2023

View reviewed changes

waydegg and others added 4 commits September 27, 2023 17:09

fixed a bunch of merge conflicts

cefd0c9

fixed commands, interrupt handler

7975109

merge README

6148346

Added ipdb

a6e8467

Fix logging config git conflict token

a306341

PCSwingle reviewed Sep 28, 2023

View reviewed changes

add setup-api-key back into main workflow

5f7118a

granawkins mentioned this pull request Sep 29, 2023

Implement basic auto-context generation #115

Merged

waydegg closed this Sep 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generic interface #106

Generic interface #106

waydegg commented Sep 23, 2023 •

edited

Loading

granawkins left a comment

granawkins Sep 27, 2023

waydegg Sep 28, 2023

granawkins Sep 27, 2023

granawkins Sep 27, 2023 •

edited

Loading

waydegg Sep 28, 2023

granawkins Sep 27, 2023

biobootloader commented Sep 27, 2023 •

edited

Loading

PCSwingle left a comment

PCSwingle Sep 27, 2023

PCSwingle Sep 27, 2023

PCSwingle Sep 27, 2023

jakethekoenig commented Sep 28, 2023

jakethekoenig commented Sep 28, 2023

PCSwingle Sep 28, 2023

granawkins commented Sep 28, 2023

waydegg commented Sep 29, 2023


		logger = logging.getLogger()

		_SESSION_STREAM: ContextVar[SessionStream] = ContextVar("mentat:session_stream")



		class Event:
		def __init__(self, channel: str, message: typing.Any) -> None:



		def get_previous_lines(code_change, num=2):
		async def print_previous_lines(code_change: CodeChange, num: int = 2):

Generic interface #106

Generic interface #106

Conversation

waydegg commented Sep 23, 2023 • edited Loading

Summary

To-do

granawkins left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

granawkins Sep 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

biobootloader commented Sep 27, 2023 • edited Loading

PCSwingle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakethekoenig commented Sep 28, 2023

jakethekoenig commented Sep 28, 2023

Choose a reason for hiding this comment

granawkins commented Sep 28, 2023

waydegg commented Sep 29, 2023

waydegg commented Sep 23, 2023 •

edited

Loading

granawkins Sep 27, 2023 •

edited

Loading

biobootloader commented Sep 27, 2023 •

edited

Loading