Generalize file-based RPC command client beyond VSCode #956

JohnEffo · 2022-08-13T16:26:52Z

What is this

Split command-client so that other application can reuse the logic for of command server for RPC using existing VS code Command-Client.

Reasoning

In slack advised by aegis, when discussing RPC mechanism for Visual Studio.

"
if you replicate the vscode file rpc, please try to port the existing code as precisely as possible, without changing the sequence of file reads/writes or anything, because we spent a while making that work well
"

Initially copied and pasted existing code, wrote the Visual Studio side of things, then thought that best port is re-use of existing code

Discussion

If this PR goes ahead is Command-Client in the correct location? Probably not, but was not sure were to place it and wanted to float this idea before continuing with anymore work.
Should the methods not be renamed so that they are not so VSCode specific? Yes probably, for the moment I'm happy with the way things work and there is existing code which would need updating (Cursorless) which I did not want to touch as part of this PR.

Apologies

I've installed pre-commit but have not gotten it working correctly as of yet, but wanted to get your opinion on, whether this is this a valid direction of travel.

for more information, see https://pre-commit.ci

pokey · 2022-08-19T14:55:13Z

apps/vscode/command_client/command_client.py

 ctx.matches = r"""
 app: vscode
+app: visual_studio
 """


We should do the following:

add a new tag called command_client,

activate the tag in both visual studio and talon

use the tag as the matches for the context here instead of listing all the apps

I think this is what I was striving for for but could no make the final leap.

pokey · 2022-08-19T14:55:38Z

apps/vscode/command_client/command_client.py

-@ctx.action_class("user")
-class UserActions:
-    def emit_pre_phrase_signal() -> bool:
-        get_signal_path("prePhrase").touch()
-
-        return True


add this class back

pokey · 2022-08-19T14:55:46Z

apps/vscode/command_client/command_client.py

+    def live_pre_phrase_signal() -> bool:
+        """Used by application contexts emit_pre_phrase_signal to emit a pre phrase signal"""
+        get_signal_path("prePhrase").touch()
+        return True
+


Remove this

pokey · 2022-08-19T14:56:03Z

apps/vscode/command_client/visual_studio.py

+visual_studio_ctx = Context()
+
+visual_studio_ctx.matches = r"""
+app: visual_studio
+"""


this can just be called ctx

Yes, originally I had this in command_client and never renamed when I copied in out.

pokey · 2022-08-19T14:56:16Z

apps/vscode/command_client/vs_code.py

+vs_code_ctx = Context()
+
+vs_code_ctx.matches = r"""
+app: vscode
+"""


this as well can just be ctx

pokey · 2022-08-19T15:19:56Z

apps/vscode/command_client/command_client.py

@@ -210,7 +199,6 @@ def run_vscode_command(

 def get_communication_dir_path():
    """Returns directory that is used by command-server for communication
-


pokey · 2022-08-19T15:20:14Z

apps/vscode/command_client/command_client.py

-
    Raises:
        Exception: If there is an issue with the file-based communication, or
        VSCode raises an exception
-


pokey · 2022-08-19T15:20:23Z

apps/vscode/command_client/command_client.py

-
    Args:
        request (Request): The request to serialize
        path (Path): The path to write to
-


doesn't look fixed? still deleting lines here

Double checked should be fixed.

pokey · 2022-08-19T15:20:30Z

apps/vscode/command_client/command_client.py

 def write_json_exclusive(path: Path, body: Any):
    """Writes jsonified object to file, failing if the file already exists
-


one more 😅

pokey · 2022-08-19T15:23:40Z

apps/vscode/command_client/vs_code.py

+    def vscode(command_id: str):
+        """Execute command via vscode command server, if available, or fallback
+        to command palette."""
+        actions.user.post_command(command_id)


What is this supposed to do? This is causing the following error when I try to run pretty much any command in VSCode (other than Cursorless commands, which don't use this function) 😅:

KeyError: "Action 'user.post_command' is not declared."

Removed I think it was part of embroyonic attempt to have both new command_server command as vscommands working for backwards compatibility, before I decided that that could wait.

pokey · 2022-08-19T15:26:35Z

Good stuff! Left a bunch of comments but I think overall this is a nice direction

In the future we should probably have an action called run_command instead of expecting all applications to just use an action called vscode 😅. Could probably add that later 🤷

for more information, see https://pre-commit.ci

pokey

Getting close. As discussed, it would be good to

rename the actions starting with vscode_ to run_command_*
Then add shims in vscode that call these actions
Open a PR in cursorless that tries the run_command_ versions, falling back to vscode_ if action not found

Also see below for the idea I had around raising an exception

pokey · 2022-08-21T10:33:00Z

apps/vscode/command_client/command_client.py

@@ -153,7 +144,7 @@ def run_vscode_command(
        if args or return_command_output:
            raise Exception("Must use command-server extension for advanced commands")
        print("Communication dir not found; falling back to command palette")
-        run_vscode_command_by_command_palette(command_id)
+        actions.user.command_client_fallback(command_id)


The idea would be to throw an exception here

pokey · 2022-08-21T10:34:42Z

apps/vscode/command_client/vs_code.py

+class VsCodeAction:
+    def command_server_directory() -> string:
+        return "vscode-command-server"
+


And then define the action vscode here, and call the run_command action, wrapped in a try-catch. If it throws an error, do the fallback

Done, this but duplicated code

pokey · 2022-08-21T10:35:08Z

apps/vscode/command_client/vs_code.py

+    def command_server_directory() -> string:
+        return "vscode-command-server"
+
+    def command_client_fallback(command_id: str):


Then you can remove this action; just inline into vscode above

apps/vscode/command_client/vs_code.py

for more information, see https://pre-commit.ci

…hnEffo/knausj_talon into feature/command-client-re-use

for more information, see https://pre-commit.ci

rntz

I pushed a few changes to this PR, I hope you don't mind. In particular:

I removed some changes that you made to the formatters that I assume weren't intended to be part of this PR.
I restored some empty lines you deleted.
I fixed the type annotation on *args in run_command and added this to its docstring; and moved the default implementation of emit_pre_phrase_signal as return False from the global context to the module where it's defined. This could have been a separate PR but they would conflict, so I thought it would be easier for pokey to review them together.

apps/vscode/command_client/command_client.py

apps/vscode/command_client/command_client_tag.py

rntz · 2022-08-24T19:19:09Z

Looking pretty good! I hope this lands soon, I will probably need this to get cursorless-in-emacs working.

for more information, see https://pre-commit.ci

apps/vscode/command_client/command_client.py

apps/vscode/command_client/command_client_tag.py

apps/vscode/command_client/visual_studio.py

apps/vscode/command_client/vscode.py

for more information, see https://pre-commit.ci

pokey

Looking pretty good; left a few more comments

apps/vscode/command_client/command_client.py

apps/vscode/command_client/vscode.py

pokey · 2022-09-03T13:14:51Z

apps/vscode/command_client/vscode.py

+# These commands are shims, to provide backwards compatibility, they may be removed in the fuuture.
+# Prefer the run_command... version in command_client.
+@mod.action_class
+class Actions:


What happened to the pre-phrase signal code? Unless I've missed something, it appears to have been dropped

Ah nvm I see it now. A bit confusing how it's declared in multiple places

What's the purpose of allowing applications to turn it off? Seems simpler to just leave it on and let all applications share the same impl

See command client 368 its in ctx which uses the match belwo to check we are in the correct context. I put a comment in command_client_tag to explain when to overwrite

ctx.matches = r""" tag: command_client """

I'll remove it from the actions so that it cannot be turned of.

Uh, this seems to have been changed into a non-working state. There's no module which declares emit_pre_phrase_signal, only a context which implements it, which should be producing an error in the talon log. We probably want the default implementation to be return False. There's no sense touching a file if we're not in a command client. I'll go change this now.

Ok, this should be working now. @pokey is this okay? Did you want us to always touch the pre-phrase file, even if we don't have a command-client app focused?

interestingly, it doesn't seem to be causing an error in the talon log. not sure why.

ok. it does error if I actually try to use the action, though. I'll try to minimize that and report it as a talon bug.

ok, I've reported the underlying bug to talon, talonvoice/talon#525.

for more information, see https://pre-commit.ci

…hnEffo/knausj_talon into feature/command-client-re-use

apps/vscode/command_client/command_client.py

concerns addressed

rntz · 2022-09-04T22:07:42Z

ok, @pokey, I think this is ready for prime time. I've merged it with my local branch and it works for me. If you're ok with it I think we should merge.

pokey

Ok I did some cleanup, and tested that things are still working for me locally. I left one comment; otherwise good to go!

apps/vscode/command_client/visual_studio.py

Co-authored-by: Pokey Rule <755842+pokey@users.noreply.github.com>

rntz · 2022-09-06T14:53:16Z

@pokey I think this is just waiting on you to clear your "changes requested" review, until you do that I can't merge it.

pokey

~~I'm happy, but let's give @JohnEffo a chance to review the change to the server directory, as that will likely break his server implementation until he's done the rename Visual Studio-side~~ whoops missed #956 (comment)

I'd also like to give @lunixbochs a chance to have a look before we merge, as this PR establishes the RPC mechanism that Visual Studio, VSCode, Emacs, and JetBrains will be using until there is an official Talon RPC, so it would be good to make sure we're not doing anything that will make our lives harder when we migrate to official Talon RPC

pokey · 2022-09-06T17:47:17Z

One other thought: we should probably move this directory in a follow-up PR, as it is still under apps/vscode. Maybe let's wait for #897 though?

# What is this Split command-client so that other application can reuse the logic for of command server for RPC using existing VS code Command-Client. ## Reasoning In slack advised by aegis, when discussing RPC mechanism for Visual Studio. " if you replicate the vscode file rpc, please try to port the existing code as precisely as possible, without changing the sequence of file reads/writes or anything, because we spent a while making that work well " Initially copied and pasted existing code, wrote the Visual Studio side of things, then thought that best port is re-use of existing code ## Discussion * If this PR goes ahead is Command-Client in the correct location? Probably not, but was not sure were to place it and wanted to float this idea before continuing with anymore work. * Should the methods not be renamed so that they are not so VSCode specific? Yes probably, for the moment I'm happy with the way things work and there is existing code which would need updating (Cursorless) which I did not want to touch as part of this PR. ## Apologies I've installed pre-commit but have not gotten it working correctly as of yet, but wanted to get your opinion on, whether this is this a valid direction of travel. Co-authored-by: johneffo <johneffo@gmailcom.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michael Arntzenius <daekharel@gmail.com> Co-authored-by: Pokey Rule <755842+pokey@users.noreply.github.com>

johneffo and others added 4 commits August 13, 2022 16:09

split command client to applicactions

423a261

Remove trailing white space

df2e523

More white space

a345741

[pre-commit.ci] auto fixes from pre-commit.com hooks

9304e22

for more information, see https://pre-commit.ci

pokey requested changes Aug 19, 2022

View reviewed changes

johneffo and others added 3 commits August 19, 2022 19:04

Address CR comments

09ef97e

[pre-commit.ci] auto fixes from pre-commit.com hooks

613281d

for more information, see https://pre-commit.ci

Remove error producing pre-phrase-signal

2f5c726

pokey reviewed Aug 21, 2022

View reviewed changes

johneffo and others added 12 commits August 21, 2022 23:03

Make VSCode Shim

dbb6893

Add Shim

211802a

[pre-commit.ci] auto fixes from pre-commit.com hooks

a7c1c52

for more information, see https://pre-commit.ci

Remove unneeded function from tag

01bc65d

Merge branch 'feature/command-client-re-use' of https://github.com/Jo…

8d57e34

…hnEffo/knausj_talon into feature/command-client-re-use

Add comment to vscode

ce14610

[pre-commit.ci] auto fixes from pre-commit.com hooks

4c9c482

for more information, see https://pre-commit.ci

fix up some types & docs

27aa1f3

implement emit_pre_phrase_signal on module instead of in global context

640af19

typo

c2fbf32

restore some lines, remove unnecessary print

295f843

remove unwanted changes

c41f842

rntz reviewed Aug 24, 2022

View reviewed changes

apps/vscode/command_client/command_client.py Outdated Show resolved Hide resolved

apps/vscode/command_client/command_client.py Outdated Show resolved Hide resolved

apps/vscode/command_client/command_client_tag.py Outdated Show resolved Hide resolved

johneffo and others added 3 commits August 25, 2022 18:55

Remove duplicated code

7717e82

Remove run_command and rename

340b707

[pre-commit.ci] auto fixes from pre-commit.com hooks

2d83c9d

for more information, see https://pre-commit.ci

rntz requested changes Aug 27, 2022

View reviewed changes

johneffo and others added 2 commits August 28, 2022 08:10

Remove string from mong language

4badb6c

[pre-commit.ci] auto fixes from pre-commit.com hooks

eb5254b

for more information, see https://pre-commit.ci

johneffo and others added 3 commits September 1, 2022 20:06

Alter missing directory error msg

4d7609f

Rename time out

054e81f

[pre-commit.ci] auto fixes from pre-commit.com hooks

c5eb3ab

for more information, see https://pre-commit.ci

rntz approved these changes Sep 2, 2022

View reviewed changes

fidgetingbits approved these changes Sep 3, 2022

View reviewed changes

pokey requested changes Sep 3, 2022

View reviewed changes

johneffo and others added 10 commits September 4, 2022 11:23

Remove client from comment

a8a1409

Moved comment

5b0d283

Removed 'final' from comment

347c9b3

[pre-commit.ci] auto fixes from pre-commit.com hooks

571abce

for more information, see https://pre-commit.ci

remove comment pre phrase from tag

b9719f5

[pre-commit.ci] auto fixes from pre-commit.com hooks

13e7d70

for more information, see https://pre-commit.ci

Merge branch 'feature/command-client-re-use' of https://github.com/Jo…

39478d5

…hnEffo/knausj_talon into feature/command-client-re-use

command server => rpc

f831999

Merge branch 'feature/command-client-re-use' of https://github.com/Jo…

033eae5

…hnEffo/knausj_talon into feature/command-client-re-use

fix emit_pre_phrase_signal

29afd9a

rntz previously requested changes Sep 4, 2022

View reviewed changes

apps/vscode/command_client/command_client.py Outdated Show resolved Hide resolved

rntz mentioned this pull request Sep 4, 2022

malformed ctx.matches hides error messages talonvoice/talon#525

Open

fix "tag: command_client"

173ebef

Fixes

d7e89b1

pokey reviewed Sep 6, 2022

View reviewed changes

apps/vscode/command_client/visual_studio.py Outdated Show resolved Hide resolved

Update apps/vscode/command_client/visual_studio.py

3d1e016

Co-authored-by: Pokey Rule <755842+pokey@users.noreply.github.com>

pokey approved these changes Sep 6, 2022

View reviewed changes

pokey changed the title ~~Feature/command client re use~~ Generalize file-based RPC command client beyond VSCode Sep 6, 2022

pokey merged commit 9e664a1 into talonhub:main Sep 10, 2022

		@@ -210,7 +199,6 @@ def run_vscode_command(

		def get_communication_dir_path():
		"""Returns directory that is used by command-server for communication

		def write_json_exclusive(path: Path, body: Any):
		"""Writes jsonified object to file, failing if the file already exists

Generalize file-based RPC command client beyond VSCode #956

Generalize file-based RPC command client beyond VSCode #956

Conversation

JohnEffo commented Aug 13, 2022

What is this

Reasoning

Discussion

Apologies

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pokey commented Aug 19, 2022

pokey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rntz left a comment

Choose a reason for hiding this comment

rntz commented Aug 24, 2022 • edited

pokey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pokey Sep 4, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rntz commented Sep 4, 2022

pokey left a comment

Choose a reason for hiding this comment

rntz commented Sep 6, 2022

pokey left a comment • edited

Choose a reason for hiding this comment

pokey commented Sep 6, 2022

rntz commented Aug 24, 2022 •

edited

pokey Sep 4, 2022 •

edited

pokey left a comment •

edited