update: log token count for code generated and button click events across the extension

[abeatrix]

RE: https://docs.google.com/document/d/1UcjUdon1XO5GoLDIJJCGBq1zyhR6VfL2FE_apSPeLQo/edit#heading=h.m8ojuhhojtjm
Close #672

Add loggings for following events

Description	Type	Example Log Entry
Chat submit from sidebar	submit	CodyVSCodeExtension:chat:submitted {"source":"chat"}
Chat submit from inline	submit	CodyVSCodeExtension:chat:submitted {"source":"inline"}
Chat submit from command menu	submit	CodyVSCodeExtension:chat:submitted {"source":"menu"}
Code generated for chat	response	CodyVSCodeExtension:chatResponse:noCode {}
Code generated for chat	response	CodyVSCodeExtension:chatResponse:hasCode {"lineCount":20,"charCount":575}
Fixup code lens button: apply	click	CodyVSCodeExtension:fixup:codeLens:clicked', { op: apply)
Fixup code lens button: show diff	click	CodyVSCodeExtension:fixup:codeLens:clicked', { op: diff)
Fixup code lens button: cancel	click	CodyVSCodeExtension:fixup:codeLens:clicked', { op: cancel)
Fixup token count for code applied	applied	CodyVSCodeExtension:fixup:applied {"lineCount":20,"charCount":575}
Chat title bar button: new convo	click	CodyVSCodeExtension:chatTitleButton:clicked {"name":"reset"}
Chat title bar button: history	click	CodyVSCodeExtension:chatTitleButton:clicked {"name":"history"}
Copy event on code generated for Inline Chat	keydown	CodyVSCodeExtension:inlineChat:event:detected {"op":"copy","lineCount":5,"charCount":76}
Paste event on code generated for Inline Chat	keydown	CodyVSCodeExtension:inlineChat:event:detected {"op":"paste","lineCount":5,"charCount":76}
Copy event on code generated for side bar chat triggered by copy button click:	click	CodyVSCodeExtension:copyButton:clicked {"lineCount":2,"charCount":27,"op":"copy"}
Paste event on code generated for side bar chat triggered by copy button click:	keydown	CodyVSCodeExtension:pasteButton:clicked {"lineCount":2,"charCount":27,"op":"paste"}
Copy event on code generated for side bar chat, but not triggered by buttons	keydown	CodyVSCodeExtension:copyKeydown:clicked {"lineCount":2,"charCount":27,"op":"copy"}
Paste event on code generated for side bar chat, but not triggered by buttons	keydown	CodyVSCodeExtension:pasteKeydown:clicked {"lineCount":2,"charCount":27,"op":"paste"}
Insert code block button	click	CodyVSCodeExtension:insertButton:clicked {“op”:“insert”,“charCount”:194,“lineCount”:11}
On “Stop Generating” button click in sidebar	click	CodyVSCodeExtension:abortButton:clicked' { source: 'sidebar' }
On “Stop Generating” button click in Inline Chat	click	CodyVSCodeExtension:abortButton:clicked' { source: inline }
Code generated for fixup task	response	CodyVSCodeExtension:fixupResponse:hasCode {"lineCount":12,"charCount":354}

CodyVSCodeExtension:completion:accepted was added by @philipp-spiess in #674

CodyVSCodeExtension:completion:suggested is not added. Will work on this in a separated PR

Test plan

Added test for token count methods.

To test the newly added token events

Start Cody from this branch in debug mode, and check the Output channel for logged events after performing an action:

[chenkc805]

Thanks! Mostly looks good to me. Let me give this a spin in the IDE to QA later today

QQ, for the first 3 events:

• Chat submit from sidebar submit CodyVSCodeExtension:chat:submitted {"source":"chat"}
• Chat submit from inline submit CodyVSCodeExtension:chat:submitted {"source":"inline"}
• Chat submit from command menu submit CodyVSCodeExtension:chat:submitted {"source":"menu"}

Don't we already have events for these as "executed" events? CC: @kelsey-brown

[abeatrix]

Don't we already have events for these as "executed" events?

@chenkc805 They are currently logged as:

CodyVSCodeExtension:recipe:chat-question:executed 
CodyVSCodeExtension:recipe:inline-chat:executed 

But this does not capture the chat-question submitted from the command menu, or if the chat submitted was a slash command, it would be logged under custom-prompt executed when it was also submitted from chat (hope this makes sense 😅 )

[kelsey-brown]

@abeatrix Is the goal to centralize the "chat" events for VSCode under one event name, add the source field to event metadata so we can distinguish where they are coming from, then retire the old events (CodyVSCodeExtension:recipe:chat-question:executed, CodyVSCodeExtension:recipe:inline-chat:executed)?

This looks good overall, I think the only thing I'd love to see if possible is for the source (command menu, sidebar, inline) and the action (copy, paste, insert) to be logged in the same place/uniformly for relevant events.

E.g. for copy events they always have copy (or insert) in the name, or they always have op:copy in the metadata. My reasoning is that we'll need to do GROUP BYs on both of these fields so we can run various aggregations, and having some uniformity ensures we won't miss anything. If it's a ton of trouble to make that happen, no worries, we can figure it out on our end. But if not, it would save us a lot of hassle in terms of data prep/cleaning. Let me know what you think!

[abeatrix]

Is the goal to centralize the "chat" events for VSCode under one event name, add the source field to event metadata so we can distinguish where they are coming from, then retire the old events (CodyVSCodeExtension:recipe:chat-question:executed, CodyVSCodeExtension:recipe:inline-chat:executed)?

@kelsey-brown basically, yes. Cause the recipe executed from chat does not always log under chat-question as mentioend

This looks good overall, I think the only thing I'd love to see if possible is for the source (command menu, sidebar, inline) and the action (copy, paste, insert) to be logged in the same place/uniformly for relevant events.

I was following what the naming of the existing event added by the data team but I can look into updating those.
Do you mean instead of CodyVSCodeExtension:pasteButton:clicked {"lineCount":2,"charCount":27,"op":"paste"} and CodyVSCodeExtension:pasteKeydown:clicked {"lineCount":2,"charCount":27,"op":"paste"}, you want: CodyVSCodeExtension:clicked {"lineCount":2,"charCount":27,"op":"paste", "source": "button"} and CodyVSCodeExtension:clicked {"lineCount":2,"charCount":27,"op":"paste", "source": "keydown"}

[kelsey-brown]

@kelsey-brown basically, yes. Cause the recipe executed from chat does not always log under chat-question as mentioend
Makes sense, this works for me then!

I was following what the naming of the existing event added by the data team but I can look into updating those.

This naming is totally fine! I actually think the easiest way to do this would just be to change these:

• CodyVSCodeExtension:inlineChat:event:detected --> CodyVSCodeExtension:inlineChat:Copy:detected
• CodyVSCodeExtension:inlineChat:event:detected --> CodyVSCodeExtension:inlineChat:Paste:detected

So all the copy events contain copy, and all the paste events contain paste. Does that make sense? It's a bit of a nitpick, but the uniformity helps us a lot with filtering and grouping!

[abeatrix]

@kelsey-brown Fair! Just updated:

• CodyVSCodeExtension:inlineChat:event:detected --> CodyVSCodeExtension:inlineChat:Copy:detected
• CodyVSCodeExtension:inlineChat:event:detected --> CodyVSCodeExtension:inlineChat:Paste:detected

[chenkc805]

Having trouble getting my local environment setup so I'll review this in pre-release. Based on the documentation here though, looks good to me.