Cody: Improve fixup response speed #1464
Assignees
Milestone
Comments
This was referenced Nov 13, 2023
umpox
added a commit
that referenced
this issue
Nov 14, 2023
Part of #1544 Part of #1464 ## Description "Ask Cody to Fix" is one of the most common command actions we currently have. It makes sense to support this better instead of treating this like a normal "edit" - Skip fetching context, from my personal testing it very rarely helps. Most information is contained within the error, or is part of `relatedInformation` attached to the diagnostic - We can start fetching more relevant context from sources directly related to the fix: #1723 - Use a more targeted prompt preamble to address the task ## Test plan - Fixups without code actions (edit/document/edit with no selection) - Fixups through code action (Ask Cody to Fix) <!-- Required. See https://docs.sourcegraph.com/dev/background-information/testing_principles. -->
umpox
added a commit
that referenced
this issue
Nov 15, 2023
Part of #1544 Part of #1464 ## Description The LLM reranker adds a significant amount of latency to reranking any fetched context. This applies to all commands, chat and edits. The latency is typically between 2-5 seconds. This PR just disables the reranker for now. I think this is worth doing as: - The added latency from the reranker is not worth the possible benefit right now. It still can sometimes rerank poorly and doesn't provide enough value to justify waiting that long. - Context doesn't have a constant value, in many cases context will provide little value (e.g. simple fixups) in some cases, like chat, context would provide a lot more value. We don't have a way of determining that right now, so we're often adding lots of latency for no good reason. I created an issue here: #1721 to evaluate a different or improved approach. Possible details there on how we could improve this. ## Test plan On a repository without embeddings, run: - Chat - Commands - Edits Review results and context included.
umpox
added a commit
that referenced
this issue
Nov 28, 2023
closes #1464 ## Description This PR: - Streamlines the `doc` intent flow for fixup tasks. Previously we would treat these as "edit", which means we would try to fetch lots of context from the selected code. That is very rarely helpful, for documenting we really just want to take the surrounding code and the current selection. I also investigating going the other way, and fetching relevant reference code for a documentable symbol. This confirmed to me that it's not necessarily helpful. It can lead to the LLM documenting code based on how it is used elsewhere rather than how it should be used. That might help for "explaining" code but not for documenting. (Related PR: #1904) ## Test plan 1. Run doc command via right click menu 2. Run doc command via sidebar 3. Run doc command via code action <!-- Required. See https://docs.sourcegraph.com/dev/background-information/testing_principles. -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Fixups are still quite slow. We should be improving the response speed
Areas identified already:
Reranker:rerank: {"duration":4398.333707809448}The text was updated successfully, but these errors were encountered: