Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task]: investigate report of inaccurate token usage & missing chat context with Cline + Claude Sonnet #301

Open
alex-mcgovern opened this issue Feb 11, 2025 · 1 comment

Comments

@alex-mcgovern
Copy link
Collaborator

Description

the audit seems quite off for cline, there are lots of code and dialogue are missing (including tokens) , it also seems to just repeat the same conversation several times, worth getting onto this as we should be capturing token usage and history incorrectly, as this is a nice feature that we bring that others don't have. The following video is a chain of conversations, all constructed from an initial prompt of 'create a weather app that shows the results based on the city a user enters', you can see a html index file, css and javascript. It seems we only capture a small part of the javascript, and miss the css and html. We also miss a lot of the dialogue coming from the LLM. I will get a ticket open, but not had time to debug yet if its back or front end as just heading out to the airport.
I ended up having a quick look as I was going to send some screenshots to this thread: https://www.reddit.com/r/ChatGPTCoding/comments/1ihekcq/hidden_token_usage_of_cline_and_roo/
Could someone take this on as its important we are auditing correctly around token usages and prompt history. (edited)

Screen.Recording.2025-02-11.at.07.28.00.mov

Additional Context

No response

@lukehinds
Copy link
Contributor

Check again when Client Implementation lands

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants