Skip to content

Conversation

@mathsgod
Copy link
Contributor

Reference to #63 (comment)

When esbuild the cli.cjs, it doesn't include the WebAssembly file. Is it possible copy the wasm file to out folder to solve the error?

…ld command to it

feat(package.json): add @dqbd/tiktoken dependency
fix(tokenCount.ts): uncomment Tiktoken import and encoding, and return token length instead of an approximation
@di-sukharev
Copy link
Owner

di-sukharev commented Mar 29, 2023

great job bro! well, i will test that later because you cant test the hook locally, I'd consider moving from esbuild to simple tsc . cause esbuild doesn't support few things which we use in the hook

you can read through https://esbuild.github.io/getting-started/#bundling-for-node

WDYT, should we move to tsc .?

@mathsgod
Copy link
Contributor Author

Although I don't know the actual problem when using in hook, but if tsc can solve the problem, why not try it on?

@di-sukharev
Copy link
Owner

bro, i want to merge it, have you tested it locally with npm run build and then npm run start -- hook set?

@di-sukharev
Copy link
Owner

@mathsgod thanks for contribution <3 this PR should solve 400 error when too much tokens are being sent

@di-sukharev di-sukharev merged commit 9f65c45 into di-sukharev:dev Apr 5, 2023
zen0bit pushed a commit to oSoWoSo/opencommit that referenced this pull request Apr 12, 2023
…ld command to it (di-sukharev#82)

feat(package.json): add @dqbd/tiktoken dependency
fix(tokenCount.ts): uncomment Tiktoken import and encoding, and return token length instead of an approximation
zen0bit pushed a commit to oSoWoSo/opencommit that referenced this pull request Apr 14, 2023
…ld command to it (di-sukharev#82)

feat(package.json): add @dqbd/tiktoken dependency
fix(tokenCount.ts): uncomment Tiktoken import and encoding, and return token length instead of an approximation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants