We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When we run llama-cli.exe, the output token generation process is logged in the main.log file.
llama-cli.exe
main.log
When we run koboldcpp in Debug Mode, the output token generation process is displayed in a console window, which looks nicer than main.log.
koboldcpp
Debug Mode
I want to log raw input tokens and the output token generation process using create_completion, similar to the methods above.
create_completion
I tried setting VERBOSE=TRUE, but it only shows basic information.
VERBOSE=TRUE
Could you help me? My apologies if this is already possible.
[Debug: Dump Raw Input Tokens, format: 6] '<s> (1)', ' [ (733)', 'INST (16289)', '] (28793)', ' Why (4315)', ' sky (7212)', ' is (34 9)', ' blue (5045)', '? (28804)', ' [ (733)', '/ (28748)', 'INST (16289)', '] (28793)', [Debug: Dump Forwarded Input Tokens, format: 6] '<s> (1)', ' [ (733)', 'INST (16289)', '] (28793)', ' Why (4315)', ' sky (7212)', ' is (34 9)', ' blue (5045)', '? (28804)', ' [ (733)', '/ (28748)', 'INST (16289)', '] (28793)', [Debug: n_past=0 Context Size = 0] Processing Prompt (13 / 13 tokens) Generating (1 / 512 tokens) [( Sky 47.31%) ( The 51.88%) ( Because 0.81%)] Generating (2 / 512 tokens) [( is 60.16%) ( appears 39.84%)] Generating (3 / 512 tokens) [( blue 100.00%)] Generating (4 / 512 tokens) [( because 84.90%) ( due 15.10%)] Generating (5 / 512 tokens) [( air 10.31%) ( of 64.01%) ( sunlight 9.57%) ( the 6.43%)] Generating (6 / 512 tokens) [( absor 6.07%) ( sc 35.81%) (, 15.98%) ( in 15.85%) ( mole 8. 46%)] Generating (7 / 512 tokens) [(bs 100.00%)] Generating (8 / 512 tokens) [( colors 98.39%) ( colours 1.61%)] Generating (9 / 512 tokens) [( in 96.37%) ( of 3.63%)] Generating (10 / 512 tokens) [( the 100.00%)] Generating (11 / 512 tokens) [( red 100.00%)] Generating (12 / 512 tokens) [( part 100.00%)] Generating (13 / 512 tokens) [( of 100.00%)] Generating (14 / 512 tokens) [( the 100.00%)] Generating (15 / 512 tokens) [( light 98.74%) ( visible 1.26%)]
[1718995964] embd_inp.size(): 8, n_consumed: 0 [1718995964] eval: [ '<s>':1, '<|user|>':32010, ' Who':11644, ' are':526, ' you':366, '?':29973, '<|end|>':32007, '<|assistant|>':32001 ] [1718995965] n_past = 8 [1718995965] last: [ '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<s>':1, '<|user|>':32010, ' Who':11644, ' are':526, ' you':366, '?':29973, '<|end|>':32007, '<|assistant|>':32001, ' I':306 ] [1718995965] n_remain: -2 [1718995965] eval: [ ' I':306 ] [1718995965] n_past = 9 [1718995965] last: [ '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<s>':1, '<|user|>':32010, ' Who':11644, ' are':526, ' you':366, '?':29973, '<|end|>':32007, '<|assistant|>':32001, ' I':306, ' am':626 ] [1718995965] n_remain: -3 [1718995965] eval: [ ' am':626 ] [1718995965] n_past = 10 [1718995965] last: [ '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<s>':1, '<|user|>':32010, ' Who':11644, ' are':526, ' you':366, '?':29973, '<|end|>':32007, '<|assistant|>':32001, ' I':306, ' am':626, ' Ph':1963 ] [1718995965] n_remain: -4 [1718995965] eval: [ ' Ph':1963 ] [1718995965] n_past = 11 [1718995965] last: [ '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<unk>':0, '<s>':1, '<|user|>':32010, ' Who':11644, ' are':526, ' you':366, '?':29973, '<|end|>':32007, '<|assistant|>':32001, ' I':306, ' am':626, ' Ph':1963, 'i':29875 ] [1718995965] n_remain: -5 [1718995965] eval: [ 'i':29875 ] [1718995965] n_past = 12
The text was updated successfully, but these errors were encountered:
No branches or pull requests
When we run
llama-cli.exe
, the output token generation process is logged in themain.log
file.When we run
koboldcpp
inDebug Mode
, the output token generation process is displayed in a console window, which looks nicer thanmain.log
.I want to log raw input tokens and the output token generation process using
create_completion
, similar to the methods above.I tried setting
VERBOSE=TRUE
, but it only shows basic information.Could you help me? My apologies if this is already possible.
Kobold Console Window with
Debug Mode
enabledmain.log from llama.cpp main
The text was updated successfully, but these errors were encountered: