boludo

boludo is a personal AI assistant that lives in your terminal. It respects your privacy by running specialized Large language models directly on your ordinary computer.

boludo is a user-friendly interface for llama.cpp.

Usage

boludo uses subcommands to call locally hosted models with specific parameters. boludo.toml is the config file with subcommands definitions (see: examples/boludo.toml). For example, if you defined someconfig, you can do:

$ boludo someconfig "How are you?"
I am fine, thanks.
$ boludo someconfig <input.txt >output.txt

In the config file, you can change the default behaviour of the model by adjusting two parameters:

• creativity modifies probabilities of the next word. For example:

  • 0.0 < creativity < 1.0 - increase predictability of result. The smaller the value, the more similar the result to the dataset used to train the model (useful for code generation or plagiarizing The New York Times)
  • creativity = 1.0 - preserves probabilities computed during model training (best for validating claims of the model author)
  • 1.0 < creativity - increase probabilities of unlikely next words (best for making the output more diverse, but higher values increase inconsistency).

  In popular LLM runners, this parameter is known as a temperature.

• cutoff discards statistically insignificant words (long tail) during the next word prediction. For example, cutoff = 0.03 means discarding words with a probability smaller than 3% of the value of the most probable word.

  In popular LLM runners, this parameter is known as a min-p.

Proofreading

Karen TheEditor is one of the most practical models. It's specialized in correcting writing (in American English). With a configuration similar to examples/boludo.toml, you can do:

$ boludo proofreader "Trolling is a art"
Trolling is an art.

With longer texts, you may be interested to know which words were changed. By using the tee and git commands, you can monitor progress and track changes:

$ boludo proofreader <original.txt | tee corrected.txt; git -c color.diff.new="italic brightgreen reverse black bold" -c color.diff.old="strike brightred reverse bold" diff --no-index --color-words=. original.txt corrected.txt

Coding

CodeNinja is a decent programmer assistant. With a configuration similar to examples/boludo.toml, you can:

• find solutions for complex tasks:

  $ boludo coder "Show me the unix shell code without description for the task: Given a text file and an integer k, print the k most common words in the file (and the number of their occurrences) in decreasing frequency."
  
  cat text_file.txt | tr -s ' ' '\n' > words
  sort words | uniq -c > uniq_words
  awk '{if (length($0)<=k) print}' uniq_words > output.txt
  rm words uniq_words

  (not quite the McIlroy's way)

• perform code completion:

  $ boludo coder "# returns anagram
  def anagram(str):"
  
     # sort the string
     sorted_str = ''.join(sorted(str.lower()))
  
     # return the result
     return sorted_str

  (it correctly recognized Python code)

• mentoring:

  $ boludo coder "Which 3 malpractices are most common among C# programmers?"
  1. Ignoring exceptions: Many C# programmers tend to ignore exceptions, which can lead to unhandled errors and unpredictable behavior in the application.
  
  2. Incorrect use of `string.Format`: Programmers often make mistakes when using `string.Format` for string manipulation, resulting in potential vulnerabilities or incorrect output.
  
  3. Lack of unit testing: Many C# developers neglect to write unit tests, which can lead to undiscovered bugs and poor code quality.
  
  To avoid these common mistakes, it's essential for C# programmers to follow best practices, utilize error-handling techniques, and practice thorough testing.

  (that's interesting - format string attack is mostly known in the C world, but it's still relevant in C#).

Installation

You can manually build boludo with commands: make && make build.

Development

Use make (GNU or BSD):

• make - install dependencies
• make test - runs test
• make check - static code analysis
• make build - compile binaries from latest commit
• make dist - compile binaries from latest commit for supported OSes
• make clean - removes compilation artifacts
• make cli-release - tag latest commit as a new release of CLI
• make info - print system info (useful for debugging).

Versioning

The repo contains command-line utility which versions are tagged as cli/vYYYY.0M.MICRO (calendar versioning).

TODO

Some ideas for further development:

• restrict allowed kernel calls (with seccomp and pledge)
• embed main command from llama.cpp using WASM (this will also make app more secure by sandboxing main command)
• implement native Go infering engine instead of llama.cpp (the gguf format is well defined)
• show spinning wheel at stderr before generation.

License

MIT (in simple words)