Skip to content

palmshed/rag

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

497 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RAG

Welcome to the RAG assistant.

This project provides a local CLI/TUI assistant focused on machine learning, science fiction, and the cosmos.

PNG image 1024x1024
Figure 1. RAG Assistant

RAG is open source under the Apache 2.0 license.

A couple of things are useful to know before you start. For one, this repository uses AsciiDoc for the primary docs; here’s an AsciiDoc quick reference.

We also keep optional artifacts (datasets, outputs) in repo-local directories. See the translating document for more information.

For a technical deep-dive into how RAG works, see RAG Explained. For contributing, see GitHub Access Guide.

Installation

macOS App Version

For macOS users, pre-built app bundles are available for easy installation:

  1. Download the appropriate DMG file:

    • RAG_Transformer.dmg - Full CLI app bundle

    • RAG_Minimal_TUI.dmg - Minimal TUI app bundle

  2. Double-click the DMG to mount it.

  3. Drag the RAG Transformer.app or RAG Minimal TUI.app to your Applications folder.

  4. Launch the app from Applications - it will open in a Terminal window.

This provides a native macOS app experience without manual dependency installation.

Development Installation

You can run the CLI or TUI directly after installing dependencies. If you run the following you may actually get a working CLI, TUI, and tests:

$ python -m venv venv
$ source venv/bin/activate  # Windows: venv\\Scripts\\activate
$ pip install -e ".[test,dev]"
$ rag --help
$ rag-agent --help
$ rag-tui --help
$ .venv313/bin/python -m pytest --cov=src --cov-report=term-missing

You can run just one of the supported entry points (CLI, TUI, or data collector). Use one of the following commands:

To run the CLI:

$ rag

To run the minimal TUI (recommended):

$ rag-agent

To run the classic TUI (with colors):

$ rag-tui

To run the TUI with minimal theme:

$ rag-tui --theme minimal

To run the data collector:

$ rag-collect

Code Review

The RAG assistant can analyze code for security issues, bugs, and code quality using the REVIEW: tool.

Review Commands

  • review diff - Review changed lines in working tree

  • review staged - Review staged changes

  • review <path> - Review a file

  • review <path>:<start>-<end> - Review line range

  • threads - List saved review threads

  • thread add <path:line> <comment> - Add a thread

Examples

Ask naturally to invoke review:

$ rag-agent
❯ analyze sample_review.py code
Review findings for sample_review.py
- sample_review.py:8 [high] `eval` executes dynamic code...
- sample_review.py:13 [high] `subprocess` call uses `shell=True`...

❯ check sample_review.py for security issues
Based on the review findings, I found the following high-severity...

❯ review sample_review.py:8-20
- sample_review.py:8 [high] `eval` executes dynamic code...
- sample_review.py:13 [high] `subprocess` call uses `shell=True`...

Or use tool prefix directly:

❯ REVIEW: review sample_review.py
Review findings for sample_review.py
- sample_review.py:8 [high] `eval` executes dynamic code...

❯ REVIEW: threads
Saved review threads (none)

Detected Issues

The review tool detects:

  • eval() / exec() - code execution risks

  • shell=True in subprocess - shell injection

  • pickle.load() - deserialization attacks

  • Bare except: - catches everything

  • Hardcoded credentials / API keys

  • innerHTML= - XSS risks

  • yaml.load() without safe loader

Shell Commands

The RAG assistant can execute shell commands using the SHELL: tool. This is useful for git operations and other command-line tasks.

Available Tools

  • SHELL: <command> - Execute any shell command (e.g., SHELL: git status)

Git Operations

You can perform git operations by asking naturally:

$ rag-agent
❯ what files changed
According to the tool result, the following files have been modified:

1. README.asc
2. skill.asc
3. src/rag/rag_engine.py
4. src/rag/tools.py

Additionally, there is an untracked file that has not been added to the commit yet:

5. sample_review.py

❯ show recent commits
Based on the tool result, here are the recent commits:

c76d123 feat: review workspace
a5a515c feat: terminal code review workspace
f81c212 ci: track macos release build scripts
0c7e685 chore: clean tracked repo artifacts
64ab28b ci: build macos dmgs in repo

❯ tell me what changed in the last commit and what does it do
Based on the `feat: review workspace` request, I'll provide a code review for the latest commit.

After analyzing the code changes, I found that the most recent commit added some new features to our RAG Assistant. Here's what changed:

* In `src/rag/memory.py`, a new function was added to handle user-provided facts and store them in memory.
* In `src/rag/rag_engine.py`, the engine now supports more advanced natural language processing (NLP) capabilities, allowing for better understanding of user queries.
* In `src/rag/review.py`, a new tool was implemented to provide code analysis and review features.
* In `src/rag/ui/minimal_tui.py` and `src/rag/ui/tui.py`, the user interface (UI) has been updated.

As for what this commit does, it primarily enhances the RAG Assistant's capabilities in terms of memory management, NLP, and code review.

❯ ollama:llama3 run git status
❯ ollama:llama3 add the files
❯ ollama:llama3 commit with message "feat: add new feature"
❯ ollama:llama3 push to remote

Or use the tool prefix directly:

❯ SHELL: git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	modified:   skill.asc
	modified:   src/rag/rag_engine.py
	modified:   src/rag/tools.py
	modified:   src/rag/ui/tui.py

❯ SHELL: git add .
(command completed with no output)

❯ SHELL: git diff --staged
diff --git a/skill.asc b/skill.asc
index c10d963..2773427 100644
--- a/skill.asc
+++ b/skill.asc
@@ -18,17 +18,16 @@ Be direct, helpful, and safe.

== Tool Calling

-When a tool is needed, respond with *exactly one* tool call line:
+When a tool is nee...

❯ SHELL: git log --oneline -3
5e208a2 docs: update webpage version
03c7778 docs: update changelog
a997df5 chore(release): 1.8.3

❯ SHELL: git branch
* main

❯ SHELL: git push
To https://github.com/bniladridas/rag
   5e208a2..183acc7  main -> main

Natural Language Examples

You can also ask naturally:

❯ show recent commits
Based on the tool result, here are the recent commits:
c76d123 feat: review workspace
a5a515c feat: terminal code review workspace
f81c212 ci: track macos release build scripts

❯ what changes made in that commit
Based on the `feat: review workspace` request, I'll provide a code review for the latest commit.

After analyzing the code changes, I found that the most recent commit added:
* In `src/rag/memory.py`, a new function was added to handle user-provided facts...
* In `src/rag/review.py`, a new tool was implemented to provide code analysis...

❯ check sample_review.py for security issues
Based on the review findings, I found the following high-severity:
- sample_review.py:8 [high] `eval` executes dynamic code...
- sample_review.py:13 [high] `subprocess` call uses `shell=True`...

Full Workflow Example

$ rag-tui --theme minimal
❯ ollama:llama3 SHELL: git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  modified:   skill.asc
  modified:   src/rag/rag_engine.py
  modified:   src/rag/tools.py
  modified:   src/rag/ui/tui.py

❯ ollama:llama3 SHELL: git add .
(command completed with no output)

❯ ollama:llama3 SHELL: git diff --staged
diff --git a/skill.asc b/skill.asc
...
❯ ollama:llama3 SHELL: git commit -m "feat: include shell tool"
[main 183acc7] test: include shell tool
 4 files changed, 195 insertions(+), 37 deletions(-)

❯ ollama:llama3 SHELL: git push
To https://github.com/bniladridas/rag
   5e208a2..183acc7  main -> main

Testing Shell Tool Directly

from src.rag.rag_engine import RAGEngine

rag = RAGEngine()
rag.set_backend('ollama')

tests = [
    'SHELL: git status',
    'SHELL: git add .',
    'SHELL: git diff --staged',
    'SHELL: git log --oneline -3',
    'SHELL: git branch',
]

for q in tests:
    print(f'Q: {q}')
    resp = rag.generate_response(q)
    print(f'A: {resp}')

Signaling an Issue

Before signaling an issue, please check that there isn’t already a similar one in the bug tracking system.

Also, if this issue has been spotted in the CLI/TUI, please cross-check that it is still present with the current dependency set. The issue may have already been corrected, but the changes have not been deployed yet.

Community & Support

Contributing

If you’d like to help out by making a change, take a look at the contributor’s guide.

About

containerized python package.

Topics

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors