Skip to content

Convert all files in git repository to .txt files. Useful for training LLMs on your codebase.

Notifications You must be signed in to change notification settings

pipeline-crawler/git2txt

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

git2txt

Convert all files in git repository to .txt files. This is useful for training LLMs on your codebase.

How to Use

  1. Create new .env file by copying example.env
cp example.env .env
  1. Add necessary fields. The default fields are good to start with.
GIT_PROJECT_DIRECTORY=/path/to/git/repo
IGNORE_FILES=.env,package-lock.json
IGNORE_DIRS=.git,.vscode,node_modules
SAVE_DIRECTORY=training_data
SKIP_EMPTY_FILES=true
  1. Install dependencies. Using a virtual environment is recommended.
python -m pip install -r requirements.txt
  1. Run program
python main.py
  1. You'll see your data files in the training_data/ directory. This will be different if you changed the path via SAVE_DIRECTORY in .env file.

Notes

  • This program requires Python version 3.6 or later. It uses the f-string formatting technique introduced in Python 3.6.

About

Convert all files in git repository to .txt files. Useful for training LLMs on your codebase.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%