How to process private repositories or local codebase

[BZHuntShampu]

Hi !

This project seems really cool and I'd love to try it on some of my repos.
However, those repos are private and can only be cloned via SSH (git clone git@github.com:<owner>/<repo>)

From the documentation, I was under the impression it is possible to provide the local path to the codebase in order to generate the README :

By providing a remote repository URL or path to your codebase [...]
[...]
-r or --repository: The URL or path to your code repository.

My issue is that the command doesn't seem to accept either git repositories in the git@github.com:<OWNER>/<REPO> format or local path to a git repository as in ~/projects/my-awesome-project

When using local path to codebase :

readmeai -k '<TOKEN>' -r ~/projects/my-awesome-project -o /tmp/readmai.md
[...]
File "/home/shampu/.local/lib/python3.11/site-packages/readmeai/main.py", line 23, in main
    config.git = conf.GitConfig(repository=repository)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/dataclasses.py", line 286, in pydantic.dataclasses._add_pydantic_validation_attributes.handle_extra_init

  File "<string>", line 5, in __init__
  File "pydantic/dataclasses.py", line 305, in pydantic.dataclasses._add_pydantic_validation_attributes.new_post_init
    return ('Field('
  File "/home/shampu/.local/lib/python3.11/site-packages/readmeai/conf.py", line 79, in __post_init__
    raise ValueError(f"Ivalid repository URL or path: {self.repository}")
ValueError: Ivalid repository URL or path: /home/shampu/projects/my-awesome-project

When providing path to private repo :

readmeai -k '<TOKEN>' -r 'git@github.com:Shampu/MyAwesomeProject.git' -o /tmp/readmai.md
[...]
File "/home/shampu/.local/lib/python3.11/site-packages/readmeai/main.py", line 23, in main
    config.git = conf.GitConfig(repository=repository)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/dataclasses.py", line 286, in pydantic.dataclasses._add_pydantic_validation_attributes.handle_extra_init

  File "<string>", line 5, in __init__
  File "pydantic/dataclasses.py", line 305, in pydantic.dataclasses._add_pydantic_validation_attributes.new_post_init
    return ('Field('
  File "/home/shampu/.local/lib/python3.11/site-packages/readmeai/conf.py", line 79, in __post_init__
    raise ValueError(f"Ivalid repository URL or path: {self.repository}")
ValueError: Ivalid repository URL or path: git@github.com:Shampu/MyAwesomeProject.git

I would love for anyone to enlighten me regarding the use of SSH cloning and local codebase for this cool project

[aanorlondo]

Hey @BZHuntShampu !

The project used to support Local repositories at some point.
Here is a commit hash that supports the feature: 5a9ab8b4c293806c8d8fe034fdd622299d8e6825

How to execute:

# 1st, you will need to clone the repo
git clone ...

# get the old local-repo compatible version
git checkout 5a9ab8b4c293806c8d8fe034fdd622299d8e6825

# save that version as a local branch for future use
git checkout -b feat/local-repo

# standard procedure
export OPENAI_API_KEY=<YOUR_KEY>
cd readme-ai #(or whatever you want to name the local repo once cloned)
python src/main.py --output <YOUR_OUTPUT_PATH>.md --repository <YOUR_LOCAL_REPO_PATH>/

# Enjoy !

PS: Dear Readme-AI team, THANK YOU SO MUCH FOR THIS PRODUCT. It is life changing 💌

[BZHuntShampu]

Thank you for your detailed answer @aanorlondo ! That seems to be working quite well.

Do we have any insights as to why the ability to process local repositories is gone ?
Is there any plan to bring it back in the future ?

[aanorlondo]

My pleasure @BZHuntShampu :)

Regarding the reasons of why it is not working, the answer should be around:

• Here:

  readme-ai/readmeai/utils.py

  Lines 86 to 100 in 3849ad1

  	def get_user_repository_name(url_or_path) -> str:
  	"""Extract username and repository name from a GitHub URL or local path."""
  	if os.path.exists(url_or_path):
  	return os.path.basename(url_or_path)
  	pattern = r"https?://github.com/([^/]+)/([^/]+)"
  	match = re.match(pattern, url_or_path)
  	if match:
  	username, reponame = match.groups()
  	return f"{username}/{reponame}"
  	else:
  	raise ValueError("Error: invalid remote repository URL or local path.")

• And Here:

  readme-ai/readmeai/utils.py

  Lines 81 to 84 in 3849ad1

  	def get_github_file_link(file: str, user_repo_name: str) -> str:
  	"""Returns the GitHub URL for a given file."""
  	return f"https://github.com/{user_repo_name}/blob/main/{file}"

• Or certainly in that file: utils.py

In my case, the remote repo and the local repo do not have the same name. Maybe it has something to do with it ?
I will try with the latest version of the repo after renaming the local repo 👍

As of when it will be back in the future, I can't answer that, I am not a contributor to the project.
Actually I'm not even sure this is a bug or a misuse of the tool ^^

@eli64s any thoughts ?

[eli64s]

Hey, thanks for raising the issue @BZHuntShampu and the analysis @aanorlondo!

I made a small update to the get_user_repository_name() method and pushed the update in readmeai version 0.3.16. I'm now able to process local repositories on my machine again. However, there is an error in the repository tree generation, which I'll look into when I get some free time. What OS are you guys using?

To get the upgrade, run the following:

pip install --upgrade readmeai

Let me know if this fixes the issue for you all.

[BZHuntShampu]

Great, thanks for the quick update @eli64s ! The updated version accepted my local repository.
You are right that the repository tree seems to be missing.

I am running Kali-Linux over WSL on Windows 11 22H2

[aanorlondo]

Thanks @eli64s for the quick intervention, you rock!

I'll let you know if it fixes the issue when I have some time.
I am running MacOS Monterey v12.5 over a M1-arm64 arch chip

@BZHuntShampu, you could run the latest version to get the last upgrades and the old version to get the tree ;)

[eli64s]

@BZHuntShampu Tree is now working once more for local repositories.

Once again update to the latest version using:

pip install --upgrade readmeai

Cheers 🙂

[BZHuntShampu]

Amazing ! Thank you for your work !