ChatGPT is able to access private GitHub repositories!

[rishideepc]

Describe the bug

I tried to ask ChatGPT for information about a private GitHub repository of mine. To my amazement, ChatGPT was able to tell me about the repository perfectly well, with all the details. What is concerning is that it is able to access a private GitHub repository, which shouldn't happen, as it might be looked at as a 'breach of privacy' by others.

To Reproduce

1. Enter the prompt: "Tell me about the GitHub repo: repoName"
2. Then on the prompted to enter username: "Username of Repository owner: userName"
3. ChatGpt is able to access the Github repository, and provide information, even though the repository is private.

Code snippets

No response

OS

Windows 11

Python version

Python v3.7.1

Library version

openai-python v0.26.4

[ceseale]

Hello @rishideepc, thank you for bringing this to our attention! We truly appreciate your vigilance. Please be assured that the model is unable to access private repositories. The output you encountered was a creative fabrication by the model.

[rishideepc]

Hey @ceseale! Thanks for the response.

I understand that it is fairly common for generative models to have such fabrications from time-to-time. And I also see that not all iterations of a particular prompt generate the same response. Which is why I double-checked before opening this issue.

But the details are scarily accurate! I mean it can even guess the tech-stack just by prompting name of the repo and the repo owner. And the repo name ain't even that generic.

[rishideepc]

Also, I believe the model cannot access the net? But my repo was created in Feb 2022, that is, after ChatGPT was trained!