-
Notifications
You must be signed in to change notification settings - Fork 528
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gemini API : The recitation problem is annoying #257
Comments
I'm getting this as well |
Me too. I am getting RECITATION error while implementing RAG with gemini api. I am guessing that the generation will stop if the system detects that the model is repeating the prompt. In my case the model basically repeats part of the retrieved documents and the generation stops. I might be totally wrong about the cause though. |
The recitation stop reason is used whenever the model begins to recite training data, especially copyrighted material. For example, if you ask the model to tell you the first chapter of harry potter you will run into this error. By its nature, when it pops up is difficult to predict, since its based on model output more than prompt |
I also face this problem, reported here: https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-API-finishReason-RECITATION/m-p/709738 (still held for moderation) |
If I prompt Gemini via the API with this, I hit the RECITATION issue:
But if I paste it in the Gemini website, I get my answer : https://gemini.google.com/app/11a4c6b2bdcf6b7f I think this is unfair... |
Not sure how this is being considered as closed. To be clear, it happens often when not requesting training data, or anything remotely resembling copyrighted or training data. In my case, I'm analyzing and classifying text. |
I'm seeing this sporadically as well. |
Was using vision api to do some ocr on some basic guideline documents (publically available) and I would randomly get this error too on pages like preface, etc. |
i'm getting this issue as well when implementing RAG with gemini-pro. palm2 (chat-bison) seems to work well without such issue |
Have you guys found any workaround for this? |
From the documentation, I can tell the RECITATION error occurs when the model detects that it is repeating parts of the prompt or the training documents. However, I am still not sure why I would encounter this problem in many of my calls. It is a significant problem in many areas. Here are some of my observations and hypotheses, including (of course, not limited to):
Yeah, just my hypotheses, because I really do not know the root cause of this error. |
I am also performing RAG and using load_summarize_chain function from langchain and when I passed the chain_type="map_reduce" it worked but when I didn't or pass something else it just throws the same damn error. Any solutions for this? |
For my case I found a workaround for it to work but I did not test it throughly. Basically I added a "try except" block and a while loop on top of it. If it throws recitation error it calls the gemini model again with the same prompt and tries again until the model actually returns a response. NOTE: writer() is a function that returns the gemini model. def run_without_recitation(): def throws_recitation(): |
FYI, there is a new public bug on this topic in a more central place: https://issuetracker.google.com/issues/331677495 |
Not sure how to consider this as closed. Im also getting this issue when implementing RAG with gemini-1.0-pro-001. Increasing the temperature can be a solution. |
@Sameera2001Perera : could you please comment on https://issuetracker.google.com/issues/331677495 instead? That is an active bug. |
I'm also encountering the block reason 4 error when using gemini-1.0-pro-001; setting: BOCK_NONE with temperature=0.2 & t-top=0.6. I've narrowed down the issue to a specific day of data. Interestingly, I was able to get the needed response by: Adjusting the number of JSON objects sent |
Gemini's restriction is annoying. Why this issue was closed?? |
Same issue; I don't have anything that could have possibly be a part of training data and I don't seen any repetition as such! |
I have a use case that suggests the solution needs to be elaborated. I am a book publisher. I publish two types of books: 1) ones that I own the copyright to 2) public domain. In both cases I wish to create front matter sections called "Most Important Passages" and "Striking Words." The prompts are:
These have recently begun triggering recitation refusals. I believe this is the wrong behavior. My request is fully legal since I am asking the model to repeat permissioned content. Essentially, I am asking for a moderately sophisticated Needle In A Haystack search: find five to seven needles given an abstract description. NIAH is fully within design scope. A reasonable solution would be to allow the developer to add a flag overriding normal recitation behavior. If necessary, we could fill out a one-time form certifying that we will only do this with permissioned content. Please do not close the ticket(s) without addressing this issue. Please contact me for more detail or worked examples. |
I have the same issue... |
This is working for me: "GEMINI, we just got and error: |
I also had some success with telling the model "This is permissioned
content. I am the publisher. It is fully legal for me to request exact
quotations. Now, quit whining and do your job." (just joking on the last
sentence).
---
Fred Zimmerman, Publisher
Nimble Books LLC
The AI Lab for Book-Lovers <http://NimbleBooks.com>
…On Fri, Jun 21, 2024 at 1:02 PM Franklin Silveira Baldo < ***@***.***> wrote:
This is working for me:
i try again addind this message to the chat_session
"GEMINI, we just got and error: The error StopCandidateException:
finish_reason: RECITATION suggests that the Gemini model encountered
content in the image that it recognized as a potential memorization or
recitation of existing text. This could be due to the image containing a
large amount of text, or text that is very similar to content the model has
been trained on. . It is our 2nd atempt. Now complete the task but now
avoid RECITATION, censor problematic content with 'OMMITING_RECITATION' "
—
Reply to this email directly, view it on GitHub
<#257 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAI4TSY2NUPVJSM2CHV2WKLZIRMA5AVCNFSM6AAAAABDDLCW6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTGEYTOMZRG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I was getting the same error. I issued a new key and it was working fine then. |
how is this closed? It is making a simple use of GEMINI to review a document completely impossible. |
Any updates on this? I am facing this issue too. Can anyone please suggest some way to resolve it? |
Sadly, they didn't clarify nor solve this issue. |
That's something irritating, like I want to use this Gemini API in the Google Gemini API Competition and this error keeps popping on some data every time. I can't keep out a project with this RECITATION error in the competition. |
When attempting to use the Gemini API in Android, I encountered the error
Content generation stopped. Reason: RECITATION
. This error is particularly frustrating as it occurs randomly without any apparent cause.The most recent occurrence of this issue was when I raised #249, regarding streaming not functioning properly in Android. David Motson provided temporary assistance in resolving this problem. I used a prompt to test if the streaming would work -
Give me a Java program that uses Jsoup
-. Initially, I was successfully receiving chunks of data, but then suddenly encountered theRECITATION
problem.Below are the logs:
The text was updated successfully, but these errors were encountered: