Dall-E Image Generator return url without authentication information

[majiayu000]

System Info

MacOS

Who can help?

No response

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

As described in the docs page:

from langchain.agents import initialize_agent, load_tools

tools = load_tools(["dalle-image-generator"])
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)
output = agent.run("Create an image of a halloween night at a haunted museum")

Below is output:

> Entering new AgentExecutor chain...
 I need to generate an image from a text description
Action: Dall-E-Image-Generator
Action Input: "Halloween night at a haunted museum"
Observation: https://oaidalleapiprodscus.blob.core.windows.net/private/org-yt03sAlJZ8YRfqIcNivAAqZu/user-xV6mgISZftMROz9SukNqCHqH/img-EKpmrjqlb1988YrkkBm0vgjr.png?st=2023-12-04T08%3A30%3A50Z&se=2023-12-04T10%3A30%3A50Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-12-03T23%3A02%3A03Z&ske=2023-12-04T23%3A02%3A03Z&sks=b&skv=2021-08-06&sig=9TvprwW3Wl3ZHj%2B2ga6juBT1KQLJIc9TUz%2BDIVcd3XA%3D
Thought: I now know the final answer
Final Answer: https://oaidalleapiprodscus.blob.core.windows.net/private/org-yt03sAlJZ8YRfqIcNivAAqZu/user-xV6mgISZftMROz9SukNqCHqH/img-EKpmrjqlb1988YrkkBm0vgjr.png?st=2023-12-04T08%3A30%3A50Z&se=2023-12-04T10%3A30%3A50Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-12-03T23%3A02%3A03Z&ske=2023-12-04T23%

> Finished chain.

Seems ok? But the url has been cut, it not origin! So you will not get the right picture.

Instead got this:

Expected behavior

Dalle is a import tool in openai.

I want get the picture which I can see it. So it should give me the origin link.

Just like this:

> Entering new AgentExecutor chain...
I can use the Dall-E-Image-Generator to generate an image of a volcano island based on the text description.
Action: Dall-E-Image-Generator
Action Input: "A volcano island"
Observation: https://oaidalleapiprodscus.blob.core.windows.net/private/org-yt03sAlJZ8YRfqIcNivAAqZu/user-xV6mgISZftMROz9SukNqCHqH/img-H3m0wSNxDXVUkUKiE9kOKgvg.png?st=2023-12-04T09%3A39%3A05Z&se=2023-12-04T11%3A39%3A05Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-12-03T22%3A42%3A28Z&ske=2023-12-04T22%3A42%3A28Z&sks=b&skv=2021-08-06&sig=WSEO5/OX5GgYaNTWxZhNmsK%2BqeaDLMEsDdGEnHX18BY%3D
Thought:I now know the final answer.
Final Answer: The image of a volcano island can be found at the following link: https://oaidalleapiprodscus.blob.core.windows.net/private/org-yt03sAlJZ8YRfqIcNivAAqZu/user-xV6mgISZftMROz9SukNqCHqH/img-H3m0wSNxDXVUkUKiE9kOKgvg.png?st=2023-12-04T09%3A39%3A05Z&se=2023-12-04T11%3A39%3A05Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-12-03T22%3A42%3A28Z&ske=2023-12-04T22%3A42%3A28Z&sks=b&skv=2021-08-06&sig=WSEO5/OX5GgYaNTWxZhNmsK%2BqeaDLMEsDdGEnHX18BY%3D

> Finished chain.
The image of a volcano island can be found at the following link: https://oaidalleapiprodscus.blob.core.windows.net/private/org-yt03sAlJZ8YRfqIcNivAAqZu/user-xV6mgISZftMROz9SukNqCHqH/img-H3m0wSNxDXVUkUKiE9kOKgvg.png?st=2023-12-04T09%3A39%3A05Z&se=2023-12-04T11%3A39%3A05Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-12-03T22%3A42%3A28Z&ske=2023-12-04T22%3A42%3A28Z&sks=b&skv=2021-08-06&sig=WSEO5/OX5GgYaNTWxZhNmsK%2BqeaDLMEsDdGEnHX18BY%3D

[dosubot]

🤖

Hello @majiayu000! I'm Dosu, a friendly bot helper for the LangChain repository. I'm here to assist you with bug solving, answering questions, and even getting you started on contributing if you're interested. While we wait for a human maintainer, I'm here to help you.

Thank you for bringing this to our attention. It seems like the issue is with the way the Dall-E Image Generator in LangChain is handling the URL generation. The run method in the DallEAPIWrapper class only extracts the URL from the response data and returns it. It does not handle any authentication information that might be required to access the image at the URL.

Here is the relevant code:

def run(self, query: str) -> str:
    """Run query through OpenAI and parse result."""
    response = self.client.images.generate(
        prompt=query,
        n=self.n,
        size=self.size,
        model=self.model,
        quality=self.quality,
    )
    image_urls = self.separator.join([item.url for item in response.data])
    return image_urls if image_urls else "No image was generated"

In this code, response.data is a list of objects, each containing information about a generated image. The code extracts the URL of each image with item.url and joins them into a string with self.separator.join. However, it does not include any authentication information that might be required to access the image at the URL.

The URL could be cut off due to several reasons:

1. If the path parameters provided do not match the placeholders in the URL, the placeholders will not be replaced and will remain in the URL. This could cause the URL to be invalid and appear to be cut off.

2. If the values of the path parameters are not properly formatted, they could cause the URL to be invalid. For example, if a path parameter value contains special characters that are not URL-encoded, it could break the URL.

3. If there is an error in the _format_url function, it could cause the URL to be cut off. For example, if there is a bug in the regular expression used to find the placeholders, it could cause some placeholders to be missed.

We will look into this issue and try to find a solution. In the meantime, you might want to check the values of the path parameters and make sure they are correctly formatted and URL-encoded. Also, please ensure that the placeholders in the URL match the keys in the path parameters dictionary.

Thank you for your patience and understanding. If you have any other questions or need further clarification, please don't hesitate to ask.

Sources

• libs/langchain/langchain/utilities/dalle_image_generator.py
• libs/langchain/langchain/chains/openai_functions/openapi.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[yuewenma]

@majiayu000 Have you solved this issue? I also encountered the same problem.

[majiayu000]

@majiayu000 Have you solved this issue? I also encountered the same problem.

I did not fix this.

[dave-16]

the Dall-E generated links will only work for 1 hour so what you should do is get the url then download it or if you want to use like firebase you can use the api to uplod to the bucket and get the downloadable url but when uploading to firebase make the expiration date to un expiredable or set duration
`def upload_image_to_firebase(image_url):
try:
# Fetch the image from the URL
response = requests.get(image_url)
if response.status_code != 200:
raise Exception(f"Failed to fetch image from {image_url}")

    # Upload the image to Firebase Storage
    blob = bucket.blob('images/image.jpg')

    # Convert image content to bytes
    image_bytes = BytesIO(response.content)

    # Upload image bytes to Firebase Storage
    blob.upload_from_file(image_bytes, content_type='image/jpeg')

    # Get the download URL with a far-future expiration time (e.g., one year)
    expiration_time = datetime.datetime.utcnow() + datetime.timedelta(days=365*10)  # Set to expire in 10 years
    download_url = blob.generate_signed_url(expiration=expiration_time, method='GET')
    
    return download_url`