How should I return multiple images from fast API ?

[yichong96]

Fast API provides documentation on how to return a single image as a file, either by using FileResponse or StreamingResponse.

Is there an example on what would be the best practice to return multiple images using fast API ?

[simondale00]

Hey, @yichong96. This may depend on your use-case.

If you're looking to return N images in an archive, FileResponse is probably still good. You just need to ZIP the images together to prepare the archive before serving the data.

If you're looking to return N actual images, I think your best option is to serve one image at a time through N requests (per FastAPI documentation on how to return an image). A single HTTP response needs to define it's content type, and I'm not aware of any content type that represents more than one image.

A potential third option could be to merge N images into a single image, and return that single image. This could be slow to process, and the client doesn't really have the option to easily chop the merged image back into N separate images.

All this said, the middle option is common practice and likely your best bet.

Best,
Simon

[cvillamizara]

Hey @simondale00 Is there any example you can provide?

[simondale00]

What kind of example are you looking for?

[yichong96]

Hi @simondale00 Thanks for the help ! As I am trying to build an image search service, one request should return multiple images. I won't be able to send N requests on the client side.

With that said, 1st option would be the best bet ?

[simondale00]

Hey @yichong96,

Is this an image search service akin to searching Google Images? Wherein a user searches a keyword in their browser, and the browser then displays search results as images in some kind of grid?

If 'yes', I think the second option is still best with some caveats. You'd probably want an API endpoint to perform the query, the result of which returns a JSON response with an array of image URLs matching the query. You could do something like this:

GET /images?q=some+image+search

HTTP/1.1 200 OK
Content-Type: application/json

{
  "query": "some image search",
  "results": [
      {
          "href": "http://somehost.com/path/to/the/first/result.jpg",
          "other_metadata_property": "some value"
      },
      { ... }
  ]
}

The client can then parse this response and determine how it wants to fetch each result.

I'd then recommend the images be served from as close to the edge as possible. Serving directly from a blob store (AWS S3?) or a CDN (AWS CloudFront, with S3 as an origin?) will be far more performant than having FastAPI fetch and serve each and every file.

Best!
Simon

[vaventt]

Hi, can you help me too, I am getting by FastAPI 1 image from video stream with cars, than I predict on this image with YOLOv8 and also make a bird_view_transformation, in final result I have 2 images, that I want to return + one integer, that represents amount of the car back to the .NET back-end, how can I do this? I can easily return 1 image with Response() converting into bytes, but can't figure out how to return 2 images and also 1 integer, I tried some json, but doesn't work, do you have any recommendation? Or for such task I need to go with FlaskAPI?

[yichong96]

Hey @simondale00 thanks so much for your detailed explanation !!

I'd then recommend the images be served from as close to the edge as possible. Serving directly from a blob store (AWS S3?) or a CDN (AWS CloudFront, with S3 as an origin?) will be far more performant than having FastAPI fetch and serve each and every file

Regarding this statement, the images would be in an object storage using s3 client to download the images. initially I had wanted the fast API server to download the images from the object store into file like objects and then send them over http back to the client.

Would you mind explaining why it would be faster for the client to retrieve the images directly from the object store than for the server to send the images via http ? I am kind of learning as well.

Thank you !!

[simondale00]

Would you mind explaining why it would be faster for the client to retrieve the images directly from the object store than for the server to send the images via http ? I am kind of learning as well.

Absolutely! Consider the two diagrams below illustrating how a client could fetch an image:

# Options 1: Fetching from a CDN
Client --get image--> [CloudFront] --if not in cache--> [S3]

# Option 2: Fetching from FastAPI
Client --get image--> [FastAPI] --get image--> [S3]

Both paths look similar, but they differ in a few ways:

1. CloudFront (or any CDN) will be globally distributed. Your FastAPI instance will only exist where you deploy it. This is impactful as data transfer speeds are limited by the speed of light (amongst other things). For the CDN option, if the client is in Korea, they could hit one of the 7 CloudFront edge locations in Asia; This will be fast. However, for the FastAPI option, if FastAPI exists in Virginia, the data is travelling half way around the world.

2. CloudFront is built as a cache. If the image is in the cache it can be returned immediately without additional network hops. Whereas FastAPI (or any web framework) won't have any cache automatically built-in and will require a second network hop to fetch the image.

3. CloudFront/S3 support significant scale, far more than you can handle with FastAPI alone.

Best!
Simon

[yichong96]

@simondale00 Thank you for your explanation once again !

I was thinking along the lines that because the images would have to be downloaded from the object store by the web server (to process the query and do image comparisons), there would be duplicate work done if the server were to send the url for the search results hosted on the object store rather than just return the already downloaded images.

Im not sure if allowing the client to download the images from object store would be faster as the bottleneck would arise from fast api downloading the images and processing the query ?

I guess it really depends on other factors as well haha.