add benchmark for clip model

[wirthual]

PR for issue #259

Add base64 encoded image to test data which can be used to run benchmark without being influenced by download speed.

Run server:

infinity_emb v2 --model-id wkcn/TinyCLIP-ViT-8M-16-Text-3M-YFCC15M

Example run on my local laptop:

make benchmark_embed_vision

ab -n 10000 -c 10 -l -s 480 \
-T 'application/json' \
-p tests/data/benchmark/benchmark_embed_image.json \
http://127.0.0.1:7997/embeddings
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        uvicorn
Server Hostname:        127.0.0.1
Server Port:            7997

Document Path:          /embeddings
Document Length:        Variable

Concurrency Level:      10
Time taken for tests:   229.917 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      111730254 bytes
Total body sent:        2310510000
HTML transferred:       110450254 bytes
Requests per second:    43.49 [#/sec] (mean)
Time per request:       229.917 [ms] (mean)
Time per request:       22.992 [ms] (mean, across all concurrent requests)
Transfer rate:          474.57 [Kbytes/sec] received
                        9813.77 kb/s sent
                        10288.34 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:    72  229  31.0    226     586
Waiting:       72  229  31.0    226     586
Total:         73  230  31.1    226     587

Percentage of the requests served within a certain time (ms)
  50%    226
  66%    232
  75%    236
  80%    239
  90%    249
  95%    267
  98%    296
  99%    378
 100%    587 (longest request)
# sudo apt-get apache2-utils

[greptile-apps]

PR Summary

This pull request adds a benchmark for the CLIP model, addressing issue #259. The main changes include:

• Added a new Makefile target benchmark_embed_vision for benchmarking with base64 encoded images
• Created a new benchmark data file with base64 encoded images to avoid measuring download speed
• Updated the Makefile to include the new benchmark command

The benchmark can be run using the command make benchmark_embed_vision, which uses ApacheBench to test the performance of the CLIP model with image embeddings.

_{1 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile}

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.22%. Comparing base (87131a4) to head (3553456).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #438   +/-   ##
=======================================
  Coverage   79.22%   79.22%           
=======================================
  Files          41       41           
  Lines        3245     3245           
=======================================
  Hits         2571     2571           
  Misses        674      674           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.