bug: non batched request executed in non FIFO manier

[stream129]

Describe the bug

In a configuration with a not embedded runner, without batching, without parallelization of workers, requests must be executed one after another according to the FIFO principle. At the moment requests are executed in LIFO mode. In some cases, this causes earlier requests to time out because they do not get a chance to execute in the required time.

I made a workaround for myself by replacing _queue.pop() with _queue.popleft() in the code below:

BentoML/src/bentoml/_internal/marshal/dispatcher.py

Line 263 in eb0ad1a

inputs_info = tuple(self._queue.pop() for _ in range(n_call_out))

and

BentoML/src/bentoml/_internal/marshal/dispatcher.py

Line 361 in eb0ad1a

inputs_info = tuple(self._queue.pop() for _ in range(n_call_out))

I'm not sure that this workaround doesn't have any side effects. But for me it works.

To reproduce

1. Make a simple service with a custom runner.
   Set the runner parameters:
   SUPPORTED RESOURCES = "nvidia.com/gpu"
   SUPPORTS CPU MULTITHREADING = True
   And BentoML configuration:

api_server:
  traffic:
    timeout: 30
  workers: 1

runners:
  traffic:
    max_concurrency: 3

2. Make sure the runner method is performed for a sufficient amount of time, such as 10 seconds.
3. Start bentoml server. Execute several requests to your runner's API one after another, but without waiting for the response from the previous launch. I used several regular browser tabs.
4. The server response is returned out of order. For example, you made 3 requests. But the answer will come first for the first request, then for the third, and only then for the second request.

Expected behavior

The BentoML server must respond in the order in which requests are received.

Environment

Environment variable

BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information

bentoml: 1.2.2
python: 3.10.13
platform: Linux-5.15.0-100-generic-x86_64-with-glibc2.35
uid_gid: 1000:1000
conda: 23.11.0
in_conda_env: True

conda_packages

name: NDA
channels:
  - pytorch
  - nvidia
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - accelerate=0.27.0=pyhd8ed1ab_0
  - aiohttp=3.9.3=py310h2372a71_1
  - aiosignal=1.3.1=pyhd8ed1ab_0
  - annotated-types=0.6.0=pyhd8ed1ab_0
  - anyio=4.3.0=pyhd8ed1ab_0
  - appdirs=1.4.4=pyh9f0ad1d_0
  - argon2-cffi=23.1.0=pyhd8ed1ab_0
  - argon2-cffi-bindings=21.2.0=py310h2372a71_4
  - asgiref=3.7.2=pyhd8ed1ab_0
  - async-timeout=4.0.3=pyhd8ed1ab_0
  - attrs=23.2.0=pyh71513ae_0
  - aws-c-auth=0.7.16=haed3651_8
  - aws-c-cal=0.6.10=ha9bf9b1_2
  - aws-c-common=0.9.14=hd590300_0
  - aws-c-compression=0.2.18=h4466546_2
  - aws-c-event-stream=0.4.2=he635cd5_6
  - aws-c-http=0.8.1=hbfc29b2_7
  - aws-c-io=0.14.6=h6b388c4_1
  - aws-c-mqtt=0.10.3=hffff1cc_2
  - aws-c-s3=0.5.2=h4893938_2
  - aws-c-sdkutils=0.1.15=h4466546_2
  - aws-checksums=0.1.18=h4466546_2
  - aws-crt-cpp=0.26.3=h137ae52_2
  - aws-sdk-cpp=1.11.267=he0cb598_3
  - bentoml=1.2.2=pyhd8ed1ab_0
  - blas=1.0=mkl
  - brotli-python=1.1.0=py310hc6cd4ac_1
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.27.0=hd590300_0
  - ca-certificates=2024.2.2=hbcca054_0
  - cattrs=23.1.2=pyhd8ed1ab_0
  - certifi=2024.2.2=pyhd8ed1ab_0
  - cffi=1.16.0=py310h2fee648_0
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - circus=0.18.0=pyhd8ed1ab_0
  - click=8.1.7=unix_pyh707e725_0
  - click-option-group=0.5.6=pyhd8ed1ab_0
  - cloudpickle=3.0.0=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - contextlib2=21.6.0=pyhd8ed1ab_0
  - cuda-cudart=12.1.105=0
  - cuda-cupti=12.1.105=0
  - cuda-libraries=12.1.0=0
  - cuda-nvrtc=12.1.105=0
  - cuda-nvtx=12.1.105=0
  - cuda-opencl=12.4.99=0
  - cuda-runtime=12.1.0=0
  - datasets=2.18.0=pyhd8ed1ab_0
  - deepmerge=1.1.1=pyhd8ed1ab_0
  - deprecated=1.2.14=pyh1a96a4e_0
  - diffusers=0.26.3=pyhd8ed1ab_0
  - dill=0.3.8=pyhd8ed1ab_0
  - exceptiongroup=1.2.0=pyhd8ed1ab_2
  - ffmpeg=4.3=hf484d3e_0
  - filelock=3.13.1=pyhd8ed1ab_0
  - freetype=2.12.1=h267a509_2
  - frozenlist=1.4.1=py310h2372a71_0
  - fs=2.4.16=pyhd8ed1ab_0
  - fsspec=2024.2.0=pyhca7485f_0
  - gflags=2.2.2=he1b5a44_1004
  - glog=0.7.0=hed5481d_0
  - gmp=6.3.0=h59595ed_1
  - gmpy2=2.1.2=py310h3ec546c_1
  - gnutls=3.6.13=h85f3911_1
  - h11=0.14.0=pyhd8ed1ab_0
  - h2=4.1.0=pyhd8ed1ab_0
  - hpack=4.0.0=pyh9f0ad1d_0
  - httpcore=1.0.4=pyhd8ed1ab_0
  - httpx=0.27.0=pyhd8ed1ab_0
  - huggingface_hub=0.21.4=pyhd8ed1ab_0
  - hyperframe=6.0.1=pyhd8ed1ab_0
  - icu=73.2=h59595ed_0
  - idna=3.6=pyhd8ed1ab_0
  - importlib-metadata=6.0.0=pyha770c72_0
  - inflection=0.5.1=pyh9f0ad1d_0
  - intel-openmp=2022.1.0=h9e868ea_3769
  - jinja2=3.1.3=pyhd8ed1ab_0
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.21.2=h659d440_0
  - lame=3.100=h166bdaf_1003
  - lcms2=2.16=hb7c19ff_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - lerc=4.0.0=h27087fc_0
  - libabseil=20240116.1=cxx17_h59595ed_2
  - libarrow=15.0.1=h6bfc85a_2_cpu
  - libarrow-acero=15.0.1=h59595ed_2_cpu
  - libarrow-dataset=15.0.1=h59595ed_2_cpu
  - libarrow-flight=15.0.1=hc6145d9_2_cpu
  - libarrow-flight-sql=15.0.1=h757c851_2_cpu
  - libarrow-gandiva=15.0.1=hb016d2e_2_cpu
  - libarrow-substrait=15.0.1=h757c851_2_cpu
  - libblas=3.9.0=16_linux64_mkl
  - libbrotlicommon=1.1.0=hd590300_1
  - libbrotlidec=1.1.0=hd590300_1
  - libbrotlienc=1.1.0=hd590300_1
  - libcblas=3.9.0=16_linux64_mkl
  - libcrc32c=1.1.2=h9c3ff4c_0
  - libcublas=12.1.0.26=0
  - libcufft=11.0.2.4=0
  - libcufile=1.9.0.20=0
  - libcurand=10.3.5.119=0
  - libcurl=8.6.0=hca28451_0
  - libcusolver=11.4.4.55=0
  - libcusparse=12.0.2.55=0
  - libdeflate=1.19=hd590300_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=hd590300_2
  - libevent=2.1.12=hf998b51_1
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=13.2.0=h807b86a_5
  - libgomp=13.2.0=h807b86a_5
  - libgoogle-cloud=2.22.0=h9be4e54_1
  - libgoogle-cloud-storage=2.22.0=hc7a4891_1
  - libgrpc=1.62.1=h15f2491_0
  - libiconv=1.17=hd590300_2
  - libjpeg-turbo=3.0.0=hd590300_1
  - liblapack=3.9.0=16_linux64_mkl
  - libllvm16=16.0.6=hb3ce162_3
  - libnghttp2=1.58.0=h47da74e_1
  - libnl=3.9.0=hd590300_0
  - libnpp=12.0.2.50=0
  - libnsl=2.0.1=hd590300_0
  - libnvjitlink=12.1.105=0
  - libnvjpeg=12.1.1.14=0
  - libparquet=15.0.1=h352af49_2_cpu
  - libpng=1.6.43=h2797004_0
  - libprotobuf=4.25.3=h08a7969_0
  - libre2-11=2023.09.01=h5a48ba9_2
  - libsodium=1.0.18=h36c2ea0_1
  - libsqlite=3.45.2=h2797004_0
  - libssh2=1.11.0=h0841786_0
  - libstdcxx-ng=13.2.0=h7e041cc_5
  - libthrift=0.19.0=hb90f79a_1
  - libtiff=4.6.0=ha9c0a0a_2
  - libutf8proc=2.8.0=h166bdaf_0
  - libuuid=2.38.1=h0b41bf4_0
  - libwebp-base=1.3.2=hd590300_0
  - libxcb=1.15=h0b41bf4_0
  - libxcrypt=4.4.36=hd590300_1
  - libxml2=2.12.5=h232c23b_0
  - libzlib=1.2.13=hd590300_5
  - llvm-openmp=15.0.7=h0cdce71_0
  - lz4-c=1.9.4=hcb278e6_0
  - markdown-it-py=3.0.0=pyhd8ed1ab_0
  - markupsafe=2.1.5=py310h2372a71_0
  - mdurl=0.1.2=pyhd8ed1ab_0
  - minio=7.2.4=pyhd8ed1ab_0
  - mkl=2022.1.0=hc2b9512_224
  - mpc=1.3.1=hfe3b2da_0
  - mpfr=4.2.1=h9458935_0
  - mpmath=1.3.0=pyhd8ed1ab_0
  - multidict=6.0.5=py310h2372a71_0
  - multiprocess=0.70.16=py310h2372a71_0
  - ncurses=6.4=h59595ed_2
  - nettle=3.6=he412f7d_0
  - networkx=3.2.1=pyhd8ed1ab_0
  - numpy=1.26.4=py310hb13e2d6_0
  - nvidia-ml-py=11.525.131=pyhd8ed1ab_0
  - openh264=2.1.1=h780b84a_0
  - openjpeg=2.5.2=h488ebb8_0
  - openssl=3.2.1=hd590300_0
  - opentelemetry-api=1.20.0=pyhd8ed1ab_0
  - opentelemetry-instrumentation=0.41b0=pyhd8ed1ab_0
  - opentelemetry-instrumentation-aiohttp-client=0.41b0=pyhd8ed1ab_0
  - opentelemetry-instrumentation-asgi=0.41b0=pyhd8ed1ab_0
  - opentelemetry-sdk=1.20.0=pyhd8ed1ab_0
  - opentelemetry-semantic-conventions=0.41b0=pyhd8ed1ab_0
  - opentelemetry-util-http=0.41b0=pyhd8ed1ab_0
  - orc=2.0.0=h1e5e2c1_0
  - packaging=24.0=pyhd8ed1ab_0
  - pandas=2.2.1=py310hcc13569_0
  - pathspec=0.12.1=pyhd8ed1ab_0
  - pillow=10.2.0=py310h01dd4db_0
  - pip=24.0=pyhd8ed1ab_0
  - pip-requirements-parser=32.0.1=pyhd8ed1ab_0
  - pip-tools=7.4.1=pyhd8ed1ab_0
  - prometheus_client=0.20.0=pyhd8ed1ab_0
  - psutil=5.9.8=py310h2372a71_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - pyarrow=15.0.1=py310hf9e7431_2_cpu
  - pyarrow-hotfix=0.6=pyhd8ed1ab_0
  - pycparser=2.21=pyhd8ed1ab_0
  - pycryptodome=3.20.0=py310hb0f0acc_0
  - pydantic=2.6.3=pyhd8ed1ab_0
  - pydantic-core=2.16.3=py310hcb5633a_0
  - pygments=2.17.2=pyhd8ed1ab_0
  - pyparsing=3.1.2=pyhd8ed1ab_0
  - pyproject_hooks=1.0.0=pyhd8ed1ab_0
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.10.13=hd12c33a_1_cpython
  - python-build=1.1.1=pyhd8ed1ab_0
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python-json-logger=2.0.7=pyhd8ed1ab_0
  - python-multipart=0.0.9=pyhd8ed1ab_0
  - python-tzdata=2024.1=pyhd8ed1ab_0
  - python-xxhash=3.4.1=py310h2372a71_0
  - python_abi=3.10=4_cp310
  - pytorch=2.2.0=py3.10_cuda12.1_cudnn8.9.2_0
  - pytorch-cuda=12.1=ha16c6d3_5
  - pytorch-mutex=1.0=cuda
  - pytz=2024.1=pyhd8ed1ab_0
  - pyyaml=6.0.1=py310h2372a71_1
  - pyzmq=25.1.2=py310h795f18f_0
  - rdma-core=50.0=hd3aeb46_1
  - re2=2023.09.01=h7f4b329_2
  - readline=8.2=h8228510_1
  - regex=2023.12.25=py310h2372a71_0
  - requests=2.31.0=pyhd8ed1ab_0
  - rich=13.7.1=pyhd8ed1ab_0
  - s2n=1.4.7=h06160fa_0
  - safetensors=0.4.2=py310hcb5633a_0
  - schema=0.7.5=pyhd8ed1ab_1
  - setuptools=69.2.0=pyhd8ed1ab_0
  - simple-di=0.1.5=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=h9fff704_0
  - sniffio=1.3.1=pyhd8ed1ab_0
  - starlette=0.37.2=pyhd8ed1ab_0
  - sympy=1.12=pypyh9d50eac_103
  - tk=8.6.13=noxft_h4845f30_101
  - tokenizers=0.15.2=py310h320607d_0
  - tomli=2.0.1=pyhd8ed1ab_0
  - torchaudio=2.2.0=py310_cu121
  - torchtriton=2.2.0=py310
  - torchvision=0.17.0=py310_cu121
  - tornado=6.4=py310h2372a71_0
  - tqdm=4.66.2=pyhd8ed1ab_0
  - transformers=4.37.2=pyhd8ed1ab_0
  - typing-extensions=4.10.0=hd8ed1ab_0
  - typing_extensions=4.10.0=pyha770c72_0
  - tzdata=2024a=h0c530f3_0
  - ucx=1.15.0=h11edf95_7
  - urllib3=2.2.1=pyhd8ed1ab_0
  - uvicorn=0.28.0=py310hff52083_0
  - watchfiles=0.21.0=py310hcb5633a_0
  - wheel=0.42.0=pyhd8ed1ab_0
  - wrapt=1.16.0=py310h2372a71_0
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xxhash=0.8.2=hd590300_0
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - yarl=1.9.4=py310h2372a71_0
  - zeromq=4.3.5=h59595ed_1
  - zipp=3.17.0=pyhd8ed1ab_0
  - zlib=1.2.13=hd590300_5
  - zstd=1.5.5=hfc55251_0
prefix: NDA

pip_packages

accelerate @ file:///home/conda/feedstock_root/build_artifacts/accelerate_1707501497624/work
aiohttp @ file:///home/conda/feedstock_root/build_artifacts/aiohttp_1710511536979/work
aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
annotated-types @ file:///home/conda/feedstock_root/build_artifacts/annotated-types_1696634205638/work
anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1708355285029/work
appdirs @ file:///home/conda/feedstock_root/build_artifacts/appdirs_1603108395799/work
argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1692818318753/work
argon2-cffi-bindings @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi-bindings_1695386546427/work
asgiref @ file:///home/conda/feedstock_root/build_artifacts/asgiref_1685243452975/work
async-timeout @ file:///home/conda/feedstock_root/build_artifacts/async-timeout_1691763562544/work
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
bentoml @ file:///home/conda/feedstock_root/build_artifacts/bentoml_1707123960469/work
Brotli @ file:///home/conda/feedstock_root/build_artifacts/brotli-split_1695989787169/work
build @ file:///home/conda/feedstock_root/build_artifacts/python-build_1709251527949/work
cattrs @ file:///home/conda/feedstock_root/build_artifacts/cattrs_1685682319569/work
certifi @ file:///home/conda/feedstock_root/build_artifacts/certifi_1707022139797/work/certifi
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1696001684923/work
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1698833585322/work
circus @ file:///home/conda/feedstock_root/build_artifacts/circus_1668846026855/work
click @ file:///home/conda/feedstock_root/build_artifacts/click_1692311806742/work
click-option-group @ file:///home/conda/feedstock_root/build_artifacts/click-option-group_1686394190925/work
cloudpickle @ file:///home/conda/feedstock_root/build_artifacts/cloudpickle_1697464713350/work
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1666700638685/work
contextlib2 @ file:///home/conda/feedstock_root/build_artifacts/contextlib2_1624848568296/work
datasets @ file:///home/conda/feedstock_root/build_artifacts/datasets_1709395865330/work
deepmerge @ file:///home/conda/feedstock_root/build_artifacts/deepmerge_1702941685750/work
Deprecated @ file:///home/conda/feedstock_root/build_artifacts/deprecated_1685233314779/work
diffusers @ file:///home/conda/feedstock_root/build_artifacts/diffusers_1707822915454/work
dill @ file:///home/conda/feedstock_root/build_artifacts/dill_1706434688412/work
exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1704921103267/work
filelock @ file:///home/conda/feedstock_root/build_artifacts/filelock_1698714947081/work
frozenlist @ file:///home/conda/feedstock_root/build_artifacts/frozenlist_1702645481127/work
fs @ file:///home/conda/feedstock_root/build_artifacts/fs_1683650158618/work
fsspec @ file:///home/conda/feedstock_root/build_artifacts/fsspec_1707102468451/work
gmpy2 @ file:///home/conda/feedstock_root/build_artifacts/gmpy2_1666808654411/work
h11 @ file:///home/conda/feedstock_root/build_artifacts/h11_1664132893548/work
h2 @ file:///home/conda/feedstock_root/build_artifacts/h2_1634280454336/work
hpack==4.0.0
httpcore @ file:///home/conda/feedstock_root/build_artifacts/httpcore_1708529260247/work
httpx @ file:///home/conda/feedstock_root/build_artifacts/httpx_1708530890843/work
huggingface_hub @ file:///home/conda/feedstock_root/build_artifacts/huggingface_hub_1709810635566/work
hyperframe @ file:///home/conda/feedstock_root/build_artifacts/hyperframe_1619110129307/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1701026962277/work
importlib-metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1672612343532/work
inflection @ file:///home/conda/feedstock_root/build_artifacts/inflection_1598089801258/work
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1704966972576/work
markdown-it-py @ file:///home/conda/feedstock_root/build_artifacts/markdown-it-py_1686175045316/work
MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1706899921127/work
mdurl @ file:///home/conda/feedstock_root/build_artifacts/mdurl_1704317613764/work
minio @ file:///home/conda/feedstock_root/build_artifacts/minio_1707620722786/work
mpmath @ file:///home/conda/feedstock_root/build_artifacts/mpmath_1678228039184/work
multidict @ file:///home/conda/feedstock_root/build_artifacts/multidict_1707040698785/work
multiprocess @ file:///home/conda/feedstock_root/build_artifacts/multiprocess_1706514640841/work
networkx @ file:///home/conda/feedstock_root/build_artifacts/networkx_1698504735452/work
numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1707225380409/work/dist/numpy-1.26.4-cp310-cp310-linux_x86_64.whl#sha256=51131fd8fc130cd168aecaf1bc0ea85f92e8ffebf211772ceb16ac2e7f10d7ca
nvidia-ml-py @ file:///home/conda/feedstock_root/build_artifacts/nvidia-ml-py_1688171990873/work
opentelemetry-api @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-api_1693876773665/work
opentelemetry-instrumentation @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-instrumentation_1694479097639/work
opentelemetry-instrumentation-aiohttp-client @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-instrumentation-aiohttp-client_1694590957674/work
opentelemetry-instrumentation-asgi @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-instrumentation-asgi_1694598742186/work
opentelemetry-sdk @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-sdk_1694936834333/work
opentelemetry-semantic-conventions @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-semantic-conventions_1694312669793/work
opentelemetry-util-http @ file:///home/conda/feedstock_root/build_artifacts/opentelemetry-util-http_1694494133477/work
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1710075952259/work
pandas @ file:///home/conda/feedstock_root/build_artifacts/pandas_1708708607448/work
pathspec @ file:///home/conda/feedstock_root/build_artifacts/pathspec_1702249949303/work
pillow @ file:///home/conda/feedstock_root/build_artifacts/pillow_1704252020178/work
pip-requirements-parser @ file:///home/conda/feedstock_root/build_artifacts/pip-requirements-parser_1672265598496/work
pip-tools @ file:///home/conda/feedstock_root/build_artifacts/pip-tools_1709736059034/work
prometheus_client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1707932675456/work
psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1705722392846/work
pyarrow==15.0.1
pyarrow-hotfix @ file:///home/conda/feedstock_root/build_artifacts/pyarrow-hotfix_1700596371886/work
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
pycryptodome @ file:///home/conda/feedstock_root/build_artifacts/pycryptodome_1704930718567/work
pydantic @ file:///home/conda/feedstock_root/build_artifacts/pydantic_1709075187841/work
pydantic_core @ file:///home/conda/feedstock_root/build_artifacts/pydantic-core_1708700740324/work
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1700607939962/work
pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1709721012883/work
pyproject_hooks @ file:///home/conda/feedstock_root/build_artifacts/pyproject_hooks_1670268672584/work
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1709299778482/work
python-json-logger @ file:///home/conda/feedstock_root/build_artifacts/python-json-logger_1677079630776/work
python-multipart @ file:///home/conda/feedstock_root/build_artifacts/python-multipart_1707760088566/work
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1706886791323/work
PyYAML @ file:///home/conda/feedstock_root/build_artifacts/pyyaml_1695373428874/work
pyzmq @ file:///home/conda/feedstock_root/build_artifacts/pyzmq_1701783169502/work
regex @ file:///home/conda/feedstock_root/build_artifacts/regex_1703393490683/work
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1684774241324/work
rich @ file:///home/conda/feedstock_root/build_artifacts/rich-split_1709150387247/work/dist
safetensors @ file:///home/conda/feedstock_root/build_artifacts/safetensors_1707377218239/work
schema @ file:///home/conda/feedstock_root/build_artifacts/schema_1684241031536/work
simple-di @ file:///home/conda/feedstock_root/build_artifacts/simple-di_1656418562483/work
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1708952932303/work
starlette @ file:///home/conda/feedstock_root/build_artifacts/starlette-recipe_1709667058396/work
sympy @ file:///home/conda/feedstock_root/build_artifacts/sympy_1684180540116/work
tokenizers @ file:///home/conda/feedstock_root/build_artifacts/tokenizers_1707733850559/work/bindings/python
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1644342247877/work
torch==2.2.0
torchaudio==2.2.0
torchvision==0.17.0
tornado @ file:///home/conda/feedstock_root/build_artifacts/tornado_1708363098266/work
tqdm @ file:///home/conda/feedstock_root/build_artifacts/tqdm_1707598593068/work
transformers @ file:///home/conda/feedstock_root/build_artifacts/transformers_1707370358661/work
triton==2.2.0
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1708904622550/work
tzdata @ file:///home/conda/feedstock_root/build_artifacts/python-tzdata_1707747584337/work
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1708239446578/work
uvicorn @ file:///home/conda/feedstock_root/build_artifacts/uvicorn-split_1710145967899/work
watchfiles @ file:///home/conda/feedstock_root/build_artifacts/watchfiles_1701077795738/work
wrapt @ file:///home/conda/feedstock_root/build_artifacts/wrapt_1699532811524/work
xxhash @ file:///home/conda/feedstock_root/build_artifacts/python-xxhash_1696486308932/work
yarl @ file:///home/conda/feedstock_root/build_artifacts/yarl_1705508292061/work
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1695255097490/work

[frostming]

This looks like a careless mistake, can you submit a PR to fix these two places?