BUG: memory leak in to_json when converting DateTime values

[swt2c]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
while True:
    dr = pd.date_range(start='1/1/2019', end='1/2/2019', freq='s', tz='UTC')
    df = pd.DataFrame({'col1': [12.34]}, index=dr)
    df.reset_index().to_json(orient='values', date_format='iso')

Issue Description

There appears to be a memory leak in Pandas to_json() when converting DateTime values. When running the reproducer, the system's memory use will continuously increase.

Expected Behavior

Memory use should be stable.

Installed Versions

Details

__ INSTALLED VERSIONS ------------------ commit : 4665c10 python : 3.11.13 python-bits : 64 OS : Linux OS-release : 6.14.0-1012-aws Version : #12~24.04.1-Ubuntu SMP Fri Aug 15 00:16:05 UTC 2025 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : C.UTF-8 LOCALE : en_US.UTF-8

pandas : 2.3.2
numpy : 2.3.2
pytz : 2025.2
dateutil : 2.9.0.post0
pip : 24.0
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : None
pyreadstat : None
pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.2
qtpy : None
pyqt5 : None

[Alvaro-Kothe]

I was able to reproduce the memory leak and noticed it doesn't occur when reset_index() is removed.

[swt2c]

Yeah the reset_index() is needed in this particular example because otherwise, the datetime64[ns, UTC] column won't be rendered by to_json().

The memory leak seems to occur when to_json() is handling datetime64[ns, UTC] values.

[swt2c]

Hi @Alvaro-Kothe and @mroeschke, thank you for the fixes. Unfortunately, they did not fix the memory leak that I'm seeing. Can someone reopen this issue please?

The reproducer shown in #62210 unfortunately is slightly different than my use case. Tweaking it to the below will show the issue that I'm still experiencing (note the date_format='iso').

import pandas as pd

for _ in range(10_000):
    df = pd.DataFrame({'col1': [12.34]}, 
                      index=pd.date_range('1/1/2019', '10/1/2019', freq="D", tz="UTC"))
    result = df.reset_index().to_json(date_format='iso')

I investigated the issue myself with valgrind and it shows the following:

==4137== 1,150,800 bytes in 27,400 blocks are definitely lost in loss record 17,727 of 17,727
==4137==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==4137==    by 0x85F5067: PyDateTimeToIso (pd_datetime.c:173)
==4137==    by 0x12B0F29B: PyDateTimeToIsoCallback (objToJSON.c:350)
==4137==    by 0x12B13B1F: Object_getStringValue (objToJSON.c:1891)
==4137==    by 0x12B1664E: encode (ultrajsonenc.c:1084)
==4137==    by 0x12B1635A: encode (ultrajsonenc.c:1027)
==4137==    by 0x12B1635A: encode (ultrajsonenc.c:1027)
==4137==    by 0x12B16C70: JSON_EncodeObject (ultrajsonenc.c:1195)
==4137==    by 0x12B14383: objToJSON (objToJSON.c:2080)
==4137==    by 0x55401A: ??? (in /usr/bin/python3.11)
==4137==    by 0x52ED62: _PyObject_MakeTpCall (in /usr/bin/python3.11)
==4137==    by 0x53C61C: _PyEval_EvalFrameDefault (in /usr/bin/python3.11)

Looking at the code, it initially looks like the fix would be to update PyDateTimeToIsoCallback to match NpyTimeDeltaToIsoCallback where it stores the return char * in GET_TC(tc)->cStr so it can be freed later. However in fb6c4e3, @WillAyd removed the call to PyObject_Free(GET_TC(tc)->cStr); so the cStr will not be freed anyway.

Thus, @WillAyd do you have any thoughts on how this should be resolved?

[Alvaro-Kothe]

Hi @swt2c, thanks for pointing out that the problem still exists. Can you checkout my pull request (#62217) and verify if the memory leak persists?