Bug tojson memleak

[vernetya]

• closes BUG: to_json memory leak (introduced in 1.1.0) #43877
• tests passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

Fix memory leak when calling to_json appeared in 1.1.0. From these changes, in get_values function, it looks like values from here are not cleaned in this path.

I run the following loop and plot the memory usage using memory_profiler:

def foo():
    data = {str(c): list(range(100_000)) for c in range(10)}
    for i in range(500):
        print(i)
        df = pd.DataFrame(data)
        df.to_json(orient='split', indent=0)

Here on linux (python 3.8.10)
pandas/main:

with changes in this PR:

On Windows 10 (python 3.8.5):
pandas/main:

with changes in this PR:

I run under valgrind (10 loops), less revealing but there're diff:
pandas/main:

...
==3116== LEAK SUMMARY:
==3116==    definitely lost: 94,024 bytes in 583 blocks
==3116==    indirectly lost: 7,248,888 bytes in 537 blocks
==3116==      possibly lost: 5,900,491 bytes in 40,529 blocks
==3116==    still reachable: 10,492,550 bytes in 67,907 blocks
==3116==                       of which reachable via heuristic:
==3116==                         stdstring          : 2,484 bytes in 62 blocks
==3116==         suppressed: 0 bytes in 0 blocks
==3116== Reachable blocks (those to which a pointer was found) are not shown.
==3116== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3116== 
==3116== For lists of detected and suppressed errors, rerun with: -s
==3116== ERROR SUMMARY: 6706 errors from 6703 contexts (suppressed: 4 from 4)

with changes in this PR:

...
==3026== LEAK SUMMARY:
==3026==    definitely lost: 92,296 bytes in 565 blocks
==3026==    indirectly lost: 47,232 bytes in 483 blocks
==3026==      possibly lost: 5,099,615 bytes in 40,511 blocks
==3026==    still reachable: 10,492,630 bytes in 67,908 blocks
==3026==                       of which reachable via heuristic:
==3026==                         stdstring          : 2,484 bytes in 62 blocks
==3026==         suppressed: 0 bytes in 0 blocks
==3026== Reachable blocks (those to which a pointer was found) are not shown.
==3026== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3026== 
==3026== For lists of detected and suppressed errors, rerun with: -s
==3026== ERROR SUMMARY: 6698 errors from 6695 contexts (suppressed: 8 from 4)

[WillAyd]

@vernetya great find! I think this will end up making its way into 1.4.1 whenever the whatsnew for that gets created. Code-wise lgtm @jreback @jbrockmendel @simonjayhawkins

[jreback]

@vernetya can you move to 1.4 (if you can do tonight will merge) otherwise can put in 1.4.1

[jreback]

@simonjayhawkins not a blocker for release (we can put in 1.4.1 as well)

[jreback]

thanks @vernetya

[jreback]

@meeseeksdev backport 1.4.x

[lumberbot-app]

Something went wrong ... Please have a look at my logs.

[vernetya]

Thanks for the backport