Skip to content

Looks like json encoder uses huge amount of memory #22

Open
@avi3tal

Description

@avi3tal

I am working with large size json body and i have encountered that python-simple-rest-client just takes around *4 memory than i would expected.

I have decided to reimplement my http client using requests directly and looks like i have solved the problem.

After debugging i cam to realise that the pure Json library is ineffective in terms of memory and this is what caused my memory to blow.

I read your code and looks like you are using json_encoder which also uses the pure json lib (unless you choose to change it - which in your case you didn't).

My recommendation:

  1. Do not use json_encoder as is but change the json lib to more effective one.
  2. maybe try to avoid even loading the json if this is only for the reason of constructing Response class. Maybe you should pass the original response.json() as is instead of converting to your own body() function.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions