Performance improvments

[Rafiot]

Need to be checked.

[coveralls]

Coverage increased (+0.04%) to 38.232% when pulling 2785d00 on perf into 742a579 on master.

[coveralls]

Coverage increased (+0.8%) to 38.948% when pulling a2b66e9 on perf into 742a579 on master.

[coveralls]

Coverage increased (+0.8%) to 38.948% when pulling a2b66e9 on perf into 742a579 on master.

[codecov]

Codecov Report

Merging #470 into master will increase coverage by 0.07%.
The diff coverage is 63.68%.

@@            Coverage Diff             @@
##           master     #470      +/-   ##
==========================================
+ Coverage   35.97%   36.05%   +0.07%     
==========================================
  Files          35       35              
  Lines        6679     6712      +33     
==========================================
+ Hits         2403     2420      +17     
- Misses       4276     4292      +16

Impacted Files	Coverage Δ
tests/testlive_comprehensive.py	0% <0%> (ø)	⬆️
pymisp/tools/sshauthkeyobject.py	38.09% <0%> (ø)	⬆️
pymisp/tools/fileobject.py	30.64% <0%> (ø)	⬆️
pymisp/tools/peobject.py	22.54% <0%> (ø)	⬆️
pymisp/tools/emailobject.py	21.56% <0%> (ø)	⬆️
tests/test.py	0% <0%> (ø)	⬆️
pymisp/tools/machoobject.py	26.86% <0%> (ø)	⬆️
pymisp/tools/elfobject.py	25.71% <0%> (ø)	⬆️
pymisp/api.py	91.42% <100%> (+1.95%)	⬆️
pymisp/__init__.py	83.78% <100%> (ø)	⬆️
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 742a579...2785d00. Read the comment docs.

[mback2k]

@Rafiot Thanks for merging my PR. It looks good so far, but I have some comments you may want to look at.

I am currently testing this PR in our application and will report back about the result soon.

[Rafiot]

Okay, my test cases are passing now... let me know how that goes for you.

[mback2k]

Thanks, with the previous changes we still had huge memory usage. So I am going to give the latest changes another try and will report back later today.

[mback2k]

Sorry for my delayed response regarding my results. Unfortunately I am still analyzing the memory usage, because we are still hitting some limits at the moment. I hope to find the root cause soon.

[Rafiot]

Hmmm that's weird. If you give a try to rapidjson, the memory use was a little bit lower on my end.

[Rafiot]

Ok, I think it is ready to be merged, the tests are passing, and it is already an improvement.

Please let me know if you figure out what is going on on your side, and where we can improve the code from there.

[mback2k]

I am going to test the latest changes now.

[mback2k]

Sorry, I forgot to submit this pending review. Have you seen my comment regarding remote_describe_types = describe_types?