Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Improving List/PList and friends #422
I started looking into this because I ran across the
The "easy" stuff:
The "medium" stuff:
The "hard" stuff:
One is whether we could implement a
Second is whether we could lean more heavily on move semantics and potentially mitigate those performance issues. I have a feeling that this would lead to a lot more work though, and I'm not entirely sure how we'd go about implementing it just yet.
Note that with my comment about
Sounds good to me -- one of those things that's technically a breaking change, but I expect more people are using the common typedefs directly. e.g. everyone uses
Yeah, would be a nice improvement.
I only did a quick skim, but didn't look like a whole lot of usages. Most being part of internal implementation details or in the parts of the "API that's public, but not really intended/expected to be used by others". I also don't expect users to be using
And for judging performance difference in this case, some quick before/after timing measurements on a representative pcap may be a sufficient sanity check.
Don't have any profiling info on the difference, might be fun to check that against our default scripts, but ultimately, usage patterns for lists/dictionaries can depend on a user's custom scripts. Who knows what crazy load they have or thing they want to do. Makes it hard to definitely declare a Best implementation, so probably the general idea should be "let's just not implement it in a way that's difficult to customize/tune later".
Not sure exactly what you're thinking here. If it's like moving away from "containers of pointers" to "containers of move-able objects", my hunch is that would be Good, but yeah, a lot more work since it's maybe all wrapped up in the general notion of changing "how we do memory management".
I would like to benchmark Dict.cc against https://github.com/google/hashtable-benchmarks but since Dict.cc doesn't implement the right interfaces doing so isn't easy.
As a first test of replacing dictionary code, 'Sessions' has this:
Replacing these with a stl map would be easier than trying to replace ALL uses of PDict. I had started trying to replace those with a stl map, but that gets involved when you run into the itercookie stuff.
A Lookup against one of those maps is done for each and every packet received, so improving the perf could be worth it.
There's also "if it's not broken, don't fix it". I'll leave that one alone and focus on the templates/iterator support for now.
Thinking more about this one, since the containers are already just holding pointers, we probably get the benefits of move semantics since it's just moving pointer locations and not entire objects.
I havne't fully digested this yet, but couple of quick comments:
(1) Dict has special logic for resizing incrementally to avoid processing spikes; that'd be something to watch out for (and agree with the "not broken" comment).
(2) I've found making something "iterable" quite painful in the past in terms of meeting all the properties, but I believe I also remember some 3rd party header library that provides most of the legwork through a wrapper; might be something to consider
I'm mostly running into the problem implementing
I shelved that for a few minutes while I look at whether there's any performance gain/loss for removing
My goal here isn't purely performance based but more towards modernizing a bit. That said, here's 10 runs of
A second run of 10 of each results in them running the same time of 1m3.223s, which means the above difference is probably noise as well.