Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Message generation performance #344

Closed
rssen opened this issue Jul 16, 2020 · 25 comments
Closed

Message generation performance #344

rssen opened this issue Jul 16, 2020 · 25 comments
Assignees
Labels
pyrflx Related to pyrflx package (Legacy Python API)

Comments

@rssen
Copy link
Contributor

rssen commented Jul 16, 2020

PyRFLX has performance problems when generating messages. See attached profiling of the generation of 100 icmp messages.

rf_1.zip

@rssen rssen added the pyrflx Related to pyrflx package (Legacy Python API) label Jul 16, 2020
@treiher treiher added this to To do in RecordFlux 0.5 via automation Jul 16, 2020
@treiher
Copy link
Collaborator

treiher commented Jul 20, 2020

A good start might be the caching of function results: https://docs.python.org/3/library/functools.html#functools.lru_cache.

@treiher
Copy link
Collaborator

treiher commented Jul 20, 2020

Unfortunatly, mypy does not support decorated properties and shows a warning for each instance. A better solution would be cached_property, but this requires at least Python 3.8.

@jklmnn jklmnn moved this from To do to In progress in RecordFlux 0.5 Jul 23, 2020
@jklmnn jklmnn self-assigned this Jul 23, 2020
jklmnn added a commit that referenced this issue Jul 27, 2020
@jklmnn
Copy link
Member

jklmnn commented Jul 28, 2020

We can't simply annotate all our functions and properties with @lru_cache because it requires the inputs to be hashable which is not true for the classes we would use it in. So we either have to do some state keeping or change the architecture to move the load into functions that can be cached (this would probably involve making expressions hashable).

@jklmnn
Copy link
Member

jklmnn commented Jul 28, 2020

I think a good first approach would be to replace all unnecessary inline calls by variables. This does't require complex state management as it is done at method scope but it might reduce the load significantly. I tested this in a single example and even with -O multiple identical function calls are not optimized (as this would require some knowledge about volatility).

#!/usr/bin/env -S python3 -O

def count() -> int:
    x = 0 
    for i in range(1, 1000):
        for j in range(i, i * 10):
            x += i + j 
    return x

def compare_twice() -> bool:
    return count() > 10 and count() < 100 

def compare_once() -> bool:
    cnt = count()
    return cnt > 10 and cnt < 100 


if __name__ == "__main__":
    print(compare_once())
    print(compare_twice())
False
False
         11 function calls in 0.715 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.715    0.715 :0(exec)
        2    0.000    0.000    0.000    0.000 :0(print)
        1    0.001    0.001    0.001    0.001 :0(setprofile)
        1    0.000    0.000    0.480    0.480 perf.py:10(compare_twice)
        1    0.000    0.000    0.235    0.235 perf.py:13(compare_once)
        1    0.000    0.000    0.715    0.715 perf.py:3(<module>)
        3    0.715    0.238    0.715    0.238 perf.py:3(count)
        1    0.000    0.000    0.715    0.715 profile:0(<code object <module> at 0x7ff3ffdcf390, file "perf.py", line 3>)
        0    0.000             0.000          profile:0(profiler)

As one can see, compare_twice requires even more than twice the time of compare_once event though it does the same.

@senier
Copy link
Member

senier commented Jul 28, 2020

Interesting! @jklmnn @treiher Are you aware of any tools that can detect such situations (say, on a per-subprogram basis)?

@treiher
Copy link
Collaborator

treiher commented Jul 28, 2020

I'm not aware of any tool which tries to do that. I suppose it is not that easy to detect valid cases reliably, as you would have to check if a called function has any side effect.

@treiher
Copy link
Collaborator

treiher commented Jul 28, 2020

this would probably involve making expressions hashable

I thought this should be already the case.

@lru_cache [...] requires the inputs to be hashable

I suppose that shouldn't be that hard. Or am I wrong?

@senier
Copy link
Member

senier commented Jul 28, 2020

Have we ever through about PyPy? Do you have opinions on that? It only supports Python 3.6.9, though.

@treiher
Copy link
Collaborator

treiher commented Jul 28, 2020

I have no experience with PyPy. The performance improvement looks nice. I don't know how hard it would be for us to keep compatibility to PyPy. It seems the issues caused by PyPy could be quite annoying. So I'm not sure if it would be worth the effort, but it might be worth a try.

@senier
Copy link
Member

senier commented Jul 28, 2020

It seems the issues caused by PyPy could be quite annoying. So I'm not sure if it would be worth the effort, but it might be worth a try.

This reads like PyPy shouldn't be our first choice, but we may consider it later should we run out of ideas.

@jklmnn
Copy link
Member

jklmnn commented Jul 28, 2020

I think even if PyPy would give us the advertised 4-5 times better performance this isn't enough. Also I've taken a deeper look into our code. The main problem isn't that we're doing overly heavy work at some point but that we've implemented many properties to be calculated dynamically and that we've taken the existence of these properties for granted. This leads to two problems: we can't really transfer those properties to state because we never paid attention to where we change them because they're recalculated anyway. And we're using them everywhere because we already have them. Especially the last point is a problem because they're connected to each other so calling one property might actually call 3 or 4 more. Also we used them for checks. E.g. we check if a property has a certain value, and then use it. Since checking and using are done in two different functions the value is actually calculated twice.
While I'm sure this can be improved with caching I'd still say we should also fix this at least to some extend architectural.

@treiher
Copy link
Collaborator

treiher commented Jul 28, 2020

I'm sure this can be improved with caching

I also think so.

I'd still say we should also fix this at least to some extend architectural.

In what way would the architecture need to be changed to alleviate this problem? Do you have concrete actions in mind?

@jklmnn
Copy link
Member

jklmnn commented Jul 28, 2020

In what way would the architecture need to be changed to alleviate this problem? Do you have concrete actions in mind?

The best example for this problem are the _has_(first|length) and _get_(first|length) functions. Both call _get_(first|length)_unchecked but the former only checks the return value to be of a certain type. The usual use case is

if self._has_length(field):
   length = self._get_length(field)

While this looks unproblematic at first it calculates the unchecked value twice. The calculation itself is not cheap either, if it is a non-scalar value it might iterate through the whole packet first.
My solution is to reduce it to a single get_(first|length) function that returns an optional type.

@treiher
Copy link
Collaborator

treiher commented Jul 28, 2020

That makes sense. Thanks for the explanation.

@jklmnn
Copy link
Member

jklmnn commented Jul 28, 2020

2521b25 implements #344 (comment) and prevents the recalculation of already known accessible fields. Comparison:

Baseline:

         39851048 function calls (39283083 primitive calls) in 53.497 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        0    0.000             0.000          profile:0(profiler)
        1    0.000    0.000   53.497   53.497 profile:0(run())
        1    0.000    0.000   53.497   53.497 :0(exec)
        1    0.000    0.000   53.497   53.497 <string>:1(<module>)
        1    0.000    0.000   53.497   53.497 ./tests/benchmark.py:78(run)
       21    0.001    0.000   53.497    2.547 ./tests/benchmark.py:24(generate)
        2    0.000    0.000    0.096    0.048 /.../RecordFlux/rflx/pyrflx/package.py:20(__getitem__)
112839/112765    1.032    0.000    3.199    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:66(copy)
   225679    0.197    0.000    0.197    0.000 :0(get)
   112839    0.097    0.000    0.097    0.000 :0(issubclass)
   225676    0.235    0.000    0.235    0.000 :0(getattr)
        2    0.000    0.000    0.096    0.048 /.../RecordFlux/rflx/pyrflx/typevalue.py:521(__copy__)
        2    0.002    0.001    0.096    0.048 /.../RecordFlux/rflx/pyrflx/typevalue.py:495(__init__)
       37    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:47(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:587(fields)
        2    0.000    0.000    0.008    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:498(<dictcomp>)
       66    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:601(types)
     6493    0.007    0.000    0.029    0.000 /.../RecordFlux/rflx/model.py:476(__hash__)
266943/260450    0.149    0.000    0.158    0.000 :0(hash)
     7073    0.010    0.000    0.014    0.000 /.../RecordFlux/rflx/identifier.py:37(__hash__)
      679    0.001    0.000    0.004    0.000 /.../RecordFlux/rflx/model.py:90(package)
      679    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/identifier.py:84(parent)
    76799    0.154    0.000    0.259    0.000 /.../RecordFlux/rflx/identifier.py:11(__init__)
  9688737    5.960    0.000    7.602    0.000 :0(isinstance)
  3698393    4.578    0.000    6.467    0.000 /.../RecordFlux/rflx/identifier.py:27(__eq__)
       33    0.000    0.000    0.007    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:112(construct)
   922361    0.968    0.000    1.642    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:137(__instancecheck__)
   922361    0.674    0.000    0.674    0.000 :0(_abc_instancecheck)
        8    0.000    0.000    0.006    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:210(__init__)
       30    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:133(__init__)
  7034814    3.622    0.000    3.622    0.000 :0(items)
    30487    0.078    0.000    0.243    0.000 /.../RecordFlux/rflx/expression.py:833(__init__)
    51561    0.077    0.000    0.108    0.000 /.../RecordFlux/rflx/expression.py:794(__init__)
   440750    0.276    0.000    0.276    0.000 /.../RecordFlux/rflx/expression.py:71(__init__)
   253377    0.290    0.000    0.426    0.000 /.../RecordFlux/rflx/expression.py:81(__hash__)
      407    0.002    0.000    0.008    0.000 /.../RecordFlux/rflx/identifier.py:56(__mul__)
    75742    0.089    0.000    0.130    0.000 /.../RecordFlux/rflx/identifier.py:43(__str__)
    75802    0.041    0.000    0.041    0.000 :0(join)
      407    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/identifier.py:70(__location)
    74682    0.059    0.000    0.059    0.000 :0(split)
3524115/3029551   16.428    0.000   32.988    0.000 /.../RecordFlux/rflx/expression.py:74(__eq__)
  7032758    5.914    0.000    5.914    0.000 /.../RecordFlux/rflx/expression.py:76(<dictcomp>)
       35    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:902(__init__)
    73025    0.108    0.000    0.233    0.000 /.../RecordFlux/rflx/model.py:482(name)
       22    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:153(__init__)
        4    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:332(__init__)
        5    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:293(__init__)
    10799    0.080    0.000    6.690    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:538(_next_field)
    11259    0.021    0.000    1.067    0.000 /.../RecordFlux/rflx/model.py:609(outgoing)
    11259    0.164    0.000    1.046    0.000 /.../RecordFlux/rflx/model.py:610(<listcomp>)
   739066    1.377    0.000    3.032    0.000 /.../RecordFlux/rflx/model.py:48(__eq__)
        2    0.000    0.000    0.000    0.000 :0(values)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:509(<listcomp>)
      208    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:287(literals)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:464(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:58(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/error.py:125(__init__)
19150/19130    0.013    0.000    0.013    0.000 :0(len)
   145083    0.238    0.000    0.329    0.000 /.../RecordFlux/rflx/expression.py:494(__init__)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:337(assign)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:340(parse)
       62    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:300(_check_length_of_assigned_value)
      462    0.023    0.000    7.745    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:752(_preset_fields)
     9374    0.022    0.000    9.189    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:598(_has_first)
    12672    0.140    0.000   12.214    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:589(_get_first_unchecked)
    44824    0.068    0.000    0.220    0.000 /.../RecordFlux/rflx/model.py:473(__init__)
    27664    0.049    0.000    2.563    0.000 /.../RecordFlux/rflx/model.py:606(incoming)
    27664    0.391    0.000    2.514    0.000 /.../RecordFlux/rflx/model.py:607(<listcomp>)
    64241    0.255    0.000   31.684    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:892(__simplified)
      462    0.414    0.001   14.986    0.032 /.../RecordFlux/rflx/pyrflx/typevalue.py:879(__update_simplified_mapping)
      462    0.060    0.000    1.039    0.002 /.../RecordFlux/rflx/pyrflx/typevalue.py:881(<dictcomp>)
    38170    0.151    0.000    1.970    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:918(set)
    44335    0.028    0.000    0.028    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:73(initialized)
      462    0.060    0.000    1.585    0.003 /.../RecordFlux/rflx/pyrflx/typevalue.py:886(<dictcomp>)
     5652    0.013    0.000    0.030    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:363(size)
    37267    0.203    0.000    1.988    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:927(last)
   123753    0.233    0.000    0.311    0.000 /.../RecordFlux/rflx/expression.py:268(__init__)
   120353    0.215    0.000    0.291    0.000 /.../RecordFlux/rflx/expression.py:220(__init__)
    39126    0.177    0.000    1.487    0.000 /.../RecordFlux/rflx/expression.py:694(simplified)
    49935    0.165    0.000    1.371    0.000 /.../RecordFlux/rflx/expression.py:627(simplified)
74049/62214    0.681    0.000    2.193    0.000 /.../RecordFlux/rflx/expression.py:339(simplified)
    49937    0.027    0.000    0.027    0.000 /.../RecordFlux/rflx/expression.py:645(neutral_element)
   311984    0.155    0.000    0.155    0.000 /.../RecordFlux/rflx/expression.py:598(simplified)
    99868    0.057    0.000    0.057    0.000 /.../RecordFlux/rflx/expression.py:624(operation)
    38894    0.082    0.000    0.188    0.000 /.../RecordFlux/rflx/expression.py:532(__sub__)
    21074    0.089    0.000    0.322    0.000 /.../RecordFlux/rflx/expression.py:857(__init__)
      462    0.064    0.000    1.811    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:887(<dictcomp>)
      462    0.061    0.000    1.950    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:888(<dictcomp>)
   191931    0.396    0.000    1.354    0.000 /.../RecordFlux/rflx/expression.py:126(substituted)
   458546    0.301    0.000    0.301    0.000 /.../RecordFlux/rflx/expression.py:1530(substitution)
   458546    2.014    0.000   24.343    0.000 /.../RecordFlux/rflx/expression.py:1535(<lambda>)
    38035    0.020    0.000    0.020    0.000 /.../RecordFlux/rflx/expression.py:159(simplified)
    12672    0.107    0.000    5.426    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:552(_prev_field)
    20639    0.015    0.000    0.015    0.000 :0(append)
73064/49494    0.394    0.000   26.768    0.001 /.../RecordFlux/rflx/expression.py:330(substituted)
73064/49494    0.222    0.000   25.972    0.001 /.../RecordFlux/rflx/expression.py:336(<listcomp>)
     8985    0.018    0.000    1.026    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:581(_has_length)
     9773    0.049    0.000    1.827    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:567(_get_length_unchecked)
    83887    0.170    0.000    0.261    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:141(size)
    84687    0.050    0.000    0.050    0.000 /.../RecordFlux/rflx/model.py:100(size)
     3298    0.005    0.000    3.058    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:601(_get_first)
80716/80476    0.513    0.000   28.523    0.000 /.../RecordFlux/rflx/expression.py:249(substituted)
   111739    0.389    0.000   26.240    0.000 /.../RecordFlux/rflx/expression.py:817(substituted)
   112837    0.147    0.000    0.147    0.000 :0(__reduce_ex__)
   112837    0.675    0.000    1.307    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:268(_reconstruct)
   112837    0.146    0.000    0.215    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:87(__newobj__)
   112837    0.069    0.000    0.069    0.000 :0(__new__)
   112840    0.136    0.000    0.136    0.000 :0(hasattr)
   112837    0.150    0.000    0.150    0.000 :0(update)
    33391    0.050    0.000    0.378    0.000 /.../RecordFlux/rflx/expression.py:1192(simplified)
    39573    0.176    0.000    0.527    0.000 /.../RecordFlux/rflx/expression.py:1132(_simplified)
      328    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:825(simplified)
    63679    0.079    0.000    0.119    0.000 /.../RecordFlux/rflx/expression.py:559(__eq__)
11484/4033    0.032    0.000    0.830    0.000 /.../RecordFlux/rflx/expression.py:463(simplified)
    11498    0.006    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:472(neutral_element)
      462    0.065    0.000   27.290    0.059 /.../RecordFlux/rflx/pyrflx/typevalue.py:814(_update_accessible_fields)
     6427    0.017    0.000    0.073    0.000 /.../RecordFlux/rflx/model.py:636(field_condition)
        1    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:395(__init__)
     6166    0.005    0.000    0.005    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:469(size)
     5882    0.013    0.000    0.205    0.000 /.../RecordFlux/rflx/expression.py:1212(simplified)
     8188    0.005    0.000    0.005    0.000 /.../RecordFlux/rflx/expression.py:670(neutral_element)
    16364    0.010    0.000    0.010    0.000 /.../RecordFlux/rflx/expression.py:667(operation)
      460    0.013    0.000   52.348    0.114 /.../RecordFlux/rflx/pyrflx/typevalue.py:695(set)
      460    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:698(set_refinement)
      100    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:283(accepted_type)
      100    0.008    0.000    0.784    0.008 /.../RecordFlux/rflx/pyrflx/typevalue.py:222(assign)
      100    0.000    0.000    0.000    0.000 :0(startswith)
      100    0.002    0.000    0.040    0.000 /.../RecordFlux/rflx/model.py:390(constraints)
      100    0.001    0.000    0.011    0.000 /.../RecordFlux/rflx/model.py:394(<dictcomp>)
      100    0.002    0.000    0.015    0.000 /.../RecordFlux/rflx/model.py:403(<listcomp>)
      100    0.001    0.000    0.009    0.000 /.../RecordFlux/rflx/model.py:407(<listcomp>)
      100    0.000    0.000    0.000    0.000 :0(extend)
      300    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/identifier.py:80(name)
     1096    0.006    0.000    0.465    0.000 /.../RecordFlux/rflx/expression.py:870(substituted)
        3    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:96(_slotnames)
    98/24    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:141(__subclasscheck__)
    98/24    0.000    0.000    0.000    0.000 :0(_abc_subclasscheck)
     4446    0.014    0.000    0.568    0.000 /.../RecordFlux/rflx/expression.py:425(simplified)
     4446    0.003    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:434(neutral_element)
    23315    0.028    0.000    0.067    0.000 :0(eq)
    33391    0.020    0.000    0.020    0.000 /.../RecordFlux/rflx/expression.py:469(operation)
     9951    0.006    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:431(operation)
     1448    0.003    0.000    0.017    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:273(expr)
     6165    0.007    0.000    0.010    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:77(_raise_initialized)
      460    0.002    0.000    0.499    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:733(<listcomp>)
     1561    0.013    0.000    0.025    0.000 :0(all)
      300    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:200(accepted_type)
      300    0.005    0.000    0.154    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:168(assign)
       60    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/model.py:262(constraints)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:254(first)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:258(last)
       60    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:1173(simplified)
     4137    0.009    0.000    0.025    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:185(expr)
      240    0.003    0.000    0.014    0.000 /.../RecordFlux/rflx/model.py:170(constraints)
      240    0.001    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:1154(simplified)
      240    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:746(simplified)
      240    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:549(__pow__)
      240    0.000    0.000    0.001    0.000 :0(lt)
      240    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:566(__lt__)
      443    0.005    0.000    0.750    0.002 /.../RecordFlux/rflx/pyrflx/typevalue.py:834(_is_valid_opaque_field)
      232    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:881(simplified)
      232    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:524(__neg__)
      766    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:120(variables)
      441    0.002    0.000    0.049    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:847(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:386(accepted_type)
       20    0.000    0.000    0.000    0.000 :0(le)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:573(__le__)
       60    0.001    0.000    1.051    0.018 /.../RecordFlux/rflx/pyrflx/typevalue.py:800(bytestring)
       60    0.005    0.000    1.048    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:779(bitstring)
      120    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:278(bitstring)
      560    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:499(__str__)
     1120    0.002    0.000    0.002    0.000 :0(format)
      660    0.001    0.000    0.026    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:5(__init__)
      660    0.001    0.000    0.025    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:45(valid_bitstring)
    22420    0.012    0.000    0.012    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:47(<genexpr>)
      640    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:22(__str__)
      380    0.002    0.000    0.013    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:195(bitstring)
       60    0.000    0.000    0.007    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:380(bitstring)
       60    0.000    0.000    0.000    0.000 :0(from_bytes)
       60    0.002    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:807(<listcomp>)
     1360    0.001    0.000    0.001    0.000 :0(to_bytes)
     5651    0.008    0.000    0.018    0.000 :0(ge)
     5651    0.007    0.000    0.010    0.000 /.../RecordFlux/rflx/expression.py:587(__ge__)
        4    0.000    0.000    0.000    0.000 :0(insert)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:803(__neg__)
      345    0.001    0.000    0.215    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:584(_get_length)
      345    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:297(set_expected_size)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:486(accepted_type)
       20    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:401(assign)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:308(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:49(join)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:38(__len__)
      325    0.002    0.000    0.024    0.000 /.../RecordFlux/rflx/expression.py:239(variables)
      650    0.003    0.000    0.011    0.000 /.../RecordFlux/rflx/expression.py:321(variables)
      650    0.002    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:322(<listcomp>)
      650    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:847(variables)
     2275    0.004    0.000    0.017    0.000 /.../RecordFlux/rflx/common.py:28(unique)
     1300    0.002    0.000    0.007    0.000 :0(add)
     1300    0.002    0.000    0.004    0.000 /.../RecordFlux/rflx/expression.py:839(name)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:480(bitstring)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:483(<listcomp>)
       19    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:81(clear)
        1    0.000    0.000    0.000    0.000 :0(setprofile)

Optimized _get_(first|length):

         36286660 function calls (35747086 primitive calls) in 49.476 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        0    0.000             0.000          profile:0(profiler)
        1    0.000    0.000   49.476   49.476 profile:0(run())
        1    0.000    0.000   49.476   49.476 :0(exec)
        1    0.000    0.000   49.476   49.476 <string>:1(<module>)
        1    0.000    0.000   49.476   49.476 ./tests/benchmark.py:78(run)
       21    0.001    0.000   49.476    2.356 ./tests/benchmark.py:24(generate)
        2    0.000    0.000    0.093    0.047 /.../RecordFlux/rflx/pyrflx/package.py:20(__getitem__)
108617/108543    0.966    0.000    3.019    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:66(copy)
   217235    0.178    0.000    0.178    0.000 :0(get)
   108617    0.091    0.000    0.091    0.000 :0(issubclass)
   217232    0.227    0.000    0.227    0.000 :0(getattr)
        2    0.000    0.000    0.093    0.047 /.../RecordFlux/rflx/pyrflx/typevalue.py:521(__copy__)
        2    0.002    0.001    0.093    0.047 /.../RecordFlux/rflx/pyrflx/typevalue.py:495(__init__)
       37    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:47(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:587(fields)
        2    0.000    0.000    0.007    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:498(<dictcomp>)
       66    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:601(types)
     6493    0.007    0.000    0.029    0.000 /.../RecordFlux/rflx/model.py:476(__hash__)
258499/252006    0.144    0.000    0.153    0.000 :0(hash)
     7073    0.010    0.000    0.014    0.000 /.../RecordFlux/rflx/identifier.py:37(__hash__)
      679    0.001    0.000    0.004    0.000 /.../RecordFlux/rflx/model.py:90(package)
      679    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/identifier.py:84(parent)
    70784    0.143    0.000    0.237    0.000 /.../RecordFlux/rflx/identifier.py:11(__init__)
  8754225    5.448    0.000    7.023    0.000 :0(isinstance)
  3299261    4.158    0.000    5.860    0.000 /.../RecordFlux/rflx/identifier.py:27(__eq__)
       33    0.000    0.000    0.006    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:112(construct)
   850293    0.923    0.000    1.575    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:137(__instancecheck__)
   850293    0.652    0.000    0.652    0.000 :0(_abc_instancecheck)
        8    0.000    0.000    0.006    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:210(__init__)
       30    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:133(__init__)
  6393514    3.273    0.000    3.273    0.000 :0(items)
    30487    0.079    0.000    0.244    0.000 /.../RecordFlux/rflx/expression.py:833(__init__)
    51561    0.078    0.000    0.110    0.000 /.../RecordFlux/rflx/expression.py:794(__init__)
   422259    0.266    0.000    0.266    0.000 /.../RecordFlux/rflx/expression.py:71(__init__)
   244933    0.286    0.000    0.417    0.000 /.../RecordFlux/rflx/expression.py:81(__hash__)
      407    0.002    0.000    0.007    0.000 /.../RecordFlux/rflx/identifier.py:56(__mul__)
    70411    0.083    0.000    0.121    0.000 /.../RecordFlux/rflx/identifier.py:43(__str__)
    70471    0.038    0.000    0.038    0.000 :0(join)
      407    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/identifier.py:70(__location)
    68667    0.052    0.000    0.052    0.000 :0(split)
3203437/2733000   15.213    0.000   30.448    0.000 /.../RecordFlux/rflx/expression.py:74(__eq__)
  6391458    5.519    0.000    5.519    0.000 /.../RecordFlux/rflx/expression.py:76(<dictcomp>)
       35    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:909(__init__)
    67694    0.099    0.000    0.216    0.000 /.../RecordFlux/rflx/model.py:482(name)
       22    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:153(__init__)
        4    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:332(__init__)
        5    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:293(__init__)
    10799    0.082    0.000    6.497    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:538(_next_field)
    11259    0.022    0.000    1.086    0.000 /.../RecordFlux/rflx/model.py:609(outgoing)
    11259    0.171    0.000    1.065    0.000 /.../RecordFlux/rflx/model.py:610(<listcomp>)
   628052    1.190    0.000    2.611    0.000 /.../RecordFlux/rflx/model.py:48(__eq__)
        2    0.000    0.000    0.000    0.000 :0(values)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:509(<listcomp>)
      208    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:287(literals)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:464(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:58(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/error.py:125(__init__)
16310/16290    0.011    0.000    0.011    0.000 :0(len)
   141324    0.237    0.000    0.326    0.000 /.../RecordFlux/rflx/expression.py:494(__init__)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:337(assign)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:340(parse)
       62    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:300(_check_length_of_assigned_value)
      462    0.022    0.000    5.152    0.011 /.../RecordFlux/rflx/pyrflx/typevalue.py:743(_preset_fields)
     9834    0.121    0.000    9.669    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:582(_get_first)
    38809    0.059    0.000    0.190    0.000 /.../RecordFlux/rflx/model.py:473(__init__)
    21649    0.038    0.000    2.035    0.000 /.../RecordFlux/rflx/model.py:606(incoming)
    21649    0.307    0.000    1.997    0.000 /.../RecordFlux/rflx/model.py:607(<listcomp>)
    55610    0.219    0.000   28.888    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:899(__simplified)
      462    0.397    0.001   14.479    0.031 /.../RecordFlux/rflx/pyrflx/typevalue.py:886(__update_simplified_mapping)
      462    0.057    0.000    1.001    0.002 /.../RecordFlux/rflx/pyrflx/typevalue.py:888(<dictcomp>)
    38632    0.156    0.000    2.010    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:925(set)
    44797    0.029    0.000    0.029    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:73(initialized)
      462    0.059    0.000    1.571    0.003 /.../RecordFlux/rflx/pyrflx/typevalue.py:893(<dictcomp>)
     5650    0.014    0.000    0.030    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:363(size)
    37363    0.209    0.000    2.026    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:934(last)
   112111    0.213    0.000    0.284    0.000 /.../RecordFlux/rflx/expression.py:268(__init__)
   117263    0.211    0.000    0.285    0.000 /.../RecordFlux/rflx/expression.py:220(__init__)
    38877    0.176    0.000    1.497    0.000 /.../RecordFlux/rflx/expression.py:694(simplified)
    47193    0.154    0.000    1.311    0.000 /.../RecordFlux/rflx/expression.py:627(simplified)
69695/58572    0.653    0.000    2.127    0.000 /.../RecordFlux/rflx/expression.py:339(simplified)
    47195    0.025    0.000    0.025    0.000 /.../RecordFlux/rflx/expression.py:645(neutral_element)
   300611    0.155    0.000    0.155    0.000 /.../RecordFlux/rflx/expression.py:598(simplified)
    94384    0.057    0.000    0.057    0.000 /.../RecordFlux/rflx/expression.py:624(operation)
    38645    0.085    0.000    0.194    0.000 /.../RecordFlux/rflx/expression.py:532(__sub__)
    21074    0.087    0.000    0.323    0.000 /.../RecordFlux/rflx/expression.py:857(__init__)
      462    0.056    0.000    1.611    0.003 /.../RecordFlux/rflx/pyrflx/typevalue.py:894(<dictcomp>)
      462    0.061    0.000    1.985    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:895(<dictcomp>)
   166805    0.349    0.000    1.200    0.000 /.../RecordFlux/rflx/expression.py:126(substituted)
   417112    0.284    0.000    0.284    0.000 /.../RecordFlux/rflx/expression.py:1530(substitution)
   417112    1.835    0.000   22.094    0.000 /.../RecordFlux/rflx/expression.py:1535(<lambda>)
    33085    0.018    0.000    0.018    0.000 /.../RecordFlux/rflx/expression.py:159(simplified)
     9834    0.084    0.000    4.289    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:552(_prev_field)
    17684    0.014    0.000    0.014    0.000 :0(append)
64164/42018    0.347    0.000   24.610    0.001 /.../RecordFlux/rflx/expression.py:330(substituted)
64164/42018    0.200    0.000   23.927    0.001 /.../RecordFlux/rflx/expression.py:336(<listcomp>)
     9834    0.046    0.000    1.587    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:567(_get_length)
    81950    0.167    0.000    0.259    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:141(size)
    82750    0.051    0.000    0.051    0.000 /.../RecordFlux/rflx/model.py:100(size)
77530/77290    0.497    0.000   26.125    0.000 /.../RecordFlux/rflx/expression.py:249(substituted)
   107517    0.367    0.000   23.913    0.000 /.../RecordFlux/rflx/expression.py:817(substituted)
   108615    0.136    0.000    0.136    0.000 :0(__reduce_ex__)
   108615    0.645    0.000    1.247    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:268(_reconstruct)
   108615    0.142    0.000    0.208    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:87(__newobj__)
   108615    0.066    0.000    0.066    0.000 :0(__new__)
   108618    0.127    0.000    0.127    0.000 :0(hasattr)
   108615    0.143    0.000    0.143    0.000 :0(update)
    32335    0.048    0.000    0.370    0.000 /.../RecordFlux/rflx/expression.py:1192(simplified)
    38325    0.170    0.000    0.516    0.000 /.../RecordFlux/rflx/expression.py:1132(_simplified)
      328    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:825(simplified)
    61716    0.079    0.000    0.119    0.000 /.../RecordFlux/rflx/expression.py:559(__eq__)
10734/3995    0.031    0.000    0.815    0.000 /.../RecordFlux/rflx/expression.py:463(simplified)
    10748    0.006    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:472(neutral_element)
      462    0.071    0.000   26.492    0.057 /.../RecordFlux/rflx/pyrflx/typevalue.py:807(_update_accessible_fields)
     6427    0.017    0.000    0.074    0.000 /.../RecordFlux/rflx/model.py:636(field_condition)
        1    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:395(__init__)
     5863    0.005    0.000    0.005    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:469(size)
     5690    0.011    0.000    0.199    0.000 /.../RecordFlux/rflx/expression.py:1212(simplified)
     7326    0.004    0.000    0.004    0.000 /.../RecordFlux/rflx/expression.py:670(neutral_element)
    14640    0.010    0.000    0.010    0.000 /.../RecordFlux/rflx/expression.py:667(operation)
      460    0.014    0.000   48.365    0.105 /.../RecordFlux/rflx/pyrflx/typevalue.py:683(set)
   104/26    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:141(__subclasscheck__)
   104/26    0.000    0.000    0.000    0.000 :0(_abc_subclasscheck)
      460    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:686(set_refinement)
      100    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:283(accepted_type)
      100    0.008    0.000    0.770    0.008 /.../RecordFlux/rflx/pyrflx/typevalue.py:222(assign)
      100    0.000    0.000    0.000    0.000 :0(startswith)
      100    0.002    0.000    0.039    0.000 /.../RecordFlux/rflx/model.py:390(constraints)
      100    0.001    0.000    0.011    0.000 /.../RecordFlux/rflx/model.py:394(<dictcomp>)
      100    0.002    0.000    0.015    0.000 /.../RecordFlux/rflx/model.py:403(<listcomp>)
      100    0.001    0.000    0.009    0.000 /.../RecordFlux/rflx/model.py:407(<listcomp>)
      100    0.000    0.000    0.000    0.000 :0(extend)
      300    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/identifier.py:80(name)
     1096    0.006    0.000    0.457    0.000 /.../RecordFlux/rflx/expression.py:870(substituted)
        3    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:96(_slotnames)
     4446    0.014    0.000    0.577    0.000 /.../RecordFlux/rflx/expression.py:425(simplified)
     4446    0.003    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:434(neutral_element)
    22565    0.028    0.000    0.066    0.000 :0(eq)
    31891    0.020    0.000    0.020    0.000 /.../RecordFlux/rflx/expression.py:469(operation)
     9951    0.006    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:431(operation)
     1448    0.003    0.000    0.018    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:273(expr)
     6165    0.008    0.000    0.011    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:77(_raise_initialized)
      460    0.002    0.000    0.475    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:724(<listcomp>)
     1561    0.013    0.000    0.026    0.000 :0(all)
      300    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:200(accepted_type)
      300    0.005    0.000    0.154    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:168(assign)
       60    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/model.py:262(constraints)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:254(first)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:258(last)
       60    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:1173(simplified)
     4137    0.009    0.000    0.026    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:185(expr)
      240    0.003    0.000    0.013    0.000 /.../RecordFlux/rflx/model.py:170(constraints)
      240    0.001    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:1154(simplified)
      240    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:746(simplified)
      240    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:549(__pow__)
      240    0.000    0.000    0.001    0.000 :0(lt)
      240    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:566(__lt__)
      443    0.010    0.000    0.340    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:828(_is_valid_opaque_field)
      232    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:881(simplified)
      232    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:524(__neg__)
      766    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:120(variables)
      441    0.002    0.000    0.049    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:854(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:386(accepted_type)
       20    0.000    0.000    0.000    0.000 :0(le)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:573(__le__)
       60    0.001    0.000    1.017    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:793(bytestring)
       60    0.004    0.000    1.013    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:772(bitstring)
      120    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:278(bitstring)
      560    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:499(__str__)
     1120    0.002    0.000    0.002    0.000 :0(format)
      660    0.001    0.000    0.027    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:5(__init__)
      660    0.001    0.000    0.026    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:45(valid_bitstring)
    22420    0.012    0.000    0.012    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:47(<genexpr>)
      640    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:22(__str__)
      380    0.002    0.000    0.014    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:195(bitstring)
       60    0.000    0.000    0.007    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:380(bitstring)
       60    0.000    0.000    0.000    0.000 :0(from_bytes)
       60    0.002    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:800(<listcomp>)
     1360    0.001    0.000    0.001    0.000 :0(to_bytes)
     5459    0.008    0.000    0.017    0.000 :0(ge)
     5459    0.007    0.000    0.010    0.000 /.../RecordFlux/rflx/expression.py:587(__ge__)
        4    0.000    0.000    0.000    0.000 :0(insert)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:803(__neg__)
      345    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:297(set_expected_size)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:486(accepted_type)
       20    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:401(assign)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:308(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:49(join)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:38(__len__)
      325    0.002    0.000    0.025    0.000 /.../RecordFlux/rflx/expression.py:239(variables)
      650    0.003    0.000    0.011    0.000 /.../RecordFlux/rflx/expression.py:321(variables)
      650    0.002    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:322(<listcomp>)
      650    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:847(variables)
     2275    0.004    0.000    0.017    0.000 /.../RecordFlux/rflx/common.py:28(unique)
     1300    0.002    0.000    0.007    0.000 :0(add)
     1300    0.002    0.000    0.004    0.000 /.../RecordFlux/rflx/expression.py:839(name)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:480(bitstring)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:483(<listcomp>)
       19    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:81(clear)
        1    0.000    0.000    0.000    0.000 :0(setprofile)

Optimized _update_accessible_fields:

         28884604 function calls (28310581 primitive calls) in 39.457 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        0    0.000             0.000          profile:0(profiler)
        1    0.000    0.000   39.457   39.457 profile:0(run())
        1    0.000    0.000   39.457   39.457 :0(exec)
        1    0.000    0.000   39.457   39.457 <string>:1(<module>)
        1    0.000    0.000   39.457   39.457 ./tests/benchmark.py:78(run)
       21    0.001    0.000   39.457    1.879 ./tests/benchmark.py:24(generate)
        2    0.000    0.000    0.101    0.051 /.../RecordFlux/rflx/pyrflx/package.py:20(__getitem__)
72288/72214    0.645    0.000    2.057    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:66(copy)
   144577    0.121    0.000    0.121    0.000 :0(get)
    72288    0.060    0.000    0.060    0.000 :0(issubclass)
   144574    0.149    0.000    0.149    0.000 :0(getattr)
        2    0.000    0.000    0.101    0.051 /.../RecordFlux/rflx/pyrflx/typevalue.py:521(__copy__)
        2    0.002    0.001    0.101    0.051 /.../RecordFlux/rflx/pyrflx/typevalue.py:495(__init__)
       37    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:47(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:587(fields)
        2    0.000    0.000    0.008    0.004 /.../RecordFlux/rflx/pyrflx/typevalue.py:498(<dictcomp>)
       66    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:601(types)
     3010    0.003    0.000    0.013    0.000 /.../RecordFlux/rflx/model.py:476(__hash__)
178795/175785    0.103    0.000    0.107    0.000 :0(hash)
     3590    0.005    0.000    0.007    0.000 /.../RecordFlux/rflx/identifier.py:37(__hash__)
      679    0.001    0.000    0.004    0.000 /.../RecordFlux/rflx/model.py:90(package)
      679    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/identifier.py:84(parent)
    57201    0.117    0.000    0.196    0.000 /.../RecordFlux/rflx/identifier.py:11(__init__)
  6881022    4.301    0.000    5.400    0.000 :0(isinstance)
  2539350    3.193    0.000    4.533    0.000 /.../RecordFlux/rflx/identifier.py:27(__eq__)
       33    0.000    0.000    0.007    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:112(construct)
   609220    0.650    0.000    1.099    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:137(__instancecheck__)
   609220    0.448    0.000    0.449    0.000 :0(_abc_instancecheck)
        8    0.001    0.000    0.007    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:210(__init__)
       30    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:133(__init__)
  5364290    2.816    0.000    2.816    0.000 :0(items)
    30487    0.081    0.000    0.249    0.000 /.../RecordFlux/rflx/expression.py:833(__init__)
    51501    0.078    0.000    0.109    0.000 /.../RecordFlux/rflx/expression.py:794(__init__)
   363868    0.221    0.000    0.221    0.000 /.../RecordFlux/rflx/expression.py:71(__init__)
   172195    0.202    0.000    0.298    0.000 /.../RecordFlux/rflx/expression.py:81(__hash__)
      407    0.002    0.000    0.007    0.000 /.../RecordFlux/rflx/identifier.py:56(__mul__)
    48451    0.058    0.000    0.084    0.000 /.../RecordFlux/rflx/identifier.py:43(__str__)
    48511    0.027    0.000    0.027    0.000 :0(join)
      407    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/identifier.py:70(__location)
    55084    0.043    0.000    0.043    0.000 :0(split)
2687785/2156129   12.831    0.000   25.441    0.000 /.../RecordFlux/rflx/expression.py:74(__eq__)
  5362234    4.579    0.000    4.579    0.000 /.../RecordFlux/rflx/expression.py:76(<dictcomp>)
       35    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:909(__init__)
    45814    0.067    0.000    0.147    0.000 /.../RecordFlux/rflx/model.py:482(name)
       22    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:153(__init__)
        4    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:332(__init__)
        5    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:293(__init__)
     7319    0.058    0.000    4.388    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:538(_next_field)
     7779    0.015    0.000    0.767    0.000 /.../RecordFlux/rflx/model.py:609(outgoing)
     7779    0.121    0.000    0.752    0.000 /.../RecordFlux/rflx/model.py:610(<listcomp>)
   425389    0.816    0.000    1.799    0.000 /.../RecordFlux/rflx/model.py:48(__eq__)
        2    0.000    0.000    0.000    0.000 :0(values)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:509(<listcomp>)
      208    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:287(literals)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:464(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:58(__init__)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/error.py:125(__init__)
12370/12350    0.009    0.000    0.009    0.000 :0(len)
   134463    0.225    0.000    0.306    0.000 /.../RecordFlux/rflx/expression.py:494(__init__)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:337(assign)
       42    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:340(parse)
       62    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:300(_check_length_of_assigned_value)
      462    0.021    0.000    5.163    0.011 /.../RecordFlux/rflx/pyrflx/typevalue.py:740(_preset_fields)
     6354    0.075    0.000    5.849    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:579(_get_first)
    25226    0.037    0.000    0.125    0.000 /.../RecordFlux/rflx/model.py:473(__init__)
    14569    0.026    0.000    1.391    0.000 /.../RecordFlux/rflx/model.py:606(incoming)
    14569    0.209    0.000    1.364    0.000 /.../RecordFlux/rflx/model.py:607(<listcomp>)
    37307    0.149    0.000   18.970    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:899(__simplified)
      462    0.419    0.001   15.868    0.034 /.../RecordFlux/rflx/pyrflx/typevalue.py:886(__update_simplified_mapping)
      462    0.061    0.000    1.064    0.002 /.../RecordFlux/rflx/pyrflx/typevalue.py:888(<dictcomp>)
    38352    0.152    0.000    1.992    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:925(set)
    44517    0.028    0.000    0.028    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:73(initialized)
      462    0.059    0.000    1.574    0.003 /.../RecordFlux/rflx/pyrflx/typevalue.py:893(<dictcomp>)
     5190    0.012    0.000    0.027    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:363(size)
    37203    0.210    0.000    2.010    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:934(last)
    86581    0.163    0.000    0.217    0.000 /.../RecordFlux/rflx/expression.py:268(__init__)
    91323    0.160    0.000    0.216    0.000 /.../RecordFlux/rflx/expression.py:220(__init__)
    38637    0.176    0.000    1.480    0.000 /.../RecordFlux/rflx/expression.py:694(simplified)
    43553    0.143    0.000    1.201    0.000 /.../RecordFlux/rflx/expression.py:627(simplified)
58600/51589    0.532    0.000    1.664    0.000 /.../RecordFlux/rflx/expression.py:339(simplified)
    43555    0.023    0.000    0.023    0.000 /.../RecordFlux/rflx/expression.py:645(neutral_element)
   258410    0.132    0.000    0.132    0.000 /.../RecordFlux/rflx/expression.py:598(simplified)
    87104    0.052    0.000    0.052    0.000 /.../RecordFlux/rflx/expression.py:624(operation)
    38425    0.084    0.000    0.191    0.000 /.../RecordFlux/rflx/expression.py:532(__sub__)
    21014    0.088    0.000    0.328    0.000 /.../RecordFlux/rflx/expression.py:857(__init__)
      462    0.076    0.000    2.181    0.005 /.../RecordFlux/rflx/pyrflx/typevalue.py:894(<dictcomp>)
      462    0.074    0.000    2.312    0.005 /.../RecordFlux/rflx/pyrflx/typevalue.py:895(<dictcomp>)
   111732    0.238    0.000    0.809    0.000 /.../RecordFlux/rflx/expression.py:126(substituted)
   278060    0.186    0.000    0.186    0.000 /.../RecordFlux/rflx/expression.py:1530(substitution)
   278060    1.241    0.000   14.715    0.000 /.../RecordFlux/rflx/expression.py:1535(<lambda>)
    22505    0.012    0.000    0.012    0.000 /.../RecordFlux/rflx/expression.py:159(simplified)
     6354    0.038    0.000    2.566    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:552(_prev_field)
     6354    0.024    0.000    1.874    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:555(<listcomp>)
42294/28372    0.232    0.000   16.510    0.001 /.../RecordFlux/rflx/expression.py:330(substituted)
42294/28372    0.133    0.000   16.049    0.001 /.../RecordFlux/rflx/expression.py:336(<listcomp>)
     6354    0.034    0.000    1.463    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:564(_get_length)
    75310    0.153    0.000    0.236    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:141(size)
    76110    0.045    0.000    0.045    0.000 /.../RecordFlux/rflx/model.py:100(size)
51750/51510    0.330    0.000   17.402    0.000 /.../RecordFlux/rflx/expression.py:249(substituted)
    71248    0.245    0.000   15.895    0.000 /.../RecordFlux/rflx/expression.py:817(substituted)
    72286    0.093    0.000    0.093    0.000 :0(__reduce_ex__)
    72286    0.432    0.000    0.838    0.000 /.../RecordFlux/venv/lib/python3.7/copy.py:268(_reconstruct)
    72286    0.094    0.000    0.139    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:87(__newobj__)
    72286    0.045    0.000    0.045    0.000 :0(__new__)
    72289    0.086    0.000    0.086    0.000 :0(hasattr)
    72286    0.097    0.000    0.097    0.000 :0(update)
    21806    0.032    0.000    0.249    0.000 /.../RecordFlux/rflx/expression.py:1192(simplified)
    25515    0.114    0.000    0.333    0.000 /.../RecordFlux/rflx/expression.py:1132(_simplified)
      308    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:825(simplified)
    41045    0.051    0.000    0.079    0.000 /.../RecordFlux/rflx/expression.py:559(__eq__)
6925/2913    0.020    0.000    0.533    0.000 /.../RecordFlux/rflx/expression.py:463(simplified)
     6939    0.004    0.000    0.004    0.000 /.../RecordFlux/rflx/expression.py:472(neutral_element)
     3116    0.002    0.000    0.002    0.000 :0(append)
      462    0.035    0.000   15.039    0.033 /.../RecordFlux/rflx/pyrflx/typevalue.py:804(_update_accessible_fields)
     2944    0.008    0.000    0.035    0.000 /.../RecordFlux/rflx/model.py:636(field_condition)
      462    0.001    0.000    0.001    0.000 :0(index)
        1    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:395(__init__)
     5763    0.005    0.000    0.005    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:469(size)
     3409    0.007    0.000    0.117    0.000 /.../RecordFlux/rflx/expression.py:1212(simplified)
     4965    0.003    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:670(neutral_element)
     9918    0.007    0.000    0.007    0.000 /.../RecordFlux/rflx/expression.py:667(operation)
      460    0.014    0.000   38.329    0.083 /.../RecordFlux/rflx/pyrflx/typevalue.py:680(set)
   104/26    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/abc.py:141(__subclasscheck__)
   104/26    0.000    0.000    0.000    0.000 :0(_abc_subclasscheck)
      460    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:683(set_refinement)
      100    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:283(accepted_type)
      100    0.008    0.000    0.794    0.008 /.../RecordFlux/rflx/pyrflx/typevalue.py:222(assign)
      100    0.000    0.000    0.000    0.000 :0(startswith)
      100    0.002    0.000    0.040    0.000 /.../RecordFlux/rflx/model.py:390(constraints)
      100    0.001    0.000    0.011    0.000 /.../RecordFlux/rflx/model.py:394(<dictcomp>)
      100    0.002    0.000    0.015    0.000 /.../RecordFlux/rflx/model.py:403(<listcomp>)
      100    0.001    0.000    0.009    0.000 /.../RecordFlux/rflx/model.py:407(<listcomp>)
      100    0.000    0.000    0.000    0.000 :0(extend)
      300    0.000    0.000    0.002    0.000 /.../RecordFlux/rflx/identifier.py:80(name)
     1036    0.006    0.000    0.427    0.000 /.../RecordFlux/rflx/expression.py:870(substituted)
        3    0.000    0.000    0.000    0.000 /.../RecordFlux/venv/lib/python3.7/copyreg.py:96(_slotnames)
     3161    0.010    0.000    0.418    0.000 /.../RecordFlux/rflx/expression.py:425(simplified)
     3161    0.002    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:434(neutral_element)
    14824    0.018    0.000    0.044    0.000 :0(eq)
    20741    0.013    0.000    0.013    0.000 /.../RecordFlux/rflx/expression.py:469(operation)
     7360    0.005    0.000    0.005    0.000 /.../RecordFlux/rflx/expression.py:431(operation)
     1448    0.003    0.000    0.018    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:273(expr)
     6165    0.007    0.000    0.011    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:77(_raise_initialized)
      460    0.002    0.000    0.477    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:721(<listcomp>)
     1521    0.013    0.000    0.026    0.000 :0(all)
      300    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:200(accepted_type)
      300    0.005    0.000    0.154    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:168(assign)
       60    0.001    0.000    0.004    0.000 /.../RecordFlux/rflx/model.py:262(constraints)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:254(first)
       60    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/model.py:258(last)
       60    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:1173(simplified)
     4137    0.009    0.000    0.026    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:185(expr)
      240    0.003    0.000    0.013    0.000 /.../RecordFlux/rflx/model.py:170(constraints)
      240    0.001    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:1154(simplified)
      240    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/expression.py:746(simplified)
      240    0.001    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:549(__pow__)
      240    0.000    0.000    0.001    0.000 :0(lt)
      240    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:566(__lt__)
      403    0.009    0.000    0.297    0.001 /.../RecordFlux/rflx/pyrflx/typevalue.py:828(_is_valid_opaque_field)
      212    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:881(simplified)
      212    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/expression.py:524(__neg__)
      706    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:120(variables)
      401    0.002    0.000    0.046    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:854(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:386(accepted_type)
       20    0.000    0.000    0.000    0.000 :0(le)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:573(__le__)
       60    0.001    0.000    1.026    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:790(bytestring)
       60    0.004    0.000    1.023    0.017 /.../RecordFlux/rflx/pyrflx/typevalue.py:769(bitstring)
      120    0.001    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:278(bitstring)
      560    0.001    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:499(__str__)
     1120    0.002    0.000    0.002    0.000 :0(format)
      660    0.001    0.000    0.027    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:5(__init__)
      660    0.001    0.000    0.026    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:45(valid_bitstring)
    22420    0.012    0.000    0.012    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:47(<genexpr>)
      640    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:22(__str__)
      380    0.002    0.000    0.014    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:195(bitstring)
       60    0.000    0.000    0.007    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:380(bitstring)
       60    0.000    0.000    0.000    0.000 :0(from_bytes)
       60    0.002    0.000    0.003    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:797(<listcomp>)
     1360    0.001    0.000    0.001    0.000 :0(to_bytes)
     3178    0.005    0.000    0.010    0.000 :0(ge)
     3178    0.004    0.000    0.006    0.000 /.../RecordFlux/rflx/expression.py:587(__ge__)
        4    0.000    0.000    0.000    0.000 :0(insert)
        2    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:803(__neg__)
      345    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:297(set_expected_size)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:486(accepted_type)
       20    0.000    0.000    0.001    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:401(assign)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:308(<listcomp>)
       40    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:49(join)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/bitstring.py:38(__len__)
      305    0.002    0.000    0.023    0.000 /.../RecordFlux/rflx/expression.py:239(variables)
      610    0.003    0.000    0.010    0.000 /.../RecordFlux/rflx/expression.py:321(variables)
      610    0.002    0.000    0.002    0.000 /.../RecordFlux/rflx/expression.py:322(<listcomp>)
      610    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/expression.py:847(variables)
     2135    0.004    0.000    0.016    0.000 /.../RecordFlux/rflx/common.py:28(unique)
     1220    0.002    0.000    0.007    0.000 :0(add)
     1220    0.002    0.000    0.004    0.000 /.../RecordFlux/rflx/expression.py:839(name)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:480(bitstring)
       20    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:483(<listcomp>)
       19    0.000    0.000    0.000    0.000 /.../RecordFlux/rflx/pyrflx/typevalue.py:81(clear)
        1    0.000    0.000    0.000    0.000 :0(setprofile)

@jklmnn
Copy link
Member

jklmnn commented Jul 29, 2020

Would there be any significant drawback in precalculating (presetting length etc, calculating accessible fields) only the next field? This would require setting all fields in order and changing a single field would require setting all following fields again (assuming the change would not change the path).
Implementing this restriction would create a significant performance gain.
@treiher @senier @rssen

@treiher
Copy link
Collaborator

treiher commented Jul 29, 2020

Would this be compatible to the current checksum functionality? In ICMP the checksum can only be calculated when also the fields behind the checksum field are set. So at least internally a field must be settable without invalidating other fields.

The idea of the current design was that it should be possible to do minor modifications to a message without the need the recreate the whole message (e.g. just changing address fields). I don't know how often such a situation will occur, but they would be more complicated to handle.

@jklmnn
Copy link
Member

jklmnn commented Jul 29, 2020

In this case the performance highly depends on the use case. I'm not really sure if we can do anything about it. Setting a single field and validating the rest of the message should be much faster than setting all fields (assuming the message stays valid). We could also introduce a flag that can be set not to prepare the whole message. The default would be the current behavior but by setting this flag in the set method one could select to only precalculate the next field (which would change how many fields are accessible after calling set).

@senier
Copy link
Member

senier commented Jul 29, 2020

he default would be the current behavior but by setting this flag in the set method one could select to only precalculate the next field (which would change how many fields are accessible after calling set).

What would happen if I try to set another (maybe valid, but not precalculated field)?

This optimization sounds sounds sensible for tools like the Fuzzer, which generates valid messages with the fields in the right order. OTOH, what's the use-case of the old interface then? IIRC, we originally designed it for the Fuzzer to explore possible paths in the message. It's not used in that way, though, as the Fuzzer uses it's own model.

@jklmnn
Copy link
Member

jklmnn commented Jul 29, 2020

What would happen if I try to set another (maybe valid, but not precalculated field)?

Only the next accessible field is calculated so setting any other field would cause an error.

OTOH, what's the use-case of the old interface then?

Well one could create a complete message with the new interface and then change a single field in the message with the old interface without loosing the complete message. This would be the most efficient use of both interfaces if one wanted to change only the address in e.g. IPv4.

@senier
Copy link
Member

senier commented Jul 29, 2020

Well one could create a complete message with the new interface and then change a single field in the message with the old interface without loosing the complete message. This would be the most efficient use of both interfaces if one wanted to change only the address in e.g. IPv4.

I see, that makes sense.

jklmnn added a commit that referenced this issue Aug 14, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 18, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 19, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
jklmnn added a commit that referenced this issue Aug 21, 2020
@jklmnn
Copy link
Member

jklmnn commented Aug 21, 2020

Since we have temporarily solved this should I close this issue? @senier @treiher

@treiher
Copy link
Collaborator

treiher commented Aug 21, 2020

If it is solved and there are no plans to further work on this issue in the near future, it should be closed.

@jklmnn jklmnn closed this as completed Aug 21, 2020
RecordFlux 0.5 automation moved this from In progress to Merged Aug 21, 2020
@treiher treiher mentioned this issue Aug 4, 2021
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pyrflx Related to pyrflx package (Legacy Python API)
Projects
No open projects
RecordFlux 0.5
  
Merged
Development

No branches or pull requests

4 participants