Skip to content

Conversation

@cyb70289
Copy link
Contributor

@cyb70289 cyb70289 commented Aug 14, 2020

For small integers(bool, int8, uint8), instead of general hash table,
using a value indexed array improves performance about 2x ~ 6x.

@github-actions
Copy link

@cyb70289
Copy link
Contributor Author

Removed unrelated tests for clarity.
100% null benchmark is not stable, can be ignored.

$ archery benchmark diff --suite-filter="arrow-compute-aggregate-benchmark" --cc=clang-9 --cxx=clang++-9

                      benchmark         baseline        contender  change % 
   ModeKernelInt8/1048576/10000   60.196 MiB/sec  477.048 MiB/sec   692.487 'null_percent': 0.01
     ModeKernelInt8/1048576/100   60.548 MiB/sec  470.066 MiB/sec   676.351, 'null_percent': 1.0
       ModeKernelInt8/1048576/0   63.520 MiB/sec  472.362 MiB/sec   643.638, 'null_percent': 0.0
      ModeKernelInt8/1048576/10   62.997 MiB/sec  420.059 MiB/sec   566.788 'null_percent': 10.0
   ModeKernelBoolean/1048576/10    7.640 MiB/sec   28.329 MiB/sec   270.793 'null_percent': 10.0
ModeKernelBoolean/1048576/10000    7.332 MiB/sec   26.496 MiB/sec   261.362 'null_percent': 0.01
  ModeKernelBoolean/1048576/100    7.415 MiB/sec   26.710 MiB/sec   260.212, 'null_percent': 1.0
    ModeKernelBoolean/1048576/0    8.079 MiB/sec   28.428 MiB/sec   251.895, 'null_percent': 0.0
       ModeKernelInt8/1048576/2   80.291 MiB/sec  265.673 MiB/sec   230.887 'null_percent': 50.0
    ModeKernelBoolean/1048576/2   12.449 MiB/sec   27.915 MiB/sec   124.235 'null_percent': 50.0
       ModeKernelInt8/1048576/1  864.966 MiB/sec    1.313 GiB/sec    55.455'null_percent': 100.0
    ModeKernelBoolean/1048576/1  115.581 MiB/sec  153.801 MiB/sec    33.067'null_percent': 100.0
  ModeKernelInt32/1048576/10000  228.572 MiB/sec  263.524 MiB/sec    15.292 'null_percent': 0.01
    ModeKernelInt32/1048576/100  230.087 MiB/sec  265.102 MiB/sec    15.218, 'null_percent': 1.0
  ModeKernelInt16/1048576/10000  115.973 MiB/sec  132.465 MiB/sec    14.221 'null_percent': 0.01
    ModeKernelInt16/1048576/100  116.657 MiB/sec  132.673 MiB/sec    13.729, 'null_percent': 1.0
     ModeKernelInt32/1048576/10  241.338 MiB/sec  270.446 MiB/sec    12.061 'null_percent': 10.0
     ModeKernelInt16/1048576/10  122.426 MiB/sec  135.675 MiB/sec    10.822 'null_percent': 10.0
  ModeKernelInt64/1048576/10000  481.018 MiB/sec  523.582 MiB/sec     8.849 'null_percent': 0.01
    ModeKernelInt64/1048576/100  485.800 MiB/sec  527.232 MiB/sec     8.529, 'null_percent': 1.0
     ModeKernelInt64/1048576/10  526.491 MiB/sec  569.412 MiB/sec     8.152 'null_percent': 10.0
      ModeKernelInt64/1048576/1    6.671 GiB/sec    7.180 GiB/sec     7.623'null_percent': 100.0
      ModeKernelInt64/1048576/2  783.253 MiB/sec  813.436 MiB/sec     3.854 'null_percent': 50.0
      ModeKernelInt32/1048576/0  257.627 MiB/sec  265.602 MiB/sec     3.096, 'null_percent': 0.0
      ModeKernelInt16/1048576/0  128.580 MiB/sec  132.340 MiB/sec     2.924, 'null_percent': 0.0
      ModeKernelInt16/1048576/2  160.189 MiB/sec  164.776 MiB/sec     2.863 'null_percent': 50.0
      ModeKernelInt64/1048576/0  514.855 MiB/sec  528.897 MiB/sec     2.727, 'null_percent': 0.0
      ModeKernelInt32/1048576/2  321.334 MiB/sec  324.092 MiB/sec     0.858 'null_percent': 50.0
      ModeKernelInt16/1048576/1    1.860 GiB/sec    1.653 GiB/sec   -11.109'null_percent': 100.0
      ModeKernelInt32/1048576/1    5.728 GiB/sec    3.263 GiB/sec   -43.038'null_percent': 100.0

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for doing this.
A more sophisticated optimization for integer arrays would be to find the min/max, and use a direct access table if max-min is small enough (e.g. < 4096). But that can be left for another JIRA.

For small integers(bool, int8, uint8), instead of general hash table,
using a value indexed array improves performance about 2x ~ 6x.

Get another 15% improvement by reading array raw_values[] directly.
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thank you @cyb70289 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants