Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider to use C++ implementation in Python runtime #469

Closed
mikir opened this issue Jan 20, 2023 · 1 comment
Closed

Consider to use C++ implementation in Python runtime #469

mikir opened this issue Jan 20, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request python Python language generator
Milestone

Comments

@mikir
Copy link
Contributor

mikir commented Jan 20, 2023

Because optimization of pure Python performance has its limitation (see #466), consider to go futher and try to use C++ implementation for some Python runtime classes.

The class BitStreamReader is a good example. Ideally, usage C++ implementation should be completely transparent for users. If zserio pypi package contains pre-compiled C++ binaries of runtime classes for current platform, they will be used automatically to speed up Python performance. If not, pure Python implementation will be used. However, it will be good if users will be able to force usage of pure Python implementation.

@mikir mikir added enhancement New feature or request python Python language generator labels Jan 20, 2023
@mikir mikir added this to the 2.10 milestone Jan 20, 2023
@mikir mikir assigned Mi-La and mikir Jan 20, 2023
Mi-La added a commit that referenced this issue Jan 27, 2023
+ fix running performance tests on multiple C++ targets
Mi-La added a commit that referenced this issue Jan 31, 2023
+ fix running performance tests on multiple C++ targets
Mi-La added a commit that referenced this issue Feb 1, 2023
TEST_ARGS are not preserved in a new Process.
Mi-La added a commit that referenced this issue Feb 2, 2023
TEST_ARGS are not preserved in a new Process.
Mi-La added a commit that referenced this issue Feb 2, 2023
Keep zserio_cpp independent of zserio package.
Mi-La added a commit that referenced this issue Feb 2, 2023
+ fix running performance tests on multiple C++ targets
Mi-La added a commit that referenced this issue Feb 2, 2023
TEST_ARGS are not preserved in a new Process.
Mi-La added a commit that referenced this issue Feb 2, 2023
Keep zserio_cpp independent of zserio package.
@Mi-La
Copy link
Contributor

Mi-La commented Feb 3, 2023

Measurements with binding of C++ BitStreamReader and BitStreamWriter:

See #466 comments for introduction to the measurements.

Currently measuring with the following command (READ,WRITE,READ_WRITE):

scripts/test_perf.sh python python-cpp -d test/language/parameterized_types/zs -s parameterized_types.zs \
        -f build/test/python/language/parameterized_types/packed_array_element_param_1000.blob \
        -b parameterized_types.packed_array_element_param.Database -n 5 -c READ
  • Python 3.8.16
Performance Tests Results - READ
========================================================================
| Generator            | Total Duration | Iterations |   Step Duration |
|----------------------------------------------------------------------|
| Python               |     3133.034ms |          5 |       626.607ms |
| Python (C++)         |     1831.597ms |          5 |       366.319ms |
========================================================================
Performance Tests Results - WRITE
========================================================================
| Generator            | Total Duration | Iterations |   Step Duration |
|----------------------------------------------------------------------|
| Python               |     5061.803ms |          5 |      1012.361ms |
| Python (C++)         |     3475.836ms |          5 |       695.167ms |
========================================================================
Performance Tests Results - READ_WRITE
========================================================================
| Generator            | Total Duration | Iterations |   Step Duration |
|----------------------------------------------------------------------|
| Python               |     8395.415ms |          5 |      1679.083ms |
| Python (C++)         |     5430.077ms |          5 |      1086.015ms |
========================================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request python Python language generator
Projects
None yet
Development

No branches or pull requests

2 participants