Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve parser #209

Merged
merged 2 commits into from Aug 9, 2023
Merged

Improve parser #209

merged 2 commits into from Aug 9, 2023

Conversation

hadrianl
Copy link
Contributor

@hadrianl hadrianl commented Aug 7, 2023

resolved #204

This PR improve the performance of taos query[c -> python parser] :

  • reduce cast, memory copy and object creation
  • convert timezone once when set_tz, not everytime _convert_XXX_to_datetime
  • totally, it take only 10%-20% cpu time when dealing with binary, nchar and timestamp

further more

  • taos_is_null had been called so many time per query
  • a vector version of taos_is_null from libtaos is needed to improve this, just like taos_get_column_data_offset
  • or maybe, introducing the third party libraries such as Cython to improve this

@sangshuduo sangshuduo requested a review from sunpe August 7, 2023 02:43
Copy link
Contributor

@sangshuduo sangshuduo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter
Copy link

Codecov Report

Merging #209 (56b8ca1) into main (9dad4b5) will decrease coverage by 0.04%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main     #209      +/-   ##
==========================================
- Coverage   81.37%   81.33%   -0.04%     
==========================================
  Files          20       20              
  Lines        2652     2652              
==========================================
- Hits         2158     2157       -1     
- Misses        494      495       +1     
Files Changed Coverage Δ
taos/field.py 79.75% <25.00%> (-0.37%) ⬇️
taos/field_v3.py 93.75% <100.00%> (-0.16%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@sunpe sunpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sunpe sunpe merged commit 28b4505 into taosdata:main Aug 9, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

taos_fetch_block _v3 is slow
4 participants