Skip to content

chore: refactor examples#209

Merged
karel-rehor merged 36 commits intomainfrom
chore/refactor-examples
Apr 29, 2026
Merged

chore: refactor examples#209
karel-rehor merged 36 commits intomainfrom
chore/refactor-examples

Conversation

@karel-rehor
Copy link
Copy Markdown
Contributor

Refactors examples.

Renames Examples dir to examples.

Further changes described in the following notes.

Refactoring Notes

  1. Remove pokemon-trainer and refactor examples to core, write and query
    1. basic-write-errorhandling.py can be removed. DONE
      • actually shows error handling for query
      • duplicates examples handle_http_error.py and handle_query_error.py
    2. basic-write-writeoptions.py to ./write refactored DONE
    3. pandas-write.py can be removed - duplicates ./pandas_write.py DONE
    4. write-batching.py can be removed - duplicates ./batching_example.py DONE
    5. write-batching-flight-call-options.py to be removed DONE
      • This is an odd example. Sets write options then performs only a query. Also flight applies only to query API.
      • apparently depends on other examples
      • write and query options are illustrated in other examples
    6. cookbook.ipynb - most of this is migrated to ./jupyter/basic-write-query.ipynb - can be removed DONE
      • what to do about step write table to parquet file? - not necessary DONE.
  2. Keep file-import as single file, with source data updatable - see updater.py DONE
  3. Keep one simple example of jupiter notebook. DONE
  4. Decide what to do with ./community examples.
    • custom_url.py DONE
      • Doesn't seem to do what is on the tin.
      • Seems more like a down sampling example
      • Doesn't query Influxdb directly but down samples from remote CSV
      • review/enhance DONE
        • rename to down sampling or similar DONE
        • use initial query DONE
        • move to ./advanced DONE
        • remove dependency on githubusercontent DONE
    • database_transfer.py
      • Simply copies data from one bucket to another
      • Useful base example
      • Current illustrative only - doesn't look functional without some undocumented setup
      • To work requires dbfrom and measurement airSensors to exist.
      • review DONE
        • move to ./advanced DONE
        • make functional kept as illustrative - advanced/downsample.py is similar and functional
  5. Root examples
    1. basic_ssl_examle.py to ./core DONE
    2. batching_example.py to ./write DONE
      1. Issue - getting interpreter shutdown before example ends. - fix, reduce data set size DONE
      2. Issue - runs very slow - fix, stop printing full data set with every write. DONE
    3. cloud_dedicated_query.py - is it necessary to have specific cloud_dedicated examples? Can this simply be documented in README.md or in code comments? (Removed DONE)
    4. cloud_dedicated_write.py - is it necessary to have specific cloud_dedicated examples? Can this simply be documented in README.md or in code comments? (Removed DONE)
    5. config.py - universal configuration file. Keep as is.
    6. example.csv - where is this used? It doesn't seem to be used in any example... ??? (Removed DONE)
    7. flight_options_example.py - to ./query DONE
      • Note only option illustrated is tls certificate.
      • after move - either enrich/refactor or verify this isn't covered elsewhere DONE
    8. handle_http_error.py - to ./write DONE
    9. handle_query_error.py - to ./query DONE
    10. pandas_write.py - to ./write DONE
    11. query_async.py - to ./query DONE
    12. query_type.py - rename to query_modes.py. Move to ./query DONE
    13. query_with_middleware.py - to ./query DONE
    14. timeouts.py - to ./core DONE
  6. Standardization
    1. Some examples show leveraging internal **kwargs like data_frame_measurement_name or data_frame_tag_columns DONE
      • Makes sense to expose these in advanced examples. (e.g. pandas examples... )
      • Perhaps though should encourage the use of a simpler standard API that hides them
    2. Remove dependencies on remote ../githubusercontent/../*.csv DONE
    3. Leverage config.py in all examples DONE
      1. Undo and use standard ENVARS instead DONE
    4. Prefer using Influxdb3 Core by default. But also document possibility of using other products.
    5. Add shebangs to functional examples DONE
    6. Ensure timestamps are current and not fixed for example to 2023 DONE
  7. Enhancements
    1. writeoptions.py - does not show much in the way of setting options. (Update and revision - DONE)
    2. basic_ssl.py - review. Seems to only show handling SSL handshake failures. (Reviewed and updated - DONE)
    3. write_pandas.py - has fixed dates from 2023, make dynamic with current. DONE
    4. flight_options.py - review. This example is nearly three years old, and flight/query options has been greatly enhanced since then. DONE
  8. Verify all refactored examples are working
    *. NOTE - If making an illustrative example functional out-of-the-box leads to too much distractive information being added, leave the example as illustrative only and add a comment that it is for purposes of illustration. However, make sure the illustrative example is still working in a concrete implementation. (e.g. query_with_middleware.py)
    *. In comments, mark examples as either illustrative or functional DONE

Standardization summary (F - functional, I - illustrative)

example F/I Head Comment shebang Config() ENVARS **kwargs (adv) github user link Timestamps Notes
advanced/database_transfer.py I Yes None None Yes Yes (pd) None N.A. - copied without change Ready
advanced/downsample.py F Yes Yes None Yes Yes (pd) None Dynamic / Current Ready
core/basic_write.py F Yes Yes None Yes None None Dynamic / Now Ready
core/basic_query.py F Yes Yes None Yes None None Read only Ready
core/basic_ssl.py F Yes Yes None Yes None None Dynamic / Now Ready
core/timeouts.py F Yes Yes None Yes None None Now (implicit) Ready
jupyter/basic-write-query.ipynb F N/A N/A N/A Yes None None Dynamic Ready
query/flight_options.py I Yes None None Yes None None Read only Ready
query/handle_query_error.py F Yes Yes None Yes None None Read only Ready
query/query_async.py F Yes Yes None Yes None None Dynamic Ready
query/query_modes.py F Yes Yes None Yes None None Dynamic Ready
query/query_with_middleware.py I Yes None None Yes None None Read only Ready
write/batching.py F Yes Yes None Yes None None Dynamic Ready
write/fileimport.py F Yes Yes None Yes Yes (file_write()) None Dynamic Ready - kwargs here are part of example
write/handle_http_error.py F Yes Yes None Yes None None N.A. Ready - shows write error
write/pandas_write.py F Yes Yes None Yes Yes (pd) None Dynamic Ready - kwargs here are part of example
write/writeoptions F Yes Yes None Yes None None Dynamic / now Ready

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate N.A.
  • Tests pass
  • Commit messages are conventional
  • Sign CLA (if not already signed)

@karel-rehor karel-rehor force-pushed the chore/refactor-examples branch from 036586f to 9a7e664 Compare April 28, 2026 15:50
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.17%. Comparing base (24b8a89) to head (270d786).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #209   +/-   ##
=======================================
  Coverage   75.17%   75.17%           
=======================================
  Files          35       35           
  Lines        2417     2417           
=======================================
  Hits         1817     1817           
  Misses        600      600           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@bednar bednar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors and standardizes the repository’s Python examples by consolidating them under a new lowercase examples/ layout, removing older/duplicated example sets, and adding updated “core / write / query / advanced / jupyter” examples intended to be either functional or explicitly illustrative.

Changes:

  • Move/replace legacy Examples/ content with a new structured examples/ tree (core/write/query/advanced/jupyter) and update CI linting to target it.
  • Add/refresh functional and illustrative example scripts (timeouts, batching, pandas, file import, query modes, async query, middleware, SSL).
  • Add supporting example assets (source data files + updater helper) and an examples/README.md plus a small prep helper script.

Reviewed changes

Copilot reviewed 45 out of 53 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
examples/write/writeoptions.py New write options example; adds illustrative callbacks and sync write usage.
examples/write/fileimport.py New file-import example that writes from local source_data files and uses batching callbacks.
examples/write/batching.py Updated batching write example; switches to env vars and reduces output verbosity.
examples/write/pandas_write.py New pandas DataFrame write example with dynamic timestamps.
examples/write/handle_http_error.py Updates write error-handling example to use env vars and adds shebang/header.
examples/write/source_data/* Adds local CSV/JSON/Feather/ORC/Parquet source data plus updater script and pokemon dataset.
examples/query/query_modes.py New example demonstrating query modes (chunk/pandas/all/schema/reader).
examples/query/query_async.py Updates async query example; adds env vars, shebang, and naming fixes.
examples/query/query_with_middleware.py Updates middleware example to use env vars and adds descriptive header.
examples/query/handle_query_error.py Updates query error-handling example to use env vars and direct imports.
examples/query/flight_options.py New illustrative example for Arrow Flight / gRPC options.
examples/core/basic_write.py New basic “Point write” example using env vars.
examples/core/basic_query.py New basic query example meant to follow basic_write.
examples/core/basic_ssl.py Updates SSL example to use env vars and expanded guidance.
examples/core/timeouts.py Updates timeouts example to use env vars and improved header docs.
examples/advanced/downsample.py New functional downsampling example using local pokemon CSV.
examples/advanced/database_transfer.py Updates illustrative transfer example to use env vars and reorganized imports.
examples/jupyter/basic-write-query.ipynb Adds a basic write/query Jupyter notebook walkthrough.
examples/prep.py Adds helper to install extra deps and chmod shebang-based example scripts.
examples/README.md New top-level documentation describing example structure and configuration.
.circleci/config.yml Updates flake8 path from Examples/ to examples/.
Comments suppressed due to low confidence (1)

examples/write/batching.py:60

  • The comments/logging imply target_sample_count is the number of points written, but the nested for gatewayId in gatewayIds loop means the script writes target_sample_count * len(gatewayIds) points. Consider renaming variables and updating any progress/summary messages so they reflect the actual number of points written (and fix the nearby comment that mentions 5,000 gatewayIds vs the 100 created).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread examples/query/flight_options.py Outdated
Comment thread examples/write/pandas_write.py
Comment thread examples/query/query_modes.py Outdated
Comment thread examples/query/flight_options.py
Comment thread examples/write/writeoptions.py Outdated
Comment thread examples/write/source_data/updater.py Outdated
Comment thread examples/write/source_data/updater.py
Comment thread examples/core/basic_query.py Outdated
Comment thread examples/write/fileimport.py Outdated
Comment thread examples/query/flight_options.py Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 45 out of 53 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread examples/query/flight_options.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 45 out of 53 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

examples/query/query_async.py:83

  • InfluxDBClient3 is created without a context manager or an explicit close(). Since the client manages network resources/threads, this example may leak resources or hang at shutdown in some environments. Prefer with InfluxDBClient3(...) as client: (even inside the async main) or ensure client.close() is called in a finally block.
    examples/write/batching.py:37
  • The write batching callbacks annotate data as str, but the client delivers the batch payload as raw line-protocol bytes (the write API encodes strings before enqueueing). Update the type annotations to bytes (and keep any formatting consistent with bytes) to prevent confusion and accidental double-encoding in copied code.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread examples/write/writeoptions.py
Comment thread examples/write/fileimport.py
Comment thread examples/prep.py Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 45 out of 53 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@karel-rehor karel-rehor merged commit 592b6f7 into main Apr 29, 2026
20 checks passed
@karel-rehor karel-rehor deleted the chore/refactor-examples branch April 29, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants