Skip to content

Docs: pems_data architecture and usage #196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 30, 2025
Merged

Docs: pems_data architecture and usage #196

merged 11 commits into from
Jul 30, 2025

Conversation

thekaveman
Copy link
Member

@thekaveman thekaveman commented Jul 28, 2025

Closes #197

  • Adds documentation for the pems_data package.
  • Installs mkdocstrings and mkdocstrings-python to autogenerate docs from Python docstrings.
  • Updates docstrings and code layout to generate more complete and useful docs

PREVIEW: https://pems-196--compiler-previews.netlify.app/development/pems_data/

@thekaveman thekaveman self-assigned this Jul 28, 2025
Copy link

github-actions bot commented Jul 28, 2025

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  pems_data/src/pems_data
  __init__.py
  cache.py
  serialization.py
  pems_data/src/pems_data/services
  stations.py
  pems_data/src/pems_data/sources
  __init__.py
  cache.py
  s3.py
Project Total  

This report was generated by python-coverage-comment-action

@thekaveman thekaveman marked this pull request as ready for review July 28, 2025 15:52
@thekaveman thekaveman requested review from a team as code owners July 28, 2025 15:52
@thekaveman
Copy link
Member Author

I opened this PR for review because I thought it would generate the docs preview... maybe it has to target the main branch?

@thekaveman thekaveman force-pushed the feat/pems-data-caching branch from 333392d to 826539e Compare July 28, 2025 22:30
@thekaveman thekaveman force-pushed the docs/pems-data branch 5 times, most recently from e9788dd to 98ae5e7 Compare July 28, 2025 22:48
@thekaveman thekaveman marked this pull request as draft July 28, 2025 22:49
@thekaveman thekaveman force-pushed the docs/pems-data branch 4 times, most recently from 6db8da9 to de39ca1 Compare July 29, 2025 00:03
@thekaveman thekaveman force-pushed the feat/pems-data-caching branch 2 times, most recently from ba373c0 to 1f4d38e Compare July 29, 2025 17:15
Base automatically changed from feat/pems-data-caching to main July 29, 2025 19:23
@thekaveman thekaveman marked this pull request as ready for review July 29, 2025 19:24
@thekaveman thekaveman marked this pull request as draft July 29, 2025 19:30
because now mkdocs needs to read metadata from pems_data
Copy link

@thekaveman thekaveman marked this pull request as ready for review July 29, 2025 19:38
notice during mkdocs build:

    Material emoji logic has been officially moved into mkdocs-material
    version 9.4. Please use Material's 'material.extensions.emoji.twemoji'
    instead of 'materialx.emoji.twemoji' in your 'mkdocs.yml' file.

    ```
    markdown_extensions:
    - pymdownx.emoji:
        emoji_index: !!python/name:material.extensions.emoji.twemoji
        emoji_generator: !!python/name:material.extensions.emoji.to_svg
    ```

    'mkdocs_material_extensions' is deprecated and will no longer be
    supported moving forward. This is the last release.
@thekaveman thekaveman added the documentation Improvements or additions to documentation label Jul 29, 2025
Copy link
Member

@lalver1 lalver1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These docs are really helpful and they look nice!

Just something I noticed, the Caching layer section has a top level mod (module) section header, but the other sections don't, they go directly to the class level. We could, for example, change data-sources.md to what is shown in the comment to have a mod top level, but it's not a big deal, just thought about mentioning it.


The data source components are responsible for the actual reading of data (the "how"). The design uses an abstract interface, `IDataSource`, to define a standard contract for any data source, making it easy to swap and compose implementations.

::: pems_data.sources.IDataSource
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also go to the top module level:

::: pems_data.sources

::: pems_data.sources.s3

::: pems_data.sources.cache

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can come back to this? I agree it would make more sense to link to the module level if e.g. there were more classes / helper functions etc. in those modules.

@thekaveman thekaveman merged commit 44e9242 into main Jul 30, 2025
9 checks passed
@thekaveman thekaveman deleted the docs/pems-data branch July 30, 2025 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document pems_data package and usage
2 participants