Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/pred2bq bulk update #230

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Commits on May 15, 2023

  1. Configuration menu
    Copy the full SHA
    c5d3250 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1ed693e View commit details
    Browse the repository at this point in the history
  3. pred2bq: Refactor executor.py.

    - Adds unit tests
    - Also adds credits to original code author
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    9708b46 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7f25633 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f4ca221 View commit details
    Browse the repository at this point in the history
  6. pred2bq: Add integration test - executor to BQ

    Adds a test that runs the executor module's Beam pipeline using
    a DirectRunner and exports prediction data to an actual BigQuery table.
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    6938e58 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    cd17b95 View commit details
    Browse the repository at this point in the history
  8. pred2bq: Update utils.py.

    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    60faa2d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    e4d78ce View commit details
    Browse the repository at this point in the history
  10. Add Vertex AI Pipelines test.

    Changes:
    - Refactors the component integrate test and adds a test to run the
      component on Vertex AI Pipelines.
    
    - Adds a Dockerfile to package the component code into
      a Docker image based on OSS TFX image. The image can then be used
      as the base image when running a pipeline in Vertex AI.
    
    - Updates the `bigquery_export` output of the predictions-to-bigquery
      to store the generated BigQuery table name. This aids with checking
      the output of the component during testing, but also allows any
      downstream component receive this component's output.
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    497e3f9 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    c68667b View commit details
    Browse the repository at this point in the history
  12. pred2bq: Add integration test with transform.

    Adds a test that integrates the transform component into the pipeline.
    Test is implemented for local runner only.
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    8f48517 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    f8a53d2 View commit details
    Browse the repository at this point in the history
  14. pred2bq: Add Transform component in Vertex AI test.

    Adds a container component stub to represent the TFX Transform component
    for integration testing on Vertex AI.
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    4fca0a9 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    b57d02a View commit details
    Browse the repository at this point in the history
  16. pred2bq: Add readme file.

    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    a33b419 View commit details
    Browse the repository at this point in the history
  17. pred2bq: Replace abseil tempfile creation.

    Replaces create_tempfile and create_tempdir calls from abseil's
    absltest.TestCase and parameterized.TestCase with equivalent
    methods from the tempfile package.
    
    The reason is that the abseil methods require parsing of the
    FLAGs variable, which may not be executed if absltest.main() is
    not invoked. This can happen when test filtering is performed, e.g.
    ```
    python -m unittest path.to.test
    ```
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    6df1007 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    685cd60 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    f68f0d7 View commit details
    Browse the repository at this point in the history
  20. Update top-level readme.

    Mentions the predictions-to-bigquery component in top-level readme.
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    6035c17 View commit details
    Browse the repository at this point in the history
  21. Update code based on reviewer comments.

    - Fix issues in pred2bq readme
    - Reverted version change in setup.py
    - Add abls-py test prerequisite in setup.py
    cfezequiel committed May 15, 2023
    Configuration menu
    Copy the full SHA
    d091922 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. Configuration menu
    Copy the full SHA
    a5de9d2 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2023

  1. Configuration menu
    Copy the full SHA
    91feb1c View commit details
    Browse the repository at this point in the history