fix(SDK): Artifact upload fails when same file referred multiple times #715

destifo · 2024-05-12T14:17:24Z

fix the bug where duplicate artifact references causing failure during artifact resolution(typegate) during runtime.
add sync mode tests for Python and Deno runtime.
add other edge test cases to artifact upload.
- test for no artifact in typegraph
- test for duplicate artifact reference in the same typegraph

Migration notes

No Migrations Needed

...

Summary by CodeRabbit

New Features
- Introduced testing functionalities for Deno and Python runtimes, supporting artifact deployment and policy enforcement.
- Added Python and Deno scripts for deploying Typegraphs and handling runtime configurations.
Bug Fixes
- Improved artifact handling and deployment logic in both Python and Deno runtimes.
Tests
- Added comprehensive test cases for Deno and Python runtimes, including scenarios for duplicate artifacts, no artifacts, and runtime synchronization.
- Enhanced testing with integration of various services like Redis and S3.
Documentation
- Updated test scripts to include detailed comments and summaries for new functionalities and test scenarios.

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com>

…time

… resolution during runtime

…est/artifact-tests

coderabbitai · 2024-05-14T11:09:12Z

Walkthrough

The recent changes introduce and enhance functionalities for testing Deno and Python runtimes within the TypeGraph framework. This involves adding new scripts for handling artifact deployments, defining policies, and setting up runtime environments. Additionally, several test cases have been implemented to ensure the proper functioning of these features, including scenarios for no artifacts, duplicate artifacts, and runtime synchronization.

Changes

Files	Change Summary
`typegate/tests/runtimes/deno/deno_duplicate_artifact.py`	Introduced `deno_duplicate_artifact` function for setting up and deploying artifacts in Deno runtime.
`typegate/tests/runtimes/deno/deno_duplicate_artifact.ts`	Added `denoDuplicateArtifact` function for handling duplicate artifact imports and deployments in Deno.
`typegate/tests/runtimes/deno/deno_no_artifact.py`	Added `deno_no_artifact` function for deploying Typegraph in Deno without artifacts.
`typegate/tests/runtimes/deno/deno_sync_test.ts`	Introduced tests for Deno runtime, covering setup, teardown, network access, and various deployment scenarios.
`typegate/tests/runtimes/deno/deno_test.ts`	Added tests for Deno runtime with Python SDK focusing on no artifacts and duplicate artifacts scenarios.
`typegate/tests/runtimes/python/python_duplicate_artifact.py`	Added `python_duplicate_artifact` function for importing modules and deploying artifacts in Python runtime.
`typegate/tests/runtimes/python/python_duplicate_artifact.ts`	Introduced `tgDuplicateArtifact` for defining and exposing Python artifacts with policies.
`typegate/tests/runtimes/python/python_no_artifact.py`	Added `python_no_artifact` function for deploying Typegraph in Python without artifacts.
`typegate/tests/runtimes/python/python_no_artifact.ts`	Defined `identityLambda` and `identityDef` functions for identity transformations in Python runtime.
`typegate/tests/runtimes/python/python_sync_test.ts`	Introduced tests for Python runtime synchronization, covering artifact uploads, multiple instances, and various deployment scenarios.
`typegate/tests/runtimes/python/python_test.ts`	Added and modified tests for Python runtime focusing on no artifacts and duplicate artifact uploads.
`typegraph/core/src/utils/postprocess/python_rt.rs`	Modified `PythonProcessor` logic for handling artifacts and dependencies.

Recent Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5db3ef8 and d93b912.

Files selected for processing (12)

typegate/tests/runtimes/deno/deno_duplicate_artifact.py (1 hunks)
typegate/tests/runtimes/deno/deno_duplicate_artifact.ts (1 hunks)
typegate/tests/runtimes/deno/deno_no_artifact.py (1 hunks)
typegate/tests/runtimes/deno/deno_sync_test.ts (1 hunks)
typegate/tests/runtimes/deno/deno_test.ts (1 hunks)
typegate/tests/runtimes/python/python_duplicate_artifact.py (1 hunks)
typegate/tests/runtimes/python/python_duplicate_artifact.ts (1 hunks)
typegate/tests/runtimes/python/python_no_artifact.py (1 hunks)
typegate/tests/runtimes/python/python_no_artifact.ts (1 hunks)
typegate/tests/runtimes/python/python_sync_test.ts (1 hunks)
typegate/tests/runtimes/python/python_test.ts (2 hunks)
typegraph/core/src/utils/postprocess/python_rt.rs (2 hunks)

Additional comments not posted (24)

typegate/tests/runtimes/python/python_no_artifact.ts (1)

1-34: LGTM! The typegraph definition for Python runtime without artifacts is clear and follows best practices.

typegate/tests/runtimes/deno/deno_duplicate_artifact.ts (1)

1-33: LGTM! The typegraph definition for Deno runtime with duplicate artifact references is clear and follows best practices.

typegate/tests/runtimes/python/python_duplicate_artifact.ts (1)

1-39: LGTM! The typegraph definition for Python runtime with duplicate artifact references is clear and follows best practices.

typegate/tests/runtimes/deno/deno_no_artifact.py (1)

1-59: LGTM! The typegraph definition for Deno runtime without artifacts is clear and follows best practices.

typegate/tests/runtimes/python/python_no_artifact.py (1)

1-58: LGTM! The typegraph definition for Python runtime without artifacts is clear and follows best practices.

typegate/tests/runtimes/python/python_duplicate_artifact.py (1)

1-67: LGTM! The typegraph definition for Python runtime with duplicate artifact references is clear and follows best practices.

typegate/tests/runtimes/deno/deno_duplicate_artifact.py (1)

1-68: LGTM! The typegraph definition for Deno runtime with duplicate artifact references is clear and follows best practices.

typegraph/core/src/utils/postprocess/python_rt.rs (1)

Line range hint 10-66: LGTM! The modifications to the PythonProcessor implementation are clear and follow best practices.

typegate/tests/runtimes/deno/deno_test.ts (2)

406-428: Test case for "Deno runtime - Python SDK: with no artifacts in sync mode" looks good.

453-477: Test case for "Deno runtime - Python SDK: with duplicate artifacts in sync mode" looks good.

typegate/tests/runtimes/deno/deno_sync_test.ts (4)

65-155: Test case for "Deno runtime - Python SDK: in sync mode" looks good.

158-199: Test case for "Deno runtime - Python SDK: file name reloading in sync mode" looks good.

201-229: Test case for "Deno runtime - Python SDK: use local imports in sync mode" looks good.

503-533: Test case for "Deno runtime - Python SDK: with no artifacts in sync mode" looks good.

typegate/tests/runtimes/python/python_test.ts (2)

439-466: Test case for "PythonRuntime - Python SDK: typegraph with no artifacts in sync mode" looks good.

527-556: Test case for "Python Runtime - Python SDK: typegraph with duplicate artifact uploads" looks good.

typegate/tests/runtimes/python/python_sync_test.ts (8)

68-161: The test case for "Python Runtime typescript SDK: with Sync Config" is comprehensive and well-structured. Ensure that the cleanUp function is robust, as suggested earlier.

163-260: The test case for "Python runtime: sync mode" is thorough and covers various query operations. Ensure that the cleanUp function is robust, as suggested earlier.

263-301: The test case for "Python runtime: multiple typegate instances sync mode" is well-structured and covers the necessary scenarios. Ensure that the cleanUp function is robust, as suggested earlier.

303-375: The test case for "Deno: def, lambda in sync mode" is comprehensive and well-structured. Ensure that the cleanUp function is robust, as suggested earlier.

378-406: The test case for "Python: infinite loop or similar in sync mode" is well-structured and covers the necessary scenarios. Ensure that the cleanUp function is robust, as suggested earlier.

408-491: The test case for "Python: typegate reloading in sync mode" is comprehensive and well-structured. Ensure that the cleanUp function is robust, as suggested earlier.

494-528: The test case for "PythonRuntime - Python SDK: typegraph with no artifacts in sync mode" is well-structured and covers the necessary scenarios. Ensure that the cleanUp function is robust, as suggested earlier.

596-632: The test case for "Python - Python SDK: typegraph with duplicate artifact uploads in sync mode" is well-structured and covers the necessary scenarios. Ensure that the cleanUp function is robust, as suggested earlier.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 5

Out of diff range and nitpick comments (4)

typegate/tests/runtimes/python/python_sync_test.ts (4)

4-13: Ensure all imported modules are necessary and used in the code.

Consider removing any unused imports to keep the code clean and maintainable.

16-18: Consider adding comments to explain the purpose of these constants.

Adding comments can improve code readability and maintainability.

31-48: The syncConfig object is well-defined, but consider adding comments to explain each section.

Adding comments can help future developers understand the configuration more easily.

530-594: Consider removing the commented-out test cases if they are no longer needed.

Keeping commented-out code can clutter the codebase. If these tests are not required, it's better to remove them.

typegate/tests/runtimes/deno/deno_test.ts

typegate/tests/runtimes/python/python_sync_test.ts

typegate/tests/runtimes/deno/deno_test.ts

typegate/tests/runtimes/python/python_test.ts

codecov · 2024-05-15T08:21:29Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.05%. Comparing base (5db3ef8) to head (d93b912).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #715      +/-   ##
==========================================
+ Coverage   73.85%   74.05%   +0.19%     
==========================================
  Files         120      120              
  Lines       13777    13777              
  Branches     1393     1408      +15     
==========================================
+ Hits        10175    10202      +27     
+ Misses       3576     3549      -27     
  Partials       26       26

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

zifeo and others added 9 commits April 27, 2024 14:16

feat: change headline

9286e1f

docs: simply pr template

0d031da

Merge remote-tracking branch 'origin/main' into polish

2002042

fix: running lock while compiling

ac51447

fix: bump

f676614

fix: lockfile

2d8669f

Update .github/pull_request_template.md

c348cd3

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com>

wip(test): add d/t edge test cases and sync test for deno, python run…

7871908

…time

fix(core SDK): fix for duplicate artifact referene breaking artifacts…

2f86b10

… resolution during runtime

destifo self-assigned this May 12, 2024

destifo requested review from afmika, Yohe-Am, Natoandro and zifeo May 12, 2024 14:17

chore(tests): add more test cases

860b9d1

destifo changed the title ~~fix(SDK): Artifact upload fails when same file refered twice~~ fix(SDK): Artifact upload fails when same file referred multiple times May 12, 2024

Base automatically changed from polish to main May 14, 2024 11:04

Merge branch 'main' of https://github.com/metatypedev/metatype into t…

d93b912

…est/artifact-tests

coderabbitai bot reviewed May 14, 2024

View reviewed changes

Yohe-Am approved these changes May 15, 2024

View reviewed changes

destifo merged commit bbdfd35 into main May 15, 2024
14 checks passed

destifo deleted the test/artifact-tests branch May 15, 2024 09:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(SDK): Artifact upload fails when same file referred multiple times #715

fix(SDK): Artifact upload fails when same file referred multiple times #715

destifo commented May 12, 2024 •

edited

coderabbitai bot commented May 14, 2024 •

edited

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

coderabbitai bot left a comment

codecov bot commented May 15, 2024

fix(SDK): Artifact upload fails when same file referred multiple times #715

fix(SDK): Artifact upload fails when same file referred multiple times #715

Conversation

destifo commented May 12, 2024 • edited

Migration notes

Summary by CodeRabbit

coderabbitai bot commented May 14, 2024 • edited

Walkthrough

Changes

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

coderabbitai bot left a comment

Choose a reason for hiding this comment

codecov bot commented May 15, 2024

Codecov Report

destifo commented May 12, 2024 •

edited

coderabbitai bot commented May 14, 2024 •

edited

CodeRabbit Configration File (`.coderabbit.yaml`)