Skip to content

Conversation

@gatear
Copy link
Contributor

@gatear gatear commented Jun 18, 2024

Generate a stream of SQL statements, support both singular and batch statements.

@what-the-diff
Copy link

what-the-diff bot commented Jun 18, 2024

PR Summary

  • Introduction of Integer Interval Class
    The PR contains a new class to represent an interval of integers. This would essentially aid in specifying the range of rows corresponding to batch SQL statements.

  • Stream Generation Method for SQL Statements
    New method has been introduced to generate a flow of SQL statements from a schema coupled with a limit. Support has been added to generate these commands in batch mode or individually.

  • Modification in SQLTransformer's Apply Method
    The apply() method in SqlTransformer has been updated to now accept a long rowId parameter. This modification lets it handle larger amount of data.

  • Updated Start and End Stream Methods
    The getStartStream() and getEndStream() methods have been updated to throw an error when an unsupported operation is performed. The error message accompanying this is informative, allowing for better understanding of the reason for the exception.

  • Added Testing Method for SQL Statement Stream
    A new testing method testSqlStream() has been added which scrutinizes the generateStream() method for generating SQL statements. The test involves generating SQL statements based on a schema and a limit, and then compares the result with the expected output. This enhances the reliability of the newly added method.

@gatear
Copy link
Contributor Author

gatear commented Jun 20, 2024

@snuyanzin it's ready for review. This work is another critical item for my use-case with Spark.

After this change I'll follow-up on this #1261 (comment)
and show some useful examples for Spark.

Thanks 🙏

@bodiam
Copy link
Contributor

bodiam commented Jun 21, 2024

@gatear This PR looks good to me, thank you for that, I think this should land in 2.3.0

Anyone else feedback?

@kingthorin
Copy link
Collaborator

It looks okay to me, but I honestly haven't worked with or on the Transformers at all.

@snuyanzin
Copy link
Collaborator

lgtm, i left a minor comment

@codecov-commenter
Copy link

codecov-commenter commented Jun 23, 2024

Codecov Report

Attention: Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.92%. Comparing base (b37c566) to head (19c0147).
Report is 170 commits behind head on main.

Files Patch % Lines
.../datafaker/transformations/sql/SqlTransformer.java 91.66% 2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1264      +/-   ##
============================================
- Coverage     92.35%   91.92%   -0.44%     
- Complexity     2821     3090     +269     
============================================
  Files           292      310      +18     
  Lines          5609     6052     +443     
  Branches        599      636      +37     
============================================
+ Hits           5180     5563     +383     
- Misses          275      326      +51     
- Partials        154      163       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gatear gatear requested a review from snuyanzin June 23, 2024 17:38
Copy link
Collaborator

@snuyanzin snuyanzin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution and addressing feedback

@snuyanzin snuyanzin merged commit 4634e8f into datafaker-net:main Jun 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants