Updates for Pipeline Deployment #104
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
|
Kudos, SonarCloud Quality Gate passed! |
cching95
approved these changes
Mar 21, 2023
GBBBAS
added a commit
that referenced
this pull request
Mar 29, 2023
* Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>
GBBBAS
added a commit
that referenced
this pull request
May 31, 2023
* Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>
GBBBAS
added a commit
that referenced
this pull request
May 31, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
GBBBAS
added a commit
that referenced
this pull request
Jun 10, 2023
* Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>
GBBBAS
added a commit
that referenced
this pull request
Jun 19, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
cching95
added a commit
that referenced
this pull request
Jul 6, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * interpolation at time with filter Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added good/bad filter on interpolation at time Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
rodalynbarce
added a commit
that referenced
this pull request
Jul 7, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * interpolation at time with filter Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added good/bad filter on interpolation at time Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
cching95
added a commit
that referenced
this pull request
Jul 7, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added mkdocs, weather src connector, test case Signed-off-by: Pawel <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Signed-off-by: Pawel <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
rodalynbarce
added a commit
that referenced
this pull request
Jul 7, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * interpolation at time with filter Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added good/bad filter on interpolation at time Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
GBBBAS
added a commit
that referenced
this pull request
Jul 7, 2023
… MISO (#321) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Adding timezone support in ISO Sources with improved testing Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding MISO Raw to SMDM Transformer with unitesting and documentation. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Restructuring SMDM Transformers, Adding Usage & Meta data support in geneeric way. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing naming conventions and adding testcases for Meta data Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Changing MDM Types to Classes instead of String for supportive columns Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing unnecessary spaces Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing bug with Python3.10 Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Adding DF show for MisoToMDM Testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * MisoToMDM Test cases: Adding show for all DFs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing timezone from dates for MISOToMDM Transformer class Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing show statements from MisoToMDM Transformer test cases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Converting Column names into CamelCasing for ISOs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing spark.py and meters utils testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing Small code smell Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 10, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added mkdocs, weather src connector, test case Signed-off-by: Pawel <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Signed-off-by: Pawel <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 10, 2023
… MISO (#321) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Adding timezone support in ISO Sources with improved testing Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding MISO Raw to SMDM Transformer with unitesting and documentation. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Restructuring SMDM Transformers, Adding Usage & Meta data support in geneeric way. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing naming conventions and adding testcases for Meta data Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Changing MDM Types to Classes instead of String for supportive columns Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing unnecessary spaces Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing bug with Python3.10 Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Adding DF show for MisoToMDM Testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * MisoToMDM Test cases: Adding show for all DFs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing timezone from dates for MISOToMDM Transformer class Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing show statements from MisoToMDM Transformer test cases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Converting Column names into CamelCasing for ISOs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing spark.py and meters utils testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing Small code smell Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 12, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added mkdocs, weather src connector, test case Signed-off-by: Pawel <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Signed-off-by: Pawel <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 12, 2023
… MISO (#321) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Adding timezone support in ISO Sources with improved testing Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding MISO Raw to SMDM Transformer with unitesting and documentation. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Restructuring SMDM Transformers, Adding Usage & Meta data support in geneeric way. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing naming conventions and adding testcases for Meta data Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Changing MDM Types to Classes instead of String for supportive columns Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing unnecessary spaces Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing bug with Python3.10 Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Adding DF show for MisoToMDM Testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * MisoToMDM Test cases: Adding show for all DFs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing timezone from dates for MISOToMDM Transformer class Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing show statements from MisoToMDM Transformer test cases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Converting Column names into CamelCasing for ISOs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing spark.py and meters utils testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing Small code smell Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 13, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added mkdocs, weather src connector, test case Signed-off-by: Pawel <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Signed-off-by: Pawel <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
JamesKnBr
added a commit
that referenced
this pull request
Jul 13, 2023
… MISO (#321) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Adding timezone support in ISO Sources with improved testing Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding MISO Raw to SMDM Transformer with unitesting and documentation. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Restructuring SMDM Transformers, Adding Usage & Meta data support in geneeric way. Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing naming conventions and adding testcases for Meta data Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Changing MDM Types to Classes instead of String for supportive columns Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing unnecessary spaces Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing bug with Python3.10 Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Eventhub Docs (#316) * Remove doc Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to environment file Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add copyright to Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove comments on Environment.yml Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change environment.yml back now that bug is fixed Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Adding DF show for MisoToMDM Testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * MisoToMDM Test cases: Adding show for all DFs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing timezone from dates for MISOToMDM Transformer class Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing show statements from MisoToMDM Transformer test cases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Converting Column names into CamelCasing for ISOs Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing spark.py and meters utils testcases Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing Small code smell Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: PaveeG <pawegor9@gmail.com>
GBBBAS
added a commit
that referenced
this pull request
Jul 27, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added weather transformer, docs, test - modified source weather Signed-off-by: PaveeG <pawegor9@gmail.com> * refactored code Signed-off-by: PaveeG <pawegor9@gmail.com> * modified weather source unit test Signed-off-by: PaveeG <pawegor9@gmail.com> * reformated code Signed-off-by: PaveeG <pawegor9@gmail.com> * updated camel casing in weather transformer and tests Signed-off-by: PaveeG <pawegor9@gmail.com> * removed spark session from tests Signed-off-by: PaveeG <pawegor9@gmail.com> * changed source test case Signed-off-by: PaveeG <pawegor9@gmail.com> * added weather transformer Signed-off-by: PaveeG <pawegor9@gmail.com> * added component Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicates Signed-off-by: PaveeG <pawegor9@gmail.com> * changed if expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * fixed logic expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicate strings in test Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
JamesKnBr
added a commit
that referenced
this pull request
Aug 1, 2023
* remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added weather transformer, docs, test - modified source weather Signed-off-by: PaveeG <pawegor9@gmail.com> * refactored code Signed-off-by: PaveeG <pawegor9@gmail.com> * modified weather source unit test Signed-off-by: PaveeG <pawegor9@gmail.com> * reformated code Signed-off-by: PaveeG <pawegor9@gmail.com> * updated camel casing in weather transformer and tests Signed-off-by: PaveeG <pawegor9@gmail.com> * removed spark session from tests Signed-off-by: PaveeG <pawegor9@gmail.com> * changed source test case Signed-off-by: PaveeG <pawegor9@gmail.com> * added weather transformer Signed-off-by: PaveeG <pawegor9@gmail.com> * added component Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicates Signed-off-by: PaveeG <pawegor9@gmail.com> * changed if expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * fixed logic expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicate strings in test Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
GBBBAS
added a commit
that referenced
this pull request
Aug 2, 2023
* add python delta source and destination Signed-off-by: JamesKnBr <james.broady@shell.com> * Updates for destination Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * add delta source and destination Signed-off-by: JamesKnBr <james.broady@shell.com> * update delta destination Signed-off-by: JamesKnBr <james.broady@shell.com> * added tests to delta source Signed-off-by: JamesKnBr <james.broady@shell.com> * Added delta destination test Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added init Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Doc updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * tidy delta source and destination Signed-off-by: JamesKnBr <james.broady@shell.com> * doc updates and paramater fix (#398) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix meta data tag name error and update unit test (#401) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Feature/00320 - Weather Transformer (#387) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Rebase main (#308) * remove print statement in twa Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * v0.2.0 (#121) * Add Releases Feed to Docs Release page (#69) * fixes Add Releases Feed to Docs page #68 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Mkdocs test fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add unit tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smell Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Packages and Doc Logos (#76) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Logo Alignment (#78) * Upgrade Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade Functions Packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logos Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add logo to pypi readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove pydantic models Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove strawberry type Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove GraphQL Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade GraphQL Package Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo Size Changes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Logo fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #77 (#79) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * add spark session (#80) * set up spark sessions Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * adding unit tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove tests for spark session Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add constants in pipeline utils Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Pipelines as Extras libraries (#82) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Spark pipeline source Event hub and destination Event hub (#83) * add source eventhub to pipelines Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates for documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * updates to mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for eventhub source Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix spelling error Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add source eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * added destination source documentation Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove unit tests to revisit later Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix mkdoc errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix code smells Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Spark pipeline source delta and destination delta (#84) * Spark pipeline source delta and destination delta Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * fix mkdocs errors Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * docstring amendments Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Added to docstrings on delta source & destination Signed-off-by: JamesKnBr <James.Broady@Shell.com> * added query type for eventhubs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> * Added delta sharing with docs (#85) Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Unit Tests for sources (#88) * add pipeline utils unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Unit Test Fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove test spark Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for destination delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix eventhub spell errors Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add pipeline eventhub body transformer (#89) * add transform eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * transformer docs set up Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * documentation for transformer eventhub body Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add transformer eventhub unit tests Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Added pipeline source autoloader and unit test (#93) Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * Pipeline Executor (#95) * Dependency Injector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Utilities Interface Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Executor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Pipeline Execute Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * fixes #90 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Step Order fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delta Table Create Utility (#97) * Add to docs site Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Delta Table Name Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Delta Table Create Test Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * remove print statement in twa (#99) Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * Pipeline Deployment (#102) * Add DBX Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment Updates (#103) * Pipeline Deployment Updates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to run logic of Pipeline Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment (#104) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit test for spark pipeline eventhub destination (#105) * eventhub destination unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for spark eventhub destination Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add spark eventhub write stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception handling for eventhub and delta Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * update exception in utilities Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Fix for Dependency Injection (#106) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Unit tests for source and destination streams (#108) * streaming unit test for source eventhub Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add delta read stream unit test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add unit test for delta destination streaming Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove extra test Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * remove code smell Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * add previous line of code back Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Pipeline Deployments and Pipeline Secrets (#111) * Fix for Dependency Injection Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Deployment And Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add System Type Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Secrets Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Refactor Pipeline Secret Conversion Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * import required objects Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smell Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Added fails tests to pipeline sources (#112) * Added fails tests to pipeline sources Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Added fails tests for pipeline destinations Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fixed code smell Signed-off-by: JamesKnBr <James.Broady@Shell.com> --------- Signed-off-by: JamesKnBr <James.Broady@Shell.com> * Fix for dockerbuild error (#114) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for docker build (#115) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Cleanup Dockerfile (#116) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pipeline Secrets, Converters and Deployment Updates (#119) * Updates for Pipelines Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DBX Documentation Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for Pipeline Deployment Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove package.json Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix code smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * time weighted average bug fix (#120) * fix twa group by dataframe return Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * bug fixes for groupby and apply method Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> * v0.4.0 (#304) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Feature/00264 (#272) * EdgeX Transformer (#271) * added transformer component Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added edgeX test Signed-off-by: JamesKnBr <james.broady@shell.com> * update test Signed-off-by: JamesKnBr <james.broady@shell.com> * test updates Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * added kafka destination write format Signed-off-by: JamesKnBr <james.broady@shell.com> * component update Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * removed timestamp format Signed-off-by: JamesKnBr <james.broady@shell.com> * updated docs Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Adding support for MISO Sources Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Fixing sonar smells Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Removing ISO Runner file Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> * Making ISO methods internal Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> * Add Partition Pruning to non broadcast join PCDM Merge (#274) * Add Partition Pruning on non broadcast join merge Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update DropDuplicates Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix Drop Duplicates Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * update docs (#286) * update docs Signed-off-by: JamesKnBr <james.broady@shell.com> * linked issues page Signed-off-by: JamesKnBr <james.broady@shell.com> * changed issues link Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Co-authored-by: JamesKnBr <james.broady@shell.com> * Fix mkdocs colour bug (#291) * update Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour issue on mkdocs Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * fix colour bug on homepage Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> * Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296) * Additional OPC Publisher Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Device ID Filter Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Docs Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update to Filter Logic Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Weather Domain and Data Model (#294) * added new overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * adding signoff Signed-off-by: PaveeG <pawegor9@gmail.com> * added overview and data model Signed-off-by: PaveeG <pawegor9@gmail.com> * Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com> Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: PaveeG <pawegor9@gmail.com> * Refactor for odbc and functions folders (#299) * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Pyspark 3.4.0 and delta-spark 2.4.0 support (#297) * Preview Support of Pyspark 3.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fix for running github actions tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for delta-spark release candidate Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Downgrade Turbodbc to include 3.8 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add Spark Connect Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to packages Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for release of 2.4.0 Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add hvac back Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Fixes for tests Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Remove Whitespace Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Build sdist and wheel Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Change micromamba Action Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Tests Workflow Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code Smells Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Sonarqube Bug Fix Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update Readme with logo Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Center table Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update readme Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Folder Refactor Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Code smell fixes Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates for refactoriung spark connector Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Add OPC UA Transformer parameters (#301) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Micromamba Github Actions Parameters (#302) * Update micromamba parameters Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Update for Sonarqube Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Upgrade FastAPI package (#303) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <james.broady@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Resolve imports on deprecated folders Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Updates to remove unwanted files Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> * Merge main to develop Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * added weather transformer, docs, test - modified source weather Signed-off-by: PaveeG <pawegor9@gmail.com> * refactored code Signed-off-by: PaveeG <pawegor9@gmail.com> * modified weather source unit test Signed-off-by: PaveeG <pawegor9@gmail.com> * reformated code Signed-off-by: PaveeG <pawegor9@gmail.com> * updated camel casing in weather transformer and tests Signed-off-by: PaveeG <pawegor9@gmail.com> * removed spark session from tests Signed-off-by: PaveeG <pawegor9@gmail.com> * changed source test case Signed-off-by: PaveeG <pawegor9@gmail.com> * added weather transformer Signed-off-by: PaveeG <pawegor9@gmail.com> * added component Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicates Signed-off-by: PaveeG <pawegor9@gmail.com> * changed if expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * fixed logic expression Signed-off-by: PaveeG <pawegor9@gmail.com> * merge Signed-off-by: PaveeG <pawegor9@gmail.com> * removed duplicate strings in test Signed-off-by: PaveeG <pawegor9@gmail.com> --------- Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: JamesKnBr <James.Broady@Shell.com> Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com> * API Updates (#404) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * AMQP value decoding from Kafka Headers (#403) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Return Properties initial value if not populated (#405) Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> * Delete run_delta_stream.py Signed-off-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> * fix code smells Signed-off-by: JamesKnBr <james.broady@shell.com> * Added test for dataframe type Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> * updated delta destination and yaml Signed-off-by: JamesKnBr <james.broady@shell.com> * remove delta-sharing from yaml Signed-off-by: JamesKnBr <james.broady@shell.com> * add python delta sharing source Signed-off-by: JamesKnBr <james.broady@shell.com> --------- Signed-off-by: JamesKnBr <james.broady@shell.com> Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com> Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com> Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Signed-off-by: JamesKnBr <James.Broady@Shell.com> Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com> Signed-off-by: PaveeG <pawegor9@gmail.com> Signed-off-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com> Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com> Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com> Co-authored-by: PaveeG <pawegor9@gmail.com> Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com> Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com> Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com> Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com> Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.