Support for Raw data to MDM Transformer class with Implementation for… · rtdip/core@23d2582

Commit

Support for Raw data to MDM Transformer class with Implementation for…
… MISO (#321)

* remove print statement in twa

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* v0.2.0 (#121)

* Add Releases Feed to Docs Release page (#69)

* fixes Add Releases Feed to Docs page #68

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Mkdocs test fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add unit tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Code Smell

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Packages and Doc Logos (#76)

* Upgrade Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logos

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logo to pypi readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove pydantic models

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove strawberry type

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo Size Changes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Logo Alignment (#78)

* Upgrade Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logos

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logo to pypi readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove pydantic models

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove strawberry type

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo Size Changes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* fixes #77 (#79)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* add spark session (#80)

* set up spark sessions

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* adding unit tests for spark session

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove tests for spark session

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix code smells

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add constants in pipeline utils

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Add Pipelines as Extras libraries (#82)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Spark pipeline source Event hub and destination Event hub (#83)

* add source eventhub to pipelines

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* updates for documentation

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* updates to mkdocs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* added unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* documentation for eventhub source

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix spelling error

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add source eventhub destination

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* added destination source documentation

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove unit tests to revisit later

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix mkdoc errors

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix code smells

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Spark pipeline source delta and destination delta (#84)

* Spark pipeline source delta and destination delta

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* fix mkdocs errors

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* docstring amendments

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Added to docstrings on delta source & destination

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* added query type for eventhubs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>

* Added delta sharing with docs (#85)

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Unit Tests for sources (#88)

* add pipeline utils unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for eventhub and delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Unit Test Fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* remove test spark

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for destination delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix eventhub spell errors

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add pipeline eventhub body transformer (#89)

* add transform eventhub

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* transformer docs set up

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* documentation for transformer eventhub body

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add transformer eventhub unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Added pipeline source autoloader and unit test (#93)

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Pipeline Executor (#95)

* Dependency Injector

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Executor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Utilities Interface

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Executor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Execute Test

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* fixes #90

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fixes for tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Step Order fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Sonarqube fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Delta Table Create Utility (#97)

* Add to docs site

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Delta Table Name

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Delta Table Create

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Delta Table Create Test

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* remove print statement in twa (#99)

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>

* Pipeline Deployment (#102)

* Add DBX Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Deployment Updates (#103)

* Pipeline Deployment Updates

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to run logic of Pipeline

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment (#104)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Unit test for spark pipeline eventhub destination (#105)

* eventhub destination unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for spark eventhub destination

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add spark eventhub write stream unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* update exception handling for eventhub and delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* update exception in utilities

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Fix for Dependency Injection (#106)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Unit tests for source and destination streams (#108)

* streaming unit test for source eventhub

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add delta read stream unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for delta destination streaming

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove extra test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove code smell

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add previous line of code back

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Pipeline Deployments and Pipeline Secrets (#111)

* Fix for Dependency Injection

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Deployment And Secrets

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add System Type Docs

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Secrets

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smell fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Refactor Pipeline Secret Conversion Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* import required objects

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smell Fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Code Smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Added fails tests to pipeline sources (#112)

* Added fails tests to pipeline sources

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Added fails tests for pipeline destinations

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Fixed code smell

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

---------

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Fix for dockerbuild error (#114)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix for docker build (#115)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Cleanup Dockerfile (#116)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Secrets, Converters and Deployment Updates (#119)

* Updates for Pipelines

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update DBX Documentation

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove package.json

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix code smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* time weighted average bug fix (#120)

* fix twa group by dataframe return

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* bug fixes for groupby and apply method

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>

* Adding timezone support in ISO Sources with improved testing

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Adding MISO Raw to SMDM Transformer with unitesting and documentation.

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Restructuring SMDM Transformers, Adding Usage & Meta data support in geneeric way.

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing naming conventions and adding testcases for Meta data

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Changing MDM Types to Classes instead of String for supportive columns

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Removing unnecessary spaces

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing bug with Python3.10

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* v0.4.0 (#304)

* EdgeX Transformer (#271)

* added transformer component

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added edgeX test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* update test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* test updates

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added kafka destination write format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* component update

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* removed timestamp format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* updated docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Feature/00264 (#272)

* EdgeX Transformer (#271)

* added transformer component

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added edgeX test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* update test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* test updates

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added kafka destination write format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* component update

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* removed timestamp format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* updated docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Adding support for MISO Sources

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing sonar smells

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing sonar smells

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Removing ISO Runner file

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Making ISO methods internal

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Add Partition Pruning to non broadcast join PCDM Merge (#274)

* Add Partition Pruning on non broadcast join merge

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update DropDuplicates

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Drop Duplicates Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* update docs (#286)

* update docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

* linked issues page

Signed-off-by: JamesKnBr <james.broady@shell.com>

* changed issues link

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: JamesKnBr <james.broady@shell.com>

* Fix mkdocs colour bug (#291)

* update

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix colour issue on mkdocs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix colour bug on homepage

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296)

* Additional OPC Publisher Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Device ID Filter

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Docs

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to Filter Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Weather Domain and Data Model (#294)

* added new overview and data model

Signed-off-by: PaveeG <pawegor9@gmail.com>

* adding signoff

Signed-off-by: PaveeG <pawegor9@gmail.com>

* added overview and data model

Signed-off-by: PaveeG <pawegor9@gmail.com>

* Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com>

Signed-off-by: PaveeG <pawegor9@gmail.com>

---------

Signed-off-by: PaveeG <pawegor9@gmail.com>

* Refactor for odbc and functions folders (#299)

* Folder Refactor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pyspark 3.4.0 and delta-spark 2.4.0 support (#297)

* Preview Support of Pyspark 3.4.0

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix for running github actions tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for delta-spark release candidate

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Downgrade Turbodbc to include 3.8

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Spark Connect

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates to packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for release of 2.4.0

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add hvac back

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fixes for tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove Whitespace

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Build sdist and wheel

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Change micromamba Action

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Tests Workflow

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Sonarqube Bug Fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Readme with logo

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Center table

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Folder Refactor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for refactoriung spark connector

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add OPC UA Transformer parameters (#301)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update for Micromamba Github Actions Parameters (#302)

* Update micromamba parameters

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update for Sonarqube

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade FastAPI package (#303)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: PaveeG <pawegor9@gmail.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: PaveeG <pawegor9@gmail.com>

* Rebase main (#308)

* remove print statement in twa

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* v0.2.0 (#121)

* Add Releases Feed to Docs Release page (#69)

* fixes Add Releases Feed to Docs page #68

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Mkdocs test fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add unit tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Code Smell

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Packages and Doc Logos (#76)

* Upgrade Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logos

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logo to pypi readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove pydantic models

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove strawberry type

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo Size Changes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Logo Alignment (#78)

* Upgrade Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logos

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add logo to pypi readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove pydantic models

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove strawberry type

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove GraphQL

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo Size Changes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Logo fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* fixes #77 (#79)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* add spark session (#80)

* set up spark sessions

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* adding unit tests for spark session

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove tests for spark session

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix code smells

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add constants in pipeline utils

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Add Pipelines as Extras libraries (#82)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Spark pipeline source Event hub and destination Event hub (#83)

* add source eventhub to pipelines

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* updates for documentation

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* updates to mkdocs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* added unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* documentation for eventhub source

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix spelling error

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add source eventhub destination

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* added destination source documentation

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove unit tests to revisit later

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix mkdoc errors

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix code smells

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Spark pipeline source delta and destination delta (#84)

* Spark pipeline source delta and destination delta

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* fix mkdocs errors

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* docstring amendments

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Added to docstrings on delta source & destination

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* added query type for eventhubs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>

* Added delta sharing with docs (#85)

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Unit Tests for sources (#88)

* add pipeline utils unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for eventhub and delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Unit Test Fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* remove test spark

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for destination delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix eventhub spell errors

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add pipeline eventhub body transformer (#89)

* add transform eventhub

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* transformer docs set up

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* documentation for transformer eventhub body

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add transformer eventhub unit tests

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Added pipeline source autoloader and unit test (#93)

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Pipeline Executor (#95)

* Dependency Injector

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Executor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Utilities Interface

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Executor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Pipeline Execute Test

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* fixes #90

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fixes for tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Step Order fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Sonarqube fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Delta Table Create Utility (#97)

* Add to docs site

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Delta Table Name

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Delta Table Create

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Delta Table Create Test

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* remove print statement in twa (#99)

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>

* Pipeline Deployment (#102)

* Add DBX Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Deployment Updates (#103)

* Pipeline Deployment Updates

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to run logic of Pipeline

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment (#104)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Unit test for spark pipeline eventhub destination (#105)

* eventhub destination unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for spark eventhub destination

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add spark eventhub write stream unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* update exception handling for eventhub and delta

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* update exception in utilities

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Fix for Dependency Injection (#106)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Unit tests for source and destination streams (#108)

* streaming unit test for source eventhub

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add delta read stream unit test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add unit test for delta destination streaming

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove extra test

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* remove code smell

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* add previous line of code back

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Pipeline Deployments and Pipeline Secrets (#111)

* Fix for Dependency Injection

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Deployment And Secrets

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add System Type Docs

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Secrets

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smell fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Refactor Pipeline Secret Conversion Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* import required objects

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smell Fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Code Smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Added fails tests to pipeline sources (#112)

* Added fails tests to pipeline sources

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Added fails tests for pipeline destinations

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Fixed code smell

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

---------

Signed-off-by: JamesKnBr <James.Broady@Shell.com>

* Fix for dockerbuild error (#114)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix for docker build (#115)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Cleanup Dockerfile (#116)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pipeline Secrets, Converters and Deployment Updates (#119)

* Updates for Pipelines

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update DBX Documentation

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove package.json

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix code smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* time weighted average bug fix (#120)

* fix twa group by dataframe return

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* bug fixes for groupby and apply method

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>

* v0.4.0 (#304)

* EdgeX Transformer (#271)

* added transformer component

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added edgeX test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* update test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* test updates

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added kafka destination write format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* component update

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* removed timestamp format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* updated docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Feature/00264 (#272)

* EdgeX Transformer (#271)

* added transformer component

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added edgeX test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* update test

Signed-off-by: JamesKnBr <james.broady@shell.com>

* test updates

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* added kafka destination write format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* component update

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>

* removed timestamp format

Signed-off-by: JamesKnBr <james.broady@shell.com>

* updated docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Adding support for MISO Sources

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing sonar smells

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing sonar smells

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Removing ISO Runner file

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Making ISO methods internal

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>

* Add Partition Pruning to non broadcast join PCDM Merge (#274)

* Add Partition Pruning on non broadcast join merge

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update DropDuplicates

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix Drop Duplicates Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* update docs (#286)

* update docs

Signed-off-by: JamesKnBr <james.broady@shell.com>

* linked issues page

Signed-off-by: JamesKnBr <james.broady@shell.com>

* changed issues link

Signed-off-by: JamesKnBr <james.broady@shell.com>

---------

Signed-off-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: JamesKnBr <james.broady@shell.com>

* Fix mkdocs colour bug (#291)

* update

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix colour issue on mkdocs

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* fix colour bug on homepage

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>

* Add Filter Parameter and TagName field mapping for OPC Publisher Transformer (#296)

* Additional OPC Publisher Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Device ID Filter

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Docs

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to Filter Logic

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Weather Domain and Data Model (#294)

* added new overview and data model

Signed-off-by: PaveeG <pawegor9@gmail.com>

* adding signoff

Signed-off-by: PaveeG <pawegor9@gmail.com>

* added overview and data model

Signed-off-by: PaveeG <pawegor9@gmail.com>

* Adding signoff Signed-off-by: Author Name <pawegor9@gmail.com>

Signed-off-by: PaveeG <pawegor9@gmail.com>

---------

Signed-off-by: PaveeG <pawegor9@gmail.com>

* Refactor for odbc and functions folders (#299)

* Folder Refactor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Pyspark 3.4.0 and delta-spark 2.4.0 support (#297)

* Preview Support of Pyspark 3.4.0

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fix for running github actions tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for delta-spark release candidate

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Downgrade Turbodbc to include 3.8

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add Spark Connect

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates to packages

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for release of 2.4.0

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add hvac back

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Fixes for tests

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove Whitespace

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Build sdist and wheel

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Change micromamba Action

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Tests Workflow

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code Smells

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Sonarqube Bug Fix

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update Readme with logo

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Center table

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update readme

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Folder Refactor

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Code smell fixes

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates for refactoriung spark connector

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add OPC UA Transformer parameters (#301)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update for Micromamba Github Actions Parameters (#302)

* Update micromamba parameters

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update for Sonarqube

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Upgrade FastAPI package (#303)

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: PaveeG <pawegor9@gmail.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <james.broady@shell.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: PaveeG <pawegor9@gmail.com>

* Resolve imports on deprecated folders

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Updates to remove unwanted files

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Signed-off-by: PaveeG <pawegor9@gmail.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Co-authored-by: IW-SS <133041871+IW-SS@users.noreply.github.com>
Co-authored-by: PaveeG <pawegor9@gmail.com>

* Merge main to develop

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove Eventhub Docs (#316)

* Remove doc

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to environment.yml

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Update to environment file

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Add copyright to Environment.yml

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Remove comments on Environment.yml

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Change environment.yml back now that bug is fixed

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

---------

Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>

* Adding DF show for MisoToMDM Testcases

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* MisoToMDM Test cases: Adding show for all DFs

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Removing timezone from dates for MISOToMDM Transformer class

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Removing show statements from MisoToMDM Transformer test cases

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Converting Column names into CamelCasing for ISOs

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing spark.py and meters utils testcases

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

* Fixing Small code smell

Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>

---------

Signed-off-by: cching95 <73163191+cching95@users.noreply.github.com>
Signed-off-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Signed-off-by: rodalynbarce <Rodalyn.Barce@shell.com>
Signed-off-by: JamesKnBr <James.Broady@Shell.com>
Signed-off-by: Shivam Saxena <shivam.saxena@innowatts.com>
Signed-off-by: JamesKnBr <james.broady@shell.com>
Signed-off-by: PaveeG <pawegor9@gmail.com>
Co-authored-by: GBBBAS <42962356+GBBBAS@users.noreply.github.com>
Co-authored-by: Amber-Rigg <44523299+Amber-Rigg@users.noreply.github.com>
Co-authored-by: NooraKubati <55835429+NooraKubati@users.noreply.github.com>
Co-authored-by: cching95 <73163191+cching95@users.noreply.github.com>
Co-authored-by: rodalynbarce <121169437+rodalynbarce@users.noreply.github.com>
Co-authored-by: JamesKnBr <James.Broady@Shell.com>
Co-authored-by: JamesKnBr <125024404+JamesKnBr@users.noreply.github.com>
Co-authored-by: rodalynbarce <Rodalyn.Barce@shell.com>
Co-authored-by: PaveeG <pawegor9@gmail.com>
Loading branch information
10 people committed Jul 12, 2023
1 parent f44a0f9 commit 23d2582
tests/sdk/python/rtdip_sdk/pipelines/transformers/spark/iso/test_miso_to_mdm.py
-Original file line number
+Diff line change
@@ Expand Up / @@ -22,6 +22,7 @@ @@
     from src.sdk.python.rtdip_sdk.pipelines._pipeline_utils.iso import MISO_SCHEMA
     from src.sdk.python.rtdip_sdk.pipelines.transformers import MISOToMDMTransformer
     from src.sdk.python.rtdip_sdk.pipelines._pipeline_utils.models import Libraries, SystemType
+    from tests.sdk.python.rtdip_sdk.pipelines._pipeline_utils.spark_configuration_constants import spark_session
     from pyspark.sql import SparkSession, DataFrame
@@ Expand Down @@
0 comments on commit `23d2582`

Please sign in to comment.
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Commit

There are no files selected for viewing

0 comments on commit `23d2582`

Commit

There are no files selected for viewing

0 comments on commit 23d2582

0 comments on commit `23d2582`
