Release version 4.0

.gitattributes

@@ -3,3 +3,4 @@
 src/test/resources/edu/ie3/datamodel/io/source/influxdb/_weather/cosmo/weather.txt eol=lf
 src/test/resources/edu/ie3/datamodel/io/source/influxdb/_weather/icon/weather.txt eol=lf
 
+gradlew eol=lf

.github/dependabot.yml

@@ -13,10 +13,6 @@ updates:
   - sebastian-peter
   - danielfeismann
   - jo-bao
-  ignore:
-  - dependency-name: org.spockframework:spock-core
-    versions:
-        - 2.3-groovy-4.0
 
 - package-ecosystem: pip
   directory: "/docs/readthedocs"

.readthedocs.yaml

@@ -7,9 +7,9 @@ version: 2
 
 # Set the version of Python and other tools you might need
 build:
-  os: ubuntu-20.04
+  os: ubuntu-22.04
   tools:
-    python: "3.9"
+    python: "3.11"
 
 # Configure python
 python:
@@ -18,4 +18,5 @@ python:
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
-  configuration: docs/readthedocs/conf.py
+  configuration: docs/readthedocs/conf.py
+  fail_on_warning: true

CHANGELOG.md

@@ -6,6 +6,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased/Snapshot]
 
+## [4.0.0] - 2023-08-01
+
+### Added
+- Copy methods for container classes [#726](https://github.com/ie3-institute/PowerSystemDataModel/issues/726)
+- Allow hierarchic grid structure for JointGridContainer [#768](https://github.com/ie3-institute/PowerSystemDataModel/issues/768)
+- Adding SQL id coordinate sources (``IdCoordinateSource``) [#689](https://github.com/ie3-institute/PowerSystemDataModel/issues/689)
+- Added some standard asset types to documentation [#642](https://github.com/ie3-institute/PowerSystemDataModel/issues/642)
+
+### Fixed
+- Fixed wrong rated power unit hint [#804](https://github.com/ie3-institute/PowerSystemDataModel/issues/804)
+- Fixed wrong hash code generation of ConnectorResult [#817](https://github.com/ie3-institute/PowerSystemDataModel/issues/817) 
+
+### Changed
+- Removing deprecated classes and methods [#540](https://github.com/ie3-institute/PowerSystemDataModel/issues/540)
+- Refactor CSV data sources [#716](https://github.com/ie3-institute/PowerSystemDataModel/issues/716)
+- Deleted parameter initFiles, set parameter append to false by default [#791](https://github.com/ie3-institute/PowerSystemDataModel/issues/791)
+- Use nio paths instead of strings for file path [#723](https://github.com/ie3-institute/PowerSystemDataModel/issues/723)
+- Data source will throw an exceptions instead of returning an empty optionals [#707](https://github.com/ie3-institute/PowerSystemDataModel/issues/707)
+- Improving `ValidationUtils` [#758](https://github.com/ie3-institute/PowerSystemDataModel/issues/758)
+
+
 ## [3.0.0] - 2023-02-16
 
 ### Added
@@ -25,26 +46,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 - Reduced code smells [#492](https://github.com/ie3-institute/PowerSystemDataModel/issues/492)
-    - Protected constructors for abstract classes
-    - Use pattern matching
-    - Remove unused imports
-    - Use enhanced switch statements
-    - Replace lambdas with method references
-    - Use `Stream#toList`
-    - Adapt visibility for JUnit 5
+  - Protected constructors for abstract classes
+  - Use pattern matching
+  - Remove unused imports
+  - Use enhanced switch statements
+  - Replace lambdas with method references
+  - Use `Stream#toList`
+  - Adapt visibility for JUnit 5
 - More code smell fixing [#633](https://github.com/ie3-institute/PowerSystemDataModel/issues/633)
-    - Use `List#of`
-    - Use direct assignment with switch/case structures
-    - Turn some classes into records
-    - Making abstract classes' constructor protected
-    - Improving some RegExs
-    - Replacing `filter(Optional::isPresent).map(Optional::get)` on streams with `flatMap(Optional::stream)`
-    - instanceof variable declarations
-    - Removing unnecessary parentheses
-    - Miscellaneous code smells
+  - Use `List#of`
+  - Use direct assignment with switch/case structures
+  - Turn some classes into records
+  - Making abstract classes' constructor protected
+  - Improving some RegExs
+  - Replacing `filter(Optional::isPresent).map(Optional::get)` on streams with `flatMap(Optional::stream)`
+  - instanceof variable declarations
+  - Removing unnecessary parentheses
+  - Miscellaneous code smells
 - Fix JavaDoc creation
-    - Create JavaDoc with java 17 instead of java 8
-    - Let JavDoc pass, if there are warnings **ATTENTION:** Should be removed, when JavaDoc is fixed! (cf. Issue [#494](https://github.com/ie3-institute/PowerSystemDataModel/issues/494))
+  - Create JavaDoc with java 17 instead of java 8
+  - Let JavDoc pass, if there are warnings **ATTENTION:** Should be removed, when JavaDoc is fixed! (cf. Issue [#494](https://github.com/ie3-institute/PowerSystemDataModel/issues/494))
 - `BufferedCsvWriter` writes columns in the order, that the headline elements are defined [#434](https://github.com/ie3-institute/PowerSystemDataModel/issues/393)
 - Cleaned up `IndividualTimeSeriesMetaInformation`-related methods in `CsvFileConnector` [#544](https://github.com/ie3-institute/PowerSystemDataModel/issues/544)
 - Fixed spotlessApply handling for `.groovy` files [#637](https://github.com/ie3-institute/PowerSystemDataModel/issues/637)
@@ -65,18 +86,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - `edu.ie3.datamodel.io.connectors.CsvFileConnector.CsvIndividualTimeSeriesMetaInformation`
   - and related methods
 - BREAKING: Comprehensive harmonization around weather sources [#267](https://github.com/ie3-institute/PowerSystemDataModel/issues/267)
-    - Adapted the expected column scheme
-      - General weather model
-        - `coordinate` to `coordinateid`
-      - DWD COSMO model
-        - `diffuseirradiation` to `diffuseirradiance`
-        - `directirradiation` to `directirradiance`
-      - ICON model:
-        - `"datum"` to `"time"`
-    - Force user to provide time stamp pattern to `CouchbaseWeatherSource` to ensure harmonized querying
+  - Adapted the expected column scheme
+    - General weather model
+      - `coordinate` to `coordinateid`
+    - DWD COSMO model
+      - `diffuseirradiation` to `diffuseirradiance`
+      - `directirradiation` to `directirradiance`
+    - ICON model:
+      - `"datum"` to `"time"`
+  - Force user to provide time stamp pattern to `CouchbaseWeatherSource` to ensure harmonized querying
 - BREAKING: Updating PowerSystemUtils dependency to 2.0-SNAPSHOT [#595](https://github.com/ie3-institute/PowerSystemDataModel/issues/595)
 - BREAKING: Generified the `LoadInput` attribute `standardLoadProfile` to `loadProfile` as it should also address the newly added `TemperatureDependantLoadProfile`s [#601](https://github.com/ie3-institute/PowerSystemDataModel/issues/601)
 - Adapted to new double converters in PSU [#705](https://github.com/ie3-institute/PowerSystemDataModel/issues/705)
+- Setting fixed groovy version and updating groovy [#788](https://github.com/ie3-institute/PowerSystemDataModel/issues/788)
 
 ## [2.1.0] - 2022-01-05
 
@@ -92,6 +114,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Writers used to write time series are closed right away
 - Changed class name in FlexOptionsResult.toString [#693](https://github.com/ie3-institute/PowerSystemDataModel/issues/693)
 - Deleted parameter decimalPlaces and changed naming of serialization method [#710](https://github.com/ie3-institute/PowerSystemDataModel/issues/710)
+- Changed switch result documentation according to the implementation [#757](https://github.com/ie3-institute/PowerSystemDataModel/issues/757)
+- Added documentation for EmResult and FlexOptionResult [#656](https://github.com/ie3-institute/PowerSystemDataModel/issues/656)
 
 ## [2.0.1] - 2021-07-08
 
@@ -195,7 +219,8 @@ coordinates or multiple exactly equal coordinates possible
 -   CsvDataSource now stops trying to get an operator for empty operator uuid field in entities
 -   CsvDataSource now parsing multiple geoJson strings correctly
 
-[Unreleased/Snapshot]: https://github.com/ie3-institute/powersystemdatamodel/compare/3.0.0...HEAD
+[Unreleased/Snapshot]: https://github.com/ie3-institute/powersystemdatamodel/compare/4.0.0...HEAD
+[4.0.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/3.0.0...4.0.0
 [3.0.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/2.1.0...3.0.0
 [2.1.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/2.0.1...2.1.0
 [2.0.1]: https://github.com/ie3-institute/powersystemdatamodel/compare/2.0.0...2.0.1

build.gradle

@@ -4,19 +4,21 @@ plugins {
   id 'maven-publish'
   id 'signing'
   id 'pmd' // code check, working on source code
-  id 'com.diffplug.spotless' version '6.15.0'//code format
-  id 'com.github.spotbugs' version '5.0.13' // code check, working on byte code
-  id 'de.undercouch.download' version '5.3.1'
+  id 'com.diffplug.spotless' version '6.20.0'//code format
+  id 'com.github.spotbugs' version '5.1.0' // code check, working on byte code
+  id 'de.undercouch.download' version '5.4.0'
   id 'kr.motd.sphinx' version '2.10.1' // documentation generation
   id 'jacoco' // java code coverage plugin
-  id "org.sonarqube" version "3.5.0.2730" // sonarqube
+  id "org.sonarqube" version "4.3.0.3225" // sonarqube
   id 'net.thauvin.erik.gradle.semver' version '1.0.4' // semantic versioning
 }
 
 ext {
   //version (changing these should be considered thoroughly!)
   javaVersion = JavaVersion.VERSION_17
-  testcontainersVersion = '1.17.6'
+  groovyVersion = "4.0"
+  groovyBinaryVersion = "4.0.13"
+  testcontainersVersion = '1.18.3'
 
   scriptsLocation = 'gradle' + File.separator + 'scripts' + File.separator //location of script plugins
 }
@@ -43,15 +45,14 @@ repositories {
 
   // sonatype snapshot repo
   maven { url 'https://s01.oss.sonatype.org/content/repositories/snapshots' }
-
 }
 
 dependencies {
 
   //	 ie³ power system utils
   implementation 'com.github.ie3-institute:PowerSystemUtils:2.0'
 
-  implementation 'tech.units:indriya:2.1.4'
+  implementation 'tech.units:indriya:2.2'
 
   // JTS Topology Suite for GeoPositions, License: EPL 1.0 / EDL 1.0
   implementation ('org.locationtech.jts:jts-core:1.19.0'){
@@ -61,13 +62,15 @@ dependencies {
   implementation 'org.locationtech.jts.io:jts-io-common:1.19.0'
 
   // Graphs
-  implementation 'org.jgrapht:jgrapht-core:1.5.1'
+  implementation 'org.jgrapht:jgrapht-core:1.5.2'
 
   // testing
-  testImplementation 'org.junit.jupiter:junit-jupiter:5.9.2'
-  testImplementation 'org.spockframework:spock-core:2.3-groovy-3.0'
+  testImplementation "org.apache.groovy:groovy:$groovyBinaryVersion"
+
+  testImplementation 'org.junit.jupiter:junit-jupiter:5.10.0'
+  testImplementation "org.spockframework:spock-core:2.3-groovy-$groovyVersion"
   testImplementation 'org.objenesis:objenesis:3.3' // Mock creation with constructor parameters
-  testImplementation 'net.bytebuddy:byte-buddy:1.13.0' // Mocks of classes
+  testImplementation 'net.bytebuddy:byte-buddy:1.14.5' // Mocks of classes
 
   // testcontainers (docker framework for testing)
   testImplementation "org.testcontainers:testcontainers:$testcontainersVersion"
@@ -77,20 +80,19 @@ dependencies {
   testImplementation "org.testcontainers:couchbase:$testcontainersVersion"
 
   // logging
-  implementation platform('org.apache.logging.log4j:log4j-bom:2.19.0')
+  implementation platform('org.apache.logging.log4j:log4j-bom:2.20.0')
   implementation 'org.apache.logging.log4j:log4j-api' // log4j
   implementation 'org.apache.logging.log4j:log4j-core' // log4j
   implementation 'org.apache.logging.log4j:log4j-slf4j-impl' // log4j -> slf4j
 
   // Databases
   implementation 'org.influxdb:influxdb-java:2.23'
-  implementation 'com.couchbase.client:java-client:3.4.3'
-  runtimeOnly 'org.postgresql:postgresql:42.5.3' // postgresql jdbc driver required during runtime
-
-  implementation 'commons-io:commons-io:2.11.0' // I/O functionalities
-  implementation 'org.apache.commons:commons-compress:1.22' // I/O functionalities
-  implementation 'org.apache.commons:commons-lang3:3.12.0'
+  implementation 'com.couchbase.client:java-client:3.4.8'
+  runtimeOnly 'org.postgresql:postgresql:42.6.0' // postgresql jdbc driver required during runtime
 
+  implementation 'commons-io:commons-io:2.13.0' // I/O functionalities
+  implementation 'org.apache.commons:commons-compress:1.23.0' // I/O functionalities
+  implementation 'org.apache.commons:commons-lang3:3.13.0'
 }
 
 tasks.withType(JavaCompile) {

docs/readthedocs/conf.py

@@ -18,12 +18,12 @@
 # -- Project information -----------------------------------------------------
 
 project = 'PowerSystemDataModel'
-copyright = u'2020. TU Dortmund University, Institute of Energy Systems, Energy Efficiency and Energy Economics, Research group Distribution grid planning and operation '
-author = 'Johannes Hiry, Debopama Sen Sarma, Chris Kittl'
+copyright = u'2023. TU Dortmund University, Institute of Energy Systems, Energy Efficiency and Energy Economics, Research group Distribution grid planning and operation '
+author = 'Institute of Energy Systems, Energy Efficiency and Energy Economics'
 
 # The full version, including alpha/beta/rc tags
-version = '1.0'
-release = '1.0.1-SNAPSHOT'
+version = '3.0'
+release = '3.0.0'
 
 pygments_style = 'tango'
 add_function_parentheses = True
@@ -35,12 +35,12 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'recommonmark',
-    'sphinx.ext.autosectionlabel'
+    'sphinx.ext.intersphinx',
+    'myst_parser'
 ]
 
-# Prefix all autogenerated labels wit the document to get unique labels (e.g. `index:Hello`)
-autosectionlabel_prefix_document = True
+myst_enable_extensions = ["dollarmath", "amsmath"]
+myst_heading_anchors = 4
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -49,7 +49,8 @@
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'venv']
-
+exclude_trees = ['.build']
+source_suffix = ['.rst', '.md']
 source_encoding = 'utf-8-sig'
 
 # -- Options for HTML output -------------------------------------------------

docs/readthedocs/models/ValidationUtils.rst → docs/readthedocs/io/ValidationUtils.md

@@ -1,32 +1,26 @@
-****************
-Validation Utils
-****************
+# Validation Utils
 This page gives an overview about the ValidationUtils in the *PowerSystemDataModel*.
 
-What are the ValidationUtils?
-=============================
+## What are the ValidationUtils?
 The methods in ValidationUtils and subclasses can be used to check that objects are valid, meaning their parameters have valid values and they are correctly connected.
 
-What is checked?
-================
+## What is checked?
 - The check methods include checks that assigned values are valid, e.g. lines are not allowed to have negative lengths or the rated power factor of any unit must be between 0 and 1.
 - Furthermore, several connections are checked, e.g. that lines only connect nodes of the same voltage level or that the voltage levels indicated for the transformer sides match the voltage levels of the nodes they are connected to.
 
-How does it work?
-=================
-- The method :code:`ValidationUtils.check(Object)` is the only method that should be called by the user.
+## How does it work?
+- The method `ValidationUtils.check(Object)` is the only method that should be called by the user.
 - This check method identifies the object class and forwards it to a specific check method for the given object
 - The overall structure of the ValidationUtils methods follows a cascading scheme, orientated along the class tree
-- Example: A :code:`LineInput lineInput` should be checked
-    1. :code:`ValidationUtils.check(lineInput)` is called
-    2. :code:`ValidationUtils.check(lineInput)` identifies the class of the object as :code:`AssetInput` and calls :code:`ValidationUtils.checkAsset(lineInput)`
-    3. :code:`ValidationUtils.checkAsset(lineInput)`, if applicable, checks those parameters that all :code:`AssetInput` have in common (e.g. operation time) and further identifies the object, more specifically, as a :code:`ConnectorInput` and calls :code:`ConnectorValidationUtils.check(lineInput)`
-    4. :code:`ConnectorValidationUtils.check(lineInput)`, if applicable, checks those parameters that all :code:`ConnectorInput` have in common and further identifies the object, more specifically, as a :code:`LineInput` and calls :code:`ConnectorValidationUtils.checkLine(lineInput)`
-    5. :code:`ConnectorValidationUtils.checkLine(lineInput)` checks all specific parameters of a :code:`LineInput`
+- Example: A `LineInput lineInput` should be checked
+    1. `ValidationUtils.check(lineInput)` is called
+    2. `ValidationUtils.check(lineInput)` identifies the class of the object as `AssetInput` and calls `ValidationUtils.checkAsset(lineInput)`
+    3. `ValidationUtils.checkAsset(lineInput)`, if applicable, checks those parameters that all `AssetInput` have in common (e.g. operation time) and further identifies the object, more specifically, as a `ConnectorInput` and calls `ConnectorValidationUtils.check(lineInput)`
+    4. `ConnectorValidationUtils.check(lineInput)`, if applicable, checks those parameters that all `ConnectorInput` have in common and further identifies the object, more specifically, as a `LineInput` and calls `ConnectorValidationUtils.checkLine(lineInput)`
+    5. `ConnectorValidationUtils.checkLine(lineInput)` checks all specific parameters of a `LineInput`
 - ValidationUtils furthermore contains several utils methods used in the subclasses
 
-Which objects are checked?
-==========================
+## Which objects are checked?
 The ValidationUtils include validation checks for...
 
 - NodeValidationUtils
@@ -80,8 +74,7 @@ The ValidationUtils include validation checks for...
     - RawGridElements
     - SystemParticipants
 
-What should be considered?
-==========================
+## What should be considered?
 - Due to many checks with if-conditions, the usage of the ValidationUtils for many objects might be runtime relevant.
 - The check for a GridContainer includes the interplay of the contained entities as well as the checks of all contained entities.
 - If new classes are introduced to the *PowerSystemDataModel*, make sure to follow the forwarding structure of the ValidationUtils methods when writing the check methods!

docs/readthedocs/io/basiciousage.md

@@ -0,0 +1,40 @@
+# I/O
+
+The PowerSystemDataModel library additionally offers I/O-capabilities.
+In the long run, it is our aim to provide many different source and sink technologies.
+Therefore, the I/O-package is structured as highly modular.
+
+```{toctree}
+---
+maxdepth: 2
+---
+csvfiles
+sql
+influxdb
+ValidationUtils.md
+```
+
+## Data sink structure
+
+[![Class diagram of data sink classes](../_static/figures/uml/DataSinkClassDiagram.png)](../_static/figures/uml/DataSinkClassDiagram.png)
+
+## Data source structure
+
+The sources are divided in three blocks:
+1. InputEntities and ResultEntities
+2. TimeSeries related sources
+3. Weather and Coordinate sources
+
+[![Class diagram of entity sources](../_static/figures/uml/EntitySourceClassDiagram.png)](../_static/figures/uml/EntitySourceClassDiagram.png)
+
+[![Class diagram of time series sources](../_static/figures/uml/TimeSeriesSourceClassDiagram.png)](../_static/figures/uml/TimeSeriesSourceClassDiagram.png)
+
+[![Class diagram of weather and coordinate sources](../_static/figures/uml/WeatherCoordinateSourceClassDiagram.png)](../_static/figures/uml/WeatherCoordinateSourceClassDiagram.png)
+
+The function to read the sources are implemented in the DataSource classes.
+
+[![Class diagram of data sources](../_static/figures/uml/FunctionalDataSourceClassDiagram.png)](../_static/figures/uml/FunctionalDataSourceClassDiagram.png)
+
+## Data deployment
+
+[![Diagram of input data deployment](../_static/figures/uml/InputDataDeployment.png)](../_static/figures/uml/InputDataDeployment.png)