Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9c38ee1
#246 Deprecated FileNamingStrategy, created EntityPersistenceNamingSt…
avgprog Feb 22, 2021
9d668ab
#279 Duplicated test cases of FileNamingStrategy to test EntityPersis…
avgprog Feb 22, 2021
5c81759
minor, task-unrelated changes and code improvements
johanneshiry Mar 5, 2021
731345c
minor change + transform sample
johanneshiry Mar 5, 2021
a303273
#279 Replaced occurences of FileNamingStrategy with EntityPersistence…
avgprog Mar 8, 2021
57e7925
Merge branch 'dev' into sb/#246-replacingFileNamingStrategy
avgprog Mar 9, 2021
214aab4
#279 Replaced FileNamingStrategy with EntityPersistenceNamingStrategy…
avgprog Mar 9, 2021
95c28c5
Merge branch 'dev' into sb/#246-replacingFileNamingStrategy
avgprog Mar 10, 2021
ce6ced5
#279 Modified javadocs and uml documentation to reflect replacement o…
avgprog Mar 10, 2021
5cc8423
Merge branch 'sb/#246-replacingFileNamingStrategy' of https://github.…
avgprog Mar 10, 2021
bbb8b5b
#279 Fixed Codacy Analysis Issues and renamed some strings to account…
avgprog Mar 10, 2021
61f91fc
Merge branch 'dev' into sb/#246-replacingFileNamingStrategy
avgprog Mar 10, 2021
82bb2ae
Merge branch 'dev' into sb/#246-replacingFileNamingStrategy
ckittl Mar 11, 2021
6488979
Merge branch 'dev' into sb/#246-replacingFileNamingStrategy
ckittl Mar 12, 2021
993227d
#279 Codacy issue handling by reducing NPath Complexity of getFileNam…
avgprog Mar 12, 2021
571defa
#279 Codacy issue handling by splitting test cases
avgprog Mar 12, 2021
974fd1d
#279 Removed redundant function calls and split test cases
avgprog Mar 22, 2021
bc53c80
#279 Split test cases to reduce the maximum number of methods in Enti…
avgprog Mar 22, 2021
669024b
#279 Removed unnecessary imports
avgprog Mar 22, 2021
94a878e
#279 Renamed the test case files
avgprog Mar 22, 2021
4f60799
#279 Deleted FileNamingStrategy and FileNamingStrategyTest
avgprog Mar 22, 2021
cb09f8f
Revert "#279 Deleted FileNamingStrategy and FileNamingStrategyTest"
avgprog Mar 29, 2021
756f88a
#279 Merged the test case files into a single file
avgprog Mar 30, 2021
8fb60a7
#279 Merged the test case files into a single file
avgprog Mar 30, 2021
7fc9c68
#279 Updated documentation and comments to reflect the change from Fi…
avgprog Mar 30, 2021
a9b5dbb
#279 Changed several function names to indicate that entity name is b…
avgprog Mar 30, 2021
9cf9e05
#279 Changed several function names to indicate that entity name is b…
avgprog Mar 30, 2021
eab5b97
Merge remote-tracking branch 'origin/dev' into sb/#246-replacingFileN…
ckittl Mar 31, 2021
a884e13
Merge remote-tracking branch 'origin/dev' into sb/#246-replacingFileN…
ckittl Mar 31, 2021
63d6cf0
Renaming methods
ckittl Mar 31, 2021
80c9d4d
Adapting documentation
ckittl Mar 31, 2021
a33889c
Removing the last not working tests
ckittl Mar 31, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions docs/readthedocs/io/csvfiles.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,25 @@ csv files

Naming of files
===============
A naming strategy provides a mapping between model classes and the file names, in which the serialized representation of
several objects of this class can be found.
Currently we offer two different, pre-defined file naming strategies, which you might extend to fit your needs:
A naming strategy provides a mapping between model classes and the human readable names of those entities to be used
within e.g. the data sinks, in which the serialized representation of several objects of this class can be found.
Currently we offer two different, pre-defined naming strategies, which you might extend to fit your needs:

1. **FileNamingStrategy**:
A basic file naming strategy that is able to add prefix and suffix to the file names. A flat folder structure is
considered. For more details see `Default naming strategy`_.
1. **EntityPersistenceNamingStrategy**:
A basic naming strategy that is able to add prefix and suffix to the names of the entities. A flat folder structure
is considered. For more details see `Default naming strategy`_.
2. **HierarchicFileNamingStrategy**:
An extended version of the FileNamingStrategy. Additionally, the `Default directory hierarchy`_ is taken into
account. Please note, that this directory hierarchy is only meant to be used in conjunction with input models.
An extended version of the EntityPersistenceNamingStrategy. Additionally, the `Default directory hierarchy`_ is taken
into account. Please note, that this directory hierarchy is only meant to be used in conjunction with input models.

However, you can control the behaviour of serialization and de-serialization of models by injecting the desired naming
strategy you like into :code:`CsvDataSource` and :code:`CsvFileSink`.

Default naming strategy
=======================
There is a default mapping from model class to file naming in the case you would like to use csv files for
There is a default mapping from model class to naming of entities in the case you would like to use csv files for
(de-)serialization of models.
You may extend / alter the naming with pre- or suffix by calling :code:`new FileNamingStrategy("prefix", "suffix")`.
You may extend / alter the naming with pre- or suffix by calling :code:`new EntityPersistenceNamingStrategy("prefix","suffix")`.

Input
-----
Expand Down Expand Up @@ -224,7 +224,7 @@ An application example to load an *exampleGrid* from csv files located in :code:
String gridName = "exampleGrid";
String csvSep = ",";
String folderPath = "./exampleGrid";
FileNamingStrategy namingStrategy = new FileNamingStrategy(); // Default naming strategy
EntityPersistenceNamingStrategy namingStrategy = new EntityPersistenceNamingStrategy(); // Default naming strategy

/* Instantiating sources */
TypeSource typeSource = new CsvTypeSource(csvSep, folderPath, namingStrategy);
Expand Down Expand Up @@ -273,7 +273,7 @@ Serializing models is a bit easier:
/* Parameterization */
String csvSep = ",";
String folderPath = "./exampleGrid";
FileNamingStrategy namingStrategy = new FileNamingStrategy();
EntityPersistenceNamingStrategy namingStrategy = new EntityPersistenceNamingStrategy();
boolean initEmptyFiles = false;

/* Instantiating the sink */
Expand Down
8 changes: 4 additions & 4 deletions docs/uml/main/DataSinkClassDiagram.puml
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,18 @@ OutputDataSink --|> DataSink

' Implementations
class CsvFileSink {
+ CsvFileSink(String, ProcessorProvider, FileNamingStrategy, boolean, String)
+ CsvFileSink(String, FileNamingStrategy, boolean, String)
+ CsvFileSink(String, ProcessorProvider, EntityPersistenceNamingStrategy, boolean, String)
+ CsvFileSink(String, EntityPersistenceNamingStrategy, boolean, String)
+ CsvFileSink(String)
- void initFiles(ProcessorProvider, FileNamingStrategy)
- void initFiles(ProcessorProvider, EntityPersistenceNamingStrategy)
- void write(UniqueEntity)
}
CsvFileSink ..|> InputDataSink
CsvFileSink ..|> OutputDataSink


class InfluxDBFileSink {
+ InfluxDbSink(InfluxDbConnector, FileNamingStrategy)
+ InfluxDbSink(InfluxDbConnector, EntityPersistenceNamingStrategy)
+ InfluxDbSink(InfluxDbConnector)
- Set<Point> extractPoints(UniqueEntity)
- String transformToMeasurementName(String)
Expand Down
18 changes: 9 additions & 9 deletions docs/uml/main/DataSourceClassDiagram.puml
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ DataSource <|-- TimeSeriesSource

class CsvTimeSeriesSource<V extends Value> {
- IndividualTimeSeries<V> timeSeries
+ {static} CsvTimeSeriesSource<? extends Value> getSource(\n\tString,\n\tString,\n\tFileNamingStrategy,\n\tsvFileConnector.CsvIndividualTimeSeriesMetaInformation)
+ {static} CsvTimeSeriesSource<? extends Value> getSource(\n\tString,\n\tString,\n\tEntityPersistenceNamingStrategy,\n\tCsvFileConnector.CsvIndividualTimeSeriesMetaInformation)
- IndividualTimeSeries<V> buildIndividualTimeSeries(\n\tUUID,\n\tfilePath,\n\tFunction<Map<String, String>,\n\tOptional<TimeBasedValue<V>>>)
- Optional<TimeBasedValue<V>> buildTimeBasedValue(\n\tMap<String, String>,\n\tClass<V>,\n\tTimeBasedSimpleValueFactory<V>)
}
Expand All @@ -127,7 +127,7 @@ interface DataConnector {
Abstract Class CsvDataSource {
- String csvSep
# CsvFileConnector connector
+ CsvDataSource(String, String, FileNamingStrategy)
+ CsvDataSource(String, String, EntityPersistenceNamingStrategy)
}
DataSource <|.. CsvDataSource

Expand All @@ -137,7 +137,7 @@ Class CsvTypeSource {
- LineTypeInputFactory lineTypeInputFactory
- Transformer3WTypeInputFactory transformer3WTypeInputFactory
- SystemParticipantTypeInputFactory systemParticipantTypeInputFactory
+ CsvTypeSource(String, String, FileNamingStrategy)
+ CsvTypeSource(String, String, EntityPersistenceNamingStrategy)
}
TypeSource <|.. CsvTypeSource
CsvDataSource <|-- CsvTypeSource
Expand All @@ -147,7 +147,7 @@ Class CsvThermalSource {
- ThermalBusInputFactory thermalBusInputFactory
- CylindricalStorageInputFactory cylindricalStorageInputFactory
- ThermalHouseInputFactory thermalHouseInputFactory
+ CsvThermalSource(String, String, FileNamingStrategy, TypeSource)
+ CsvThermalSource(String, String, EntityPersistenceNamingStrategy, TypeSource)
}
ThermalSource <|.. CsvThermalSource
CsvDataSource <|-- CsvThermalSource
Expand All @@ -160,7 +160,7 @@ Class CsvRawGridSource {
- Transformer3WInputFactory transformer3WInputFactory
- SwitchInputFactory switchInputFactory
- MeasurementUnitInputFactory measurementUnitInputFactory
+ CsvRawGridSource(String, String, FileNamingStrategy, TypeSource)
+ CsvRawGridSource(String, String, EntityPersistenceNamingStrategy, TypeSource)
}
RawGridSource <|.. CsvRawGridSource
CsvDataSource <|-- CsvRawGridSource
Expand All @@ -179,7 +179,7 @@ Class CsvSystemParticipantSource {
- StorageInputFactory storageInputFactory
- WecInputFactory wecInputFactory
- EvcsInputFactory evcsInputFactory
+ CsvSystemParticipantSource(String, String, FileNamingStrategy, TypeSource, ThermalSource, RawGridSource)
+ CsvSystemParticipantSource(String, String, EntityPersistenceNamingStrategy, TypeSource, ThermalSource, RawGridSource)
}
SystemParticipantSource <|.. CsvSystemParticipantSource
CsvDataSource <|-- CsvSystemParticipantSource
Expand All @@ -189,18 +189,18 @@ Class CsvGraphicSource {
- RawGridSource rawGridSource
- LineGraphicInputFactory lineGraphicInputFactory
- NodeGraphicInputFactory nodeGraphicInputFactory
+ CsvGraphicSource(String, String, FileNamingStrategy, TypeSource, RawGridSource)
+ CsvGraphicSource(String, String, EntityPersistenceNamingStrategy, TypeSource, RawGridSource)
}
GraphicSource <|.. CsvGraphicSource
CsvDataSource <|-- CsvGraphicSource

Class CsvFileConnector {
- Map<Class<? extends UniqueEntity>, BufferedCsvWriter> entityWriters
- Map<UUID, BufferedCsvWriter> timeSeriesWriters
- FileNamingStrategy fileNamingStrategy
- EntityPersistenceNamingStrategy entityPersistenceNamingStrategy
- String baseDirectoryName
- {static} String FILE_ENDING
+ CsvFileConnector(String, FileNamingStrategy)
+ CsvFileConnector(String, EntityPersistenceNamingStrategy)
+ BufferedCsvWriter getOrInitWriter(Class<? extends UniqueEntity>, String[], String)
+ BufferedCsvWriter getOrInitWriter(T, String[], String)
+ BufferedCsvWriter initWriter(String, CsvFileDefinition)
Expand Down
34 changes: 19 additions & 15 deletions src/main/java/edu/ie3/datamodel/io/connectors/CsvFileConnector.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import edu.ie3.datamodel.io.csv.*;
import edu.ie3.datamodel.io.csv.timeseries.ColumnScheme;
import edu.ie3.datamodel.io.csv.timeseries.IndividualTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.naming.EntityPersistenceNamingStrategy;
import edu.ie3.datamodel.models.UniqueEntity;
import edu.ie3.datamodel.models.timeseries.TimeSeries;
import edu.ie3.datamodel.models.timeseries.TimeSeriesEntry;
Expand Down Expand Up @@ -40,17 +41,18 @@ public class CsvFileConnector implements DataConnector {
private final Map<UUID, BufferedCsvWriter> timeSeriesWriters = new HashMap<>();
// ATTENTION: Do not finalize. It's meant for lazy evaluation.
private Map<UUID, CsvIndividualTimeSeriesMetaInformation> individualTimeSeriesMetaInformation;
private final FileNamingStrategy fileNamingStrategy;
private final EntityPersistenceNamingStrategy entityPersistenceNamingStrategy;
private final String baseDirectoryName;

private static final String FILE_ENDING = ".csv";
private static final String FILE_SEPARATOR_REGEX = "[\\\\/]";
private static final String FILE_SEPARATOR_REPLACEMENT =
File.separator.equals("\\") ? "\\\\" : "/";

public CsvFileConnector(String baseDirectoryName, FileNamingStrategy fileNamingStrategy) {
public CsvFileConnector(
String baseDirectoryName, EntityPersistenceNamingStrategy entityPersistenceNamingStrategy) {
this.baseDirectoryName = baseDirectoryName;
this.fileNamingStrategy = fileNamingStrategy;
this.entityPersistenceNamingStrategy = entityPersistenceNamingStrategy;
}

public synchronized BufferedCsvWriter getOrInitWriter(
Expand Down Expand Up @@ -136,7 +138,8 @@ private BufferedCsvWriter initWriter(String baseDirectory, CsvFileDefinition fil

/**
* Initializes a file reader for the given class that should be read in. The expected file name is
* determined based on {@link FileNamingStrategy} of the this {@link CsvFileConnector} instance
* determined based on {@link EntityPersistenceNamingStrategy} of the this {@link
* CsvFileConnector} instance
*
* @param clz the class of the entity that should be read
* @return the reader that contains information about the file to be read in
Expand All @@ -146,7 +149,7 @@ public BufferedReader initReader(Class<? extends UniqueEntity> clz) throws FileN
String filePath = null;
try {
filePath =
fileNamingStrategy
entityPersistenceNamingStrategy
.getFilePath(clz)
.orElseThrow(
() ->
Expand Down Expand Up @@ -209,7 +212,8 @@ public Optional<IndividualTimeSeriesMetaInformation> getIndividualTimeSeriesMeta
String filePathWithoutEnding = removeFileEnding(filePath);
IndividualTimeSeriesMetaInformation metaInformation =
(IndividualTimeSeriesMetaInformation)
fileNamingStrategy.extractTimeSeriesMetaInformation(filePathWithoutEnding);
entityPersistenceNamingStrategy.extractTimeSeriesMetaInformation(
filePathWithoutEnding);
return new CsvIndividualTimeSeriesMetaInformation(
metaInformation, filePathWithoutEnding);
})
Expand Down Expand Up @@ -249,7 +253,7 @@ public Map<ColumnScheme, Set<TimeSeriesReadingData>> initTimeSeriesReader(
* @throws FileNotFoundException If the file is not present
*/
public BufferedReader initIdCoordinateReader() throws FileNotFoundException {
String filePath = fileNamingStrategy.getIdCoordinateFileName();
String filePath = entityPersistenceNamingStrategy.getIdCoordinateEntityName();
return initReader(filePath);
}

Expand All @@ -267,7 +271,7 @@ private Set<String> getIndividualTimeSeriesFilePaths() {
.filter(
path -> {
String withoutEnding = removeFileEnding(path.toString());
return fileNamingStrategy
return entityPersistenceNamingStrategy
.getIndividualTimeSeriesPattern()
.matcher(withoutEnding)
.matches();
Expand Down Expand Up @@ -297,7 +301,7 @@ private Optional<TimeSeriesReadingData> buildReadingData(
String filePathString, ColumnScheme... columnSchemes) {
try {
FileNameMetaInformation metaInformation =
fileNamingStrategy.extractTimeSeriesMetaInformation(filePathString);
entityPersistenceNamingStrategy.extractTimeSeriesMetaInformation(filePathString);
if (!IndividualTimeSeriesMetaInformation.class.isAssignableFrom(metaInformation.getClass())) {
log.error(
"The time series file '{}' does not represent an individual time series.",
Expand Down Expand Up @@ -359,10 +363,10 @@ private String removeFileEnding(String input) {
private <T extends TimeSeries<E, V>, E extends TimeSeriesEntry<V>, V extends Value>
CsvFileDefinition buildFileDefinition(T timeSeries, String[] headLineElements, String csvSep)
throws ConnectorException {
String directoryPath = fileNamingStrategy.getDirectoryPath(timeSeries).orElse("");
String directoryPath = entityPersistenceNamingStrategy.getDirectoryPath(timeSeries).orElse("");
String fileName =
fileNamingStrategy
.getFileName(timeSeries)
entityPersistenceNamingStrategy
.getEntityName(timeSeries)
.orElseThrow(
() ->
new ConnectorException(
Expand All @@ -382,10 +386,10 @@ CsvFileDefinition buildFileDefinition(T timeSeries, String[] headLineElements, S
private CsvFileDefinition buildFileDefinition(
Class<? extends UniqueEntity> clz, String[] headLineElements, String csvSep)
throws ConnectorException {
String directoryPath = fileNamingStrategy.getDirectoryPath(clz).orElse("");
String directoryPath = entityPersistenceNamingStrategy.getDirectoryPath(clz).orElse("");
String fileName =
fileNamingStrategy
.getFileName(clz)
entityPersistenceNamingStrategy
.getEntityName(clz)
.orElseThrow(
() ->
new ConnectorException(
Expand Down
13 changes: 13 additions & 0 deletions src/main/java/edu/ie3/datamodel/io/csv/FileNamingStrategy.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
import edu.ie3.datamodel.io.csv.timeseries.ColumnScheme;
import edu.ie3.datamodel.io.csv.timeseries.IndividualTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.csv.timeseries.LoadProfileTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.naming.EntityPersistenceNamingStrategy;
import edu.ie3.datamodel.io.naming.HierarchicFileNamingStrategy;
import edu.ie3.datamodel.io.source.TimeSeriesMappingSource;
import edu.ie3.datamodel.models.UniqueEntity;
import edu.ie3.datamodel.models.input.AssetInput;
Expand Down Expand Up @@ -39,7 +41,9 @@
*
* @version 0.1
* @since 03.02.20
* @deprecated replaced by {@link EntityPersistenceNamingStrategy}
*/
@Deprecated
public class FileNamingStrategy {

protected static final Logger logger = LogManager.getLogger(FileNamingStrategy.class);
Expand Down Expand Up @@ -118,6 +122,15 @@ public FileNamingStrategy(String prefix) {
this(prefix, "");
}

/**
* Create a {@link EntityPersistenceNamingStrategy} from a {@link FileNamingStrategy}
*
* @return an instance of {@link EntityPersistenceNamingStrategy}
*/
public EntityPersistenceNamingStrategy asEntityPersistenceNamingStrategy() {
return new EntityPersistenceNamingStrategy(this.prefix, this.suffix);
}

/**
* Prepares the prefix by appending an underscore and bringing it to lower case
*
Expand Down
Loading