Skip to content

v1.0.2
Compare
Choose a tag to compare
@rolnico rolnico released this 01 Aug 09:13
· 11 commits to release-v1.0.0 since this release
v1.0.2
dbf05c5

Introducing powsybl-metrix

Actions Status Coverage Status Quality Gate MPL-2.0 License Slack

This module is based on powsybl-core network modeling and framework.
It provides two main tools : MappingTool and MetrixTool
It can also be used as a library to integrate its functionalities
into java applications.

The MappingTool provide a mapping configuration from csv timeseries to network.
This configuration allows to generate iidm network at specified time. Furthermore,
it allows using the second tool, MetrixTool.

MetrixTool can compute power load flow and optimal power load flow for the multi variant
networks generated by the mapping tool. Its output are completed by synthesis.

MetrixTool calls metrix simulator as an external program. This program has its own requirements.

Install

Run ./install.sh

Usage

itools mapping and itools metrix are the command line tools that are provided to use and test the mapping and
metrix.
Their respective usage is detailed in the following sections.
For more advanced and intensive usage it is recommended to embed this package as a library as the time series storage is
not optimized.
Powsybl AFS can be use for that matter.

Mapping

To use the mapping, add com.powsybl:powsybl-metrix-mapping module to your dependencies.

Then you need :

  • a case file
  • a mapping groovy script configuration
  • a time series store
// Entry timeseries
InMemoryTimeSeriesStore store = new InMemoryTimeSeriesStore();
store.importTimeSeries(Collections.singletonList(Paths.get("/path/to/timeseries.csv")));

// Mapping file
Path mappingFile = Paths.get("path/to/mappingFile");

// Network
Network network = NetworkXml.read(Paths.get("/path/to/network.xiidm"));

// Computation parameters
MappingParameters mappingParameters = MappingParameters.load();
int firstVariant = ...
int maxVariantCount = ...
ComputationRange computationRange = new ComputationRange(store.getTimeSeriesDataVersions(), firstVariant, maxVariantCount);

// Generate mapping configuration
TimeSeriesMappingConfig config;
try (Reader reader = Files.newBufferedReader(mappingFile, StandardCharsets.UTF_8)) {
    TimeSeriesDslLoader dslLoader = new TimeSeriesDslLoader(reader, mappingFile.getFileName().toString());
    config = dslLoader.load(network, mappingParameters, store, new DataTableStore(), computationRange);
}

// Export result in csv
TimeSeriesMappingConfigCsvWriter csvWriter = new TimeSeriesMappingConfigCsvWriter(config, network, store, computationRange, mappingParameters.getWithTimeSeriesStats());
csvWriter.writeMappingCsv(Paths.get("/path/to/output"));

// Compute mapping on network
TimeSeriesMappingLogger logger = new TimeSeriesMappingLogger();
List<TimeSeriesMapperObserver> observers = new ArrayList<>();

// Add network generation computation
DataSource dataSource = DataSourceUtil.createDataSource(Paths.get("/path/to/networkOutputDir"), network.getId(), null);
observers.add(new NetworkPointWriter(network, dataSource));

// Add timeseries mapping export
TimeSeriesIndex index = new TimeSeriesMappingConfigTableLoader(config,store).checkIndexUnicity();
int lastPoint = Math.min(firstVariant + maxVariantCount, index.getPointCount()) - 1;
Range<Integer> range = Range.closed(firstVariant, lastPoint);
observers.add(new EquipmentTimeSeriesWriterObserver(network, config, maxVariantCount, range, Paths.get("/path/to/equipmentTimeSeriesDir")));
observers.add(new EquipmentGroupTimeSeriesWriterObserver(network, config, maxVariantCount, range, Paths.get("/path/to/equipmentTimeSeriesDir")));

// Apply mapping to network
TimeSeriesMapper mapper = new TimeSeriesMapper(config, network, logger);
TimeSeriesMapperParameters parameters = new TimeSeriesMapperParameters(store.getTimeSeriesDataVersions(), range, true, true, false, mappingParameters.getToleranceThreshold());
mapper.mapToNetwork(store, parameters, observers);

Further documentation is available on the dedicated page on our website.

Metrix

To use metrix with java, add com.powsybl:powsybl-metrix-integration module to your dependencies.

Then you need :

  • a case file
  • a mapping groovy script configuration
  • a time series store
  • a metrix configuration script
  • (optional) a contingency configuration script
  • (optional) a remedial actions configuration file
ComputationManager computationManager = LocalComputationManager.getDefault();

// Network
NetworkSource networkSource = new DefaultNetworkSourceImpl(Paths.get("/path/to/case.xiidm"), computationManager)
    
// Timeseries
InMemoryTimeSeriesStore store = new InMemoryTimeSeriesStore();
store.importTimeSeries(Collections.singletonList(Paths.get("/path/to/timeseries.csv")));

// Contingencies
ContingenciesProvider contingenciesProvider = new GroovyDslContingenciesProvider(Paths.get("/path/to/contingencies.groovy"));

// Mapping
Supplier<Reader> mappingReader = () -> Files.newBufferedReader(Paths.get("/path/to/mapping.groovy"), StandardCharsets.UTF_8);

// Metrix config
Supplier<Reader> metrixDslReader = () -> Files.newBufferedReader(Paths.get("/path/to/metrixConfig.groovy"), StandardCharsets.UTF_8);

// Remedial actions
Supplier<Reader> remedialActionsReader = () -> Files.newBufferedReader(Paths.get("/path/to/remedialActions.txt"), StandardCharsets.UTF_8);

// Result timeseries store
FileSystemTimeseriesStore resultStore = new FileSystemTimeseriesStore(Paths.get("/path/to/outputdir"));

// Result listener
ResultListener listener = new ResultListener() {
    @Override
    public void onChunkResult(int version, int chunk, List<TimeSeries> timeSeriesList, Network networkPoint){
        resultStore.importTimeSeries(timeSeriesList, version, false);
    }
}

// Run metrix configuration analysis
MetrixAnalysis metrixAnalysis = new MetrixAnalysis(networkSource, mappingReader, metrixDslReader, remedialActionsReader, contingenciesProvider, store, logger, computationRange);
MetrixAnalysisResult analysisResult = metrixAnalysis.runAnalysis("extern tool");

// Run metrix
Metrix metrix = new Metrix(remedialActionsReader, store, resultStore, logArchive, computationManager, logger, analysisResult)
MetrixRunParameters runParams = new MetrixRunParameters(firstVariant, variantCount, versions, chunkSize, true, true, false);
metrix.run(runParams, listener);

Further documentation is available on the dedicated page on our website.

Metrix simulator

Metrix simulator is an independant c++ executable. It must be installed before using powsybl-metrix.

It has its own toolchain and requirements.

Inputs / outputs

inputs:

  • fort.json: aggregate file containing network data, mapping and metrix options. The name of this file is frozen
  • variant file: this file describes the network modification to apply to the information present in the fort.json file for each defined variant. The name of this name is chosen at metrix launch (see --help)
  • counter file: this file describes the list of topological actions that are allowed as preemptive actions to counter an issue. The name of this name can be chosen at metrix launch (see --help)
  • index first variant: the first variant to process in the variant file. Is chosen at launch.
  • number of variants: the number of variants to process. Is chosen at launch.

outputs:

  • user log file: the user log file, displaying user friendly information about the finished run. These information are duplicated in the developper log file
  • developper log files: a set of more developpement-axed log files, with more information
  • result files: the result files (1 by variant processed)

all input options are described in --help option

Requirements

To build metrix-simulator, you need:

  • A C++ compiler that supports C++11 (clang 3.3 or higher, g++ 5.0 or higher)
  • CMake (3.12 or higher)
  • Make
  • Boost development packages (1.66 or higher)
Ubuntu 20.04
$> apt install -y cmake g++ git libboost-all-dev libxml2-dev make
Ubuntu 18.04
$> apt install -y g++ git libboost-all-dev libxml2-dev make wget

Note: Under Ubuntu 18.04, the default CMake package is too old (3.10), so you have to install it manually:

$> wget https://cmake.org/files/v3.12/cmake-3.12.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.12.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.12.0-Linux-x86_64/bin:$PATH
CentOS 8
$> yum install -y boost-devel gcc-c++ git libxml2-devel make wget

Note: Under CentOS 8, the default CMake package is too old (3.11.4), so you have to install it manually:

$> wget https://cmake.org/files/v3.12/cmake-3.12.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.12.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.12.0-Linux-x86_64/bin:$PATH
CentOS 7
$> yum install -y gcc-c++ git libxml2-devel make wget

Note: Under CentOS 7, the default boost-devel package is too old (1.53), so we install Boost 1.66 from epel-release.

$> yum install -y epel-release
$> yum install -y boost166-devel
$> export BOOST_INCLUDEDIR=/usr/include/boost166
$> export BOOST_LIBRARYDIR=/usr/lib64/boost166

Note: Under CentOS 7, the default CMake package is too old (2.8.12), so you have to install it manually:

$> wget https://cmake.org/files/v3.12/cmake-3.12.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.12.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.12.0-Linux-x86_64/bin:$PATH
Build sources

1 - Clone the project

$> git clone https://github.com/powsybl/powsybl-metrix.git
$> cd powsybl-metrix/metrix-simulator

2 - Build the project, with 3rd parties
First build the 3rd parties

$> mkdir build
$> mkdir build/external
$> cd build/external
$> cmake ../../external -DCMAKE_BUILD_TYPE=<BUILD_TYPE_3PARTIES>
$> cmake --build .

Then build the executable

$> cd ..
$> cmake .. -DCMAKE_INSTALL_PREFIX=<PREFIX> -DCMAKE_BUILD_TYPE=<BUILD_TYPE>
$> cmake --build . --target install

The following CMAKE options can be set for the executable configuration:

  • USE_SIRIUS_SHARED (default = OFF): If active, project will link using the shared library of sirius solver instead of static library
  • METRIX_RUN_ALL_TESTS (default = ON): If inactive, projects will execute a reduced scope of tests
Contributing
Checkstyle

This project uses clang-tidy to verify the code style. This tool is provided with the clang extra tools. To enable the code style checking, add the -DCMAKE_CXX_CLANG_TIDY=clang-tidy flag to the configure command.

A clang-format file is also provided to format the code by using clang-format. Most of IDEs have a option to format files using clang-format automatically.

Code coverage

This project uses either gcov or llvm-cov to compute the code coverage. We also use gcovr (4.2 or higher) to generate both sonar and HTML reports. To compute the code coverage, add the -DCODE_COVERAGE=TRUE flag to the configure command.

Run metrix simulator

The following environments variables must be defined in order to run metrix properly:
METRIX_ETC: location of the .dic files for language region (some of these dictionnaries are exported in 'etc' directory in install directory)

All options are detailed in the helper

$> ./metrix-simulator --help
Usage:
 metrix-simulator <errorFilepath> <variantFilepath> <resultsFilepath> <firstVariantIndex> <numberVariants> <paradesFilepath> 
<paradesFilepath> = "parades.csv" by default 
[options] 
Metrix options:
  -h [ --help ]                 Display help message
  --log-level arg               Logger level (allowed values are critical, 
                                error, warning, info, debug, trace): default is
                                info
  -p [ --print-log ]            Print developer log in standard output
  --verbose-config              Activate debug/trace logs relative to 
                                configuration
  --verbose-constraints         Activate debug/trace logs relative to 
                                constraint detection
  --write-constraints           Write the constraints in a dedicated file
  --print-constraints           Trace in logs the constraints matrix (time 
                                consuming even if trace logs are not active), 
                                log level at trace is required
  --write-sensitivity           Write the sensivity matrix in a dedicated file
  --write-report                Write the rate matrix report in a dedicated 
                                file
  --check-constraints-level arg Check adding constraints:
                                0: no check (default)
                                1: When adding a constraint, perform a load 
                                flow to check transit (more time consuming)
                                2: When adding a constraint, run every incident
                                to check that we didn't forget a constraint 
                                (even more time consuming
  --compare-reports             Compare load flow reports after application of 
                                report factors to check trigger of coupling
  --no-incident-group           Ignore incident if a group of N-K is not 
                                available
  --all-outputs                 Display all values in results files
  --mps-file                    Export MPS file
Assets 2