-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* WIP * WIP CaseID * WIP Buses * WIP Loads * WIP FixedShunts * WIP Generators * WIP Branches * Add some more docs on internals * WIP Transformers * WIP Q record * WIP specify version * Expand v33 test data * WIP TwoTerminalDCLines * Include version in Network * WIP MultiTerminalDCLines * More hiding of types in printing * WIP MultiSectionLineGroups * WIP FACTSDevices * WIP SwitchedShunts * Remove testfile used in development * Remove GNEDevice placeholder code * Refactor Branches30/33 to be defined in loop * Update docs now v30 and v33 both supported * fixup! Update docs now v30 and v33 both supported * fixup! Update docs now v30 and v33 both supported * Add more specific tests for v33 data * fixup! Update docs now v30 and v33 both supported * fixup! WIP FixedShunts
- Loading branch information
1 parent
1270ec3
commit 5841551
Showing
13 changed files
with
1,635 additions
and
836 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,25 +1,29 @@ | ||
# Alternatives | ||
|
||
In Julia, I am are aware of two other open-source packages with functionality to parse PSS/E files: | ||
## Julia | ||
|
||
I am are aware of two other open-source packages with functionality to parse PSS/E files: | ||
- [PowerModels.jl](https://lanl-ansi.github.io/PowerModels.jl/stable/parser/#PTI-Data-Files-(PSS/E)) | ||
- [PowerSystems.jl](https://nrel-siip.github.io/PowerSystems.jl/stable/modeler_guide/generated_parsing/) | ||
|
||
I have not used either so cannot recommend one over the other. | ||
From what I can see, these parsers are almost identical to each other. | ||
It seems PowerSystems.jl originally vendored the PowerModels.jl code, but the parsers may have diverged slightly over time. | ||
|
||
Importantly, these alternatives take a completely different approach to this package. | ||
Both of these packages only support parsing v33 PSS/E files, and only parse a subset of the data categories in the file. | ||
Whereas PowerFlowData.jl (this package) can parse all data categories from both v30 and v33 PSS/E files. | ||
|
||
Importantly, these alternatives also take a completely different approach to this package. | ||
These other parsers read the `.raw` files as a `String` (e.g. using `readlines`), then operate on string data, and parse strings into other Julia types as necessary. | ||
PowerFlowData.jl reads the `.raw` files as a bytes buffer (`Vector{UInt8}`) then parses the bytes directly into Julia types, | ||
which is much faster and more memory efficient. | ||
|
||
PowerFlowData.jl (this package) reads the `.raw` files as a bytes buffer (`Vector{UInt8}`), then parses the bytes directly into Julia types. | ||
This package was originally developed as a fun exercise, but should correctly implement the full data format specification. | ||
Please feel encouraged to give this package a try, and open issues if you encounter any | ||
problems or missing features. | ||
|
||
Hopefully this will be much faster and more memory efficient, but benchmarks pending. | ||
For now, this package is being developed as a fun exercise. | ||
Feel encouraged to give this package a try (and open issues!), but these alternative parsers are surely more battle-tested! | ||
## Others | ||
|
||
## Implementation details | ||
In Python, there is a package named [`grg-pssedata`](https://github.com/lanl-ansi/grg-pssedata), which says it parses PSSE v33 data files. | ||
|
||
We use [Parsers.jl](https://github.com/JuliaData/Parsers.jl/) to parse bytes into Julia types. | ||
Broadly speaking, we use `Parsers.Options` to configure the parsing based on the `.raw` format (e.g. `,` characters are delimiters), and then `Parsers.xparse` to actually parse the bytes between delimiters into the expected Julia types. | ||
The expected Julia type depends on the category of data we are reading at that point in the file (buses, loads, …); | ||
if the PSS/E user manual says "load records" should come after "bus records", and each load record should have 12 columns with the first column containing an integer "bus number", then we try to parse the first value in a load record as an `Int`, and so on. | ||
In Matlab, the [`MATPOWER`](https://github.com/MATPOWER/matpower/) package says it has parsing functionality for (unspecified) PSSE data files. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Implementation details | ||
|
||
This page has rough notes about the internals of PowerFlowData.jl. | ||
|
||
We use [Parsers.jl](https://github.com/JuliaData/Parsers.jl/) to parse bytes into Julia types. | ||
Broadly speaking, we use `Parsers.Options` to configure the parsing based on the `.raw` format (e.g. `,` characters are delimiters), and then `Parsers.xparse` to actually parse the bytes between delimiters into the expected Julia types. | ||
The expected Julia type depends on the category of data we are reading at that point in the file (buses, loads, …); | ||
if the PSS/E user manual says "load records" should come after "bus records", and each load record should have 12 columns with the first column containing an integer "bus number", then we try to parse the first value in a load record as an `Int`, and so on. | ||
|
||
The aim is to capture the domain knowledge / format specification in the types, | ||
and keep the parsing code as minimal as possible (relying on the types for the domain knowledge). | ||
When trying to support multiple versions of the format, the rough strategy is: | ||
* if a wholly new data category exists in the new format, create a new `Records` subtype | ||
* if a data category has added a new column to the end of a record, add a new `Union{T, Missing}` field to the existing `Records` subtype | ||
* if a data category has added/removed columns in the middle of a record, or changed the element type of a column, create a new `Records` subtype with a common supertype as the existing `Records` subtype for the category | ||
(e.g. `Buses30 <: Buses`, `Buses33 <: Buses`, `Buses <: Records`) | ||
* if a version-specific `Records` subtype is being used, update the top-level `parse_network` function to parse the records into the appropriate type, and in the order expected by the version of the format | ||
(using the version number extracted as part of the Case ID data). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.