Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parsing of multi-terminal DC lines #43

Merged
merged 8 commits into from
Oct 26, 2021
Merged

Conversation

nickrobinson251
Copy link
Owner

@nickrobinson251 nickrobinson251 commented Oct 26, 2021

there's a reason i left this til last... 😂 so it turns out "multi-terminal DC line" data is about as wild as "transformers" data, but in a wholey different way.

Originally i thought this PSSE / power flow data format was like a bunch of CSV files appended together, each with a different schema.
i.e. each "section" would be made up of an arbitrary number of rows, but the rows would have a fixed, documented, known number of columns.

But it turns out some sections have variable number of columns (not "empty cells" like you might have in a CSV, but just no representation in the file that there are columns with no data). So basically rows in the same section can have different schemas. E.g. Shunts, Impedance Corrections, and maybe others (i'll need to go back and review)

And then it turns out in some sections each "row" is actually multiple lines of data, so to read one "record" you actually have to read multiple lines.

[aside: Also the manual i have (apart from being really old) is really inconsistent about how it uses the work "record", sometimes using it to mean a line in the data and sometimes a whole row-like-thing that may actually be multiple rows (so a records might be made up of 3 records 😆)]

The Transformers data is still the maddest.. combining "multiple lines" with "schema changes" (#17)

But multi-terminal DC lines are fun too! Each entry (what this package calls a "record") is kinda like its own little file:

  • there's a line saying with some data, some of it kinda like metadata saying what's coming next
  • then some rows about one thing, with one schema
  • then some rows about another thing, with another schema
  • then some rows about a third thing, with a third schema
    and then there may be another entry like this or not

trying to find PSSE files in the wild is kinda difficult, but i couldn't find any that actually contain "multi-terminal DC lines"

And i'm not aware of any other parsers that support parsing multi-terminal DC lines

so this was probably just a fun exercise

anyway, here it is

Also i slightly feel like there's 1 or 2 many layers of nesting going on in the solution in this PR... but i'll review with a fresh mind tomorrow

src/types.jl Outdated Show resolved Hide resolved
src/types.jl Outdated Show resolved Hide resolved
src/types.jl Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

Merging #43 (6903252) into main (f234d9c) will increase coverage by 0.22%.
The diff coverage is 96.42%.

❗ Current head 6903252 differs from pull request most recent head 9188da6. Consider uploading reports for the commit 9188da6 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main      #43      +/-   ##
==========================================
+ Coverage   93.91%   94.13%   +0.22%     
==========================================
  Files           3        3              
  Lines         230      273      +43     
==========================================
+ Hits          216      257      +41     
- Misses         14       16       +2     
Impacted Files Coverage Δ
src/types.jl 95.50% <92.59%> (-1.47%) ⬇️
src/parsing.jl 98.84% <100.00%> (+0.15%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f234d9c...9188da6. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants