Skip to content

Add SRF module enhancements#4

Merged
lispandfound merged 18 commits intomainfrom
srf
Aug 20, 2024
Merged

Add SRF module enhancements#4
lispandfound merged 18 commits intomainfrom
srf

Conversation

@lispandfound
Copy link
Copy Markdown
Contributor

This PR adds a few SRF module enhancements:

  1. A segment view that lets you iterate over fault segments in an SRF efficiently,
  2. A slipt property that is the total slip at each timestep for each patch. The SRF spec stores slip in three components slipt1, slipt2, slipt3, but most often we care about the total slip in all three directions.
  3. A slip property that is the total slip for each patch. The slip parameter is related to slipt as the integral of slipt over time.
  4. Documentation in the class docstring of each column in the points and header dataframe.

These changes are used for the plots I'm generating, and also by Cesar, who is also using the srf module for his work.

@lispandfound
Copy link
Copy Markdown
Contributor Author

Actually made a reasonably large change here to allow me to do the kind of at-scale processing we need to be able to do for type-5 verification. The main change is that now we actually read the SRF entirely in C code. For my testing this improves the performance of the SRF reader by something like 10x. I'm hoping this will cut down on the time to process the SRF analysis from 10 hours to 1 hour. Accordingly, the slip values are now recorded in sparse matrices that are also built in C, and so are no longer part of the points DataFrame. This is disappointing because it makes code handling slip a little more complicated, but is necessary to parse 100Mb+ SRF files efficiently.

As for why we can't use read_csv and friends from pandas? Because the SRF file format has variable-width rows and weird formatting that prevents it from being read in this way. I've tried all the tricks I can think of over the past couple of weeks but I cannot see an efficient SRF parser written in python only with pandas data importing.

Copy link
Copy Markdown
Contributor

@claudio525 claudio525 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Only comment I have is that the C-code needs a lot more comments imo. Adding both docstrings and block comments would make it significantly easier to read & work with this code.

@lispandfound
Copy link
Copy Markdown
Contributor Author

Yeah I forgot to add comments to the srf C code

Copy link
Copy Markdown
Contributor

@AndrewRidden-Harper AndrewRidden-Harper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@lispandfound
Copy link
Copy Markdown
Contributor Author

Needs to a be a draft temporarily while I figure out the SRF writing glitches.

@lispandfound lispandfound marked this pull request as draft August 16, 2024 04:41
@lispandfound lispandfound marked this pull request as ready for review August 18, 2024 22:22
Copy link
Copy Markdown
Member

@sungeunbae sungeunbae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Copy Markdown
Contributor

@AndrewRidden-Harper AndrewRidden-Harper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice visual explanation in the comments

@lispandfound lispandfound merged commit e1d814d into main Aug 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants