Skip to content

Performance Improvements to preprocess + small changes in main - #2

Merged
grosenberger merged 2 commits into
grosenberger:masterfrom
dgorhe:master
Oct 5, 2022
Merged

grosenberger merged 2 commits into
grosenberger:masterfrom
dgorhe:master

Conversation

@dgorhe

@dgorhe dgorhe commented Oct 4, 2022 •

Copy link
Copy Markdown
Collaborator

Hey there, I'm working in Marko Jovanovic's lab at Columbia University and I wanted to pull in some changes to SECAT. We use SECAT a lot but it takes a lot of time to run so I made some performance improvements that should help the average user. Some changes are more oriented toward general usability. The changes include:

  • Only importing used functions from package namespace in main (e.g. from pandas import read_csv vs import pandas as pd)
  • Using C engine for read_csv calls in preprocess.py wherever a separator is specified to improve parsing performance
  • Adding a progress bar for namespace parsing in uniprot class within preprocess.py
  • Checking if the df['run_id'] column is all strings before using the pandas.DataFrame.apply method within quantification.read_long() and quantification_read_matrix() in preprocess.py
  • Multiprocessing support within normalization.slide_normalize() to use all available cores to normalize windows in parallel

Let me know if you have any feedback for these changes. Thanks!

dgorhe added 2 commits October 4, 2022 14:37
… normalization.normalize() function; progress bar for uniprot entry appending
@dgorhe
dgorhe marked this pull request as ready for review October 4, 2022 20:25

@grosenberger grosenberger left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks for the PR!

@grosenberger
grosenberger merged commit 6bb4bbf into grosenberger:master Oct 5, 2022
grosenberger pushed a commit that referenced this pull request Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants