Add write_gtf: serialize a DataFrame back to GTF (fixes #21)#72
Merged
Conversation
Adds the inverse of read_gtf. A DataFrame produced by read_gtf -- whether the attribute column was expanded into per-key columns or left as a raw 'attribute' string -- can be written back out and re-read to recover an equivalent DataFrame. This builds on the approach in #46 by Benoitdw, with fixes: - Python 3.9 compatibility: use typing.Union / typing.Optional instead of the `str | Path` union syntax, which requires Python 3.10+ (the project targets >=3.9). - No silent data loss: attribute values are omitted only when None (the way read_gtf represents an absent key), not when merely falsy. Values like 0 or the empty string are now written and survive a round trip. - Missing values in the fixed columns are serialized as '.', and null scores/frames round-trip correctly. - The attribute field ends with a trailing ';', matching the canonical GTF format emitted by Ensembl/GENCODE and re-parsed by read_gtf. - A column literally named 'attribute' (unexpanded read) is emitted verbatim, so expand_attribute_column=False round-trips too. - Accepts a pandas DataFrame as well as polars (result_type="pandas"). - Raises ValueError if any required fixed column is missing. Tests cover round trips for RefSeq and Ensembl fixtures (expanded and unexpanded), pandas input, header lines, missing-value handling, the falsy-value regression, and the missing-column error. Co-authored-by: Benoitdw <bw@oncodna.com> Claude-Session: https://claude.ai/code/session_014fNxSBm5zvdsDNwN3nhmpS
Coverage Report for CI Build 28970590726Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage increased (+0.5%) to 95.618%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
…tions Addresses the review findings on the first draft: - Faithful missing-value handling. read_gtf represents an absent attribute as null (numeric *_version columns) or the empty string (string columns) and cannot tell absent from present-but-empty. The writer now omits both null and "" so that read -> write -> read reproduces the parsed frame exactly, including column order, on multi-feature GTFs (GENCODE). A non-empty falsy value such as "0" is still written (the original #46 bug). - Vectorized. The attribute field and full line are built with polars expressions (concat_str/format) instead of per-row Python; ~200k rows x 6 attributes now writes in ~0.1s. - gzip output. A path ending in .gz is gzip-compressed, mirroring read_gtf which transparently reads gzipped GTFs. - Dropped the redundant fixed-column filter; fixed columns are always written in canonical order (the missing-column guard runs first). - Documented that "\"" and ";" in attribute values cannot round-trip (a limitation shared with read_gtf, which strips quotes and splits on ";"). Tests now cover both directions across RefSeq/Ensembl/GENCODE/StringTie fixtures: write(read(...)) recovers the parsed frame and is byte-idempotent (a fixed point); read(write(...)) recovers a DataFrame's values. Plus gzip, pandas input, the "0"-vs-empty-vs-null semantics, "." for missing fixed values, header lines, the structural-character limitation, and the missing-column error. Co-authored-by: Benoitdw <bw@oncodna.com> Claude-Session: https://claude.ai/code/session_014fNxSBm5zvdsDNwN3nhmpS
Follow-ups from review of #72: - Detect the gzip suffix case-insensitively (.GZ as well as .gz). - Note in the docstring that a literal tab or newline in an attribute value is unrepresentable too, alongside the existing " and ; caveat. - Add tests for the zero-row DataFrame (writes only header lines), a DataFrame with only the fixed columns (valid 9-field line, empty attribute), and case-insensitive .GZ detection. Co-authored-by: Benoitdw <bw@oncodna.com> Claude-Session: https://claude.ai/code/session_014fNxSBm5zvdsDNwN3nhmpS
Minor release: adds write_gtf, the inverse of read_gtf, for serializing a DataFrame of genomic features back to GTF (#21). Claude-Session: https://claude.ai/code/session_014fNxSBm5zvdsDNwN3nhmpS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
write_gtf, the inverse ofread_gtf, closing #21 (the library could read GTF but not write it). A DataFrame fromread_gtf— whether attributes were expanded into per-key columns or left as a rawattributestring — can be written back out and re-read to recover an equivalent DataFrame.Credit
This adopts and finishes @Benoitdw's work in #46, which I've closed in favor of this branch. Original authorship is preserved via a
Co-authored-bytrailer.What changed vs. #46
Building on the original approach, this fixes several correctness/compat issues that were blocking it:
str | Pathunion syntax, which requires Python 3.10+; the project targets>=3.9. Switched totyping.Union/typing.Optional.and (row[field])), silently losing0,"",False. Attributes are now omitted only when the value isNone(howread_gtfmarks an absent key), so falsy-but-present values survive the round trip.Nonein the fixed columns is serialized as., and null scores/frames round-trip correctly.;, matching Ensembl/GENCODE output andread_gtf's parser.attribute(fromexpand_attribute_column=False) is emitted verbatim, so that mode round-trips too.result_type="pandas").ValueErrorif a required fixed column is missing.Testing
tests/test_write_gtf.py(8 tests) covers:0/""preserved,Noneomitted).) in fixed columnsValueErrorFull suite: 67 passed.
ruff check+ruff format --checkclean. Verified the module parses under the Python 3.9 grammar.Fixes #21.
https://claude.ai/code/session_014fNxSBm5zvdsDNwN3nhmpS