fix for 61123 read_excel nrows param reads extra rows #61129
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue: GH-61123
When reading Excel files with
pd.read_excel
and specifyingnrows=4
, the behavior differs depending on whether there’s a blank row between tables. For a file with two tables (each with a header and 3 data rows),nrows=4
should yield a DataFrame with one header and 3 data rows (shape(3, n)
). However:test1.xlsx
(with a blank row), it correctly reads the first table (header + 3 rows).test2.xlsx
(no blank row), it incorrectly includes the second table’s header as a data row, resulting in a shape of(4, n)
.This inconsistency occurs because
read_excel
doesn’t properly respect table boundaries when tables are adjacent, despite thenrows
limit.Fix:
pandas/io/excel/_base.py
and related reader modules (_openpyxl.py
,_pyxlsb.py
,_xlrd.py
) to ensurenrows
limits reading to the specified number of rows, excluding subsequent table headers even when tables are adjacent.test_excel_read_tables_with_and_without_blank_row
inpandas/tests/io/excel/test_readers.py
to verify thatnrows=4
consistently returns a DataFrame with shape(3, 2)
(header + 3 data rows) for both cases.Changes:
nrows
without parsing beyond table boundaries.openpyxl
,pyxlsb
, andxlrd
engines.Verification:
test1.xlsx
(blank row) andtest2.xlsx
(no blank row).(3, 2)
and only the first table’s data.Steps to Test:
pytest pandas/tests/io/excel/test_readers.py::TestReaders::test_excel_read_tables_with_and_without_blank_row
.df1.shape == (3, 2)
anddf2.shape == (3, 2)
match the expected output.Related Files:
pandas/io/excel/_base.py
pandas/io/excel/_openpyxl.py
pandas/io/excel/_pyxlsb.py
pandas/io/excel/_xlrd.py
pandas/tests/io/excel/test_readers.py
Closes #61123