Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: test_nodata respect endianness #15933

Merged
merged 2 commits into from Apr 26, 2022

Conversation

tylerjereddy
Copy link
Contributor

@tylerjereddy tylerjereddy commented Apr 3, 2022

Fixes #15931

  • the nodata.arff file read by this test contains
    no data whatsoever that could possibly indicate
    the desired endianness of the empty data fields,
    so adjust the test to expect the native endianness,
    and pass on the big endian gcc compile farm machine
    gcc203

  • even if there were actual floating point values
    in the fields, I still don't see how we'd retrive
    the desired endianness from those--it seems like
    we'd still end up with the native endianness of
    the machine reading the file, and this is indeed
    what happens for me when I test on gcc203 with
    our loadarff docstring example:

Python 3.9.12 (main, Mar 24 2022, 13:02:21)
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.io import arff
>>> from io import StringIO
>>> content = """
... @relation foo
... @attribute width  numeric
... @attribute height numeric
... @attribute color  {red,green,blue,yellow,black}
... @data
... 5.0,3.25,blue
... 4.5,3.75,green
... 3.0,4.00,red
... """
>>> f = StringIO(content)
>>> data, meta = arff.loadarff(f)
>>> data
array([(5. , 3.25, b'blue'), (4.5, 3.75, b'green'), (3. , 4.  ,
b'red')],
      dtype=[('width', '>f8'), ('height', '>f8'), ('color', 'S6')])
>>>
  • the only other solution I could think of is if this file
    format specification stipulated that retrieved data were
    to be encoded as little endian--I didn't find evidence of
    that based on the descriptions I could find online for
    this ASCII file format

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks right to me, thanks for the detailed description @tylerjereddy

scipy/io/arff/tests/test_arffread.py Show resolved Hide resolved
* the `nodata.arff` file read by this test contains
no data whatsoever that could possibly indicate
the desired endianness of the empty data fields,
so adjust the test to expect the native endianness
and pass on the big endian gcc compile farm machine
`gcc203`

* even if there were actual floating point values
in the fields, I still don't see how we'd retrive
the desired endianness from those--it seems like
we'd still end up with the native endianness of
the machine reading the file, and this is indeed
what happens for me when I test on `gcc203` with
our `loadarff` docstring example:

```python
Python 3.9.12 (main, Mar 24 2022, 13:02:21)
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.io import arff
>>> from io import StringIO
>>> content = """
... @relation foo
... @Attribute width  numeric
... @Attribute height numeric
... @Attribute color  {red,green,blue,yellow,black}
... @DaTa
... 5.0,3.25,blue
... 4.5,3.75,green
... 3.0,4.00,red
... """
>>> f = StringIO(content)
>>> data, meta = arff.loadarff(f)
>>> data
array([(5. , 3.25, b'blue'), (4.5, 3.75, b'green'), (3. , 4.  ,
b'red')],
      dtype=[('width', '>f8'), ('height', '>f8'), ('color', 'S6')])
>>>
```
* account for all cases in the conditional used in
`test_nodata()`, based on reviewer feedback
@rgommers rgommers added the maintenance Items related to regular maintenance tasks label Apr 26, 2022
Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now, thanks @tylerjereddy

@rgommers rgommers added this to the 1.9.0 milestone Apr 26, 2022
@rgommers rgommers merged commit 787e010 into scipy:main Apr 26, 2022
@tylerjereddy tylerjereddy deleted the treddy_issue_15931 branch April 26, 2022 20:53
swallan pushed a commit to swallan/scipy that referenced this pull request Apr 28, 2022
* the `nodata.arff` file read by this test contains
no data whatsoever that could possibly indicate
the desired endianness of the empty data fields,
so adjust the test to expect the native endianness
and pass on the big endian gcc compile farm machine
`gcc203`

* even if there were actual floating point values
in the fields, I still don't see how we'd retrive
the desired endianness from those--it seems like
we'd still end up with the native endianness of
the machine reading the file, and this is indeed
what happens for me when I test on `gcc203` with
our `loadarff` docstring example:

```python
Python 3.9.12 (main, Mar 24 2022, 13:02:21)
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.io import arff
>>> from io import StringIO
>>> content = """
... @relation foo
... @Attribute width  numeric
... @Attribute height numeric
... @Attribute color  {red,green,blue,yellow,black}
... @DaTa
... 5.0,3.25,blue
... 4.5,3.75,green
... 3.0,4.00,red
... """
>>> f = StringIO(content)
>>> data, meta = arff.loadarff(f)
>>> data
array([(5. , 3.25, b'blue'), (4.5, 3.75, b'green'), (3. , 4.  ,
b'red')],
      dtype=[('width', '>f8'), ('height', '>f8'), ('color', 'S6')])
>>>
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Items related to regular maintenance tasks scipy.io
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: scipy/io/arff/tests/test_arffread.py::TestNoData::test_nodata failure on big endian platforms
2 participants