Several datetime tests fail on arm64, armel, armhf, mips, mips64el, mipsel, s390x. NaT seems to be handled differently.

[detrout]

For example this chunk of unit test code from pandas.tests.frame.test_indexing.py test_where_datetime

from datetime import datetime
from pandas import DataFrame, date_range
import numpy as np
from pandas.util.testing import assert_frame_equal

df = DataFrame(dict(A=date_range('20130102', periods=5),
                    B=date_range('20130104', periods=5),
                    C=np.random.randn(5)))

stamp = datetime(2013, 1, 3)
result = df[df > stamp]
expected = df.copy()
expected.loc[[0, 1], 'A'] = np.nan
assert_frame_equal(result, expected)

produces these result & expected on arm64

>>> result
           A          B         C
0 1970-01-01 2013-01-04 -0.238902
1 1970-01-01 2013-01-05  0.055246
2 2013-01-04 2013-01-06  0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08  2.580124
>>> expected
           A          B         C
0        NaT 2013-01-04 -0.238902
1        NaT 2013-01-05  0.055246
2 2013-01-04 2013-01-06  0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08  2.580124
>>> 

and these results on amd64

           A          B         C
0        NaT 2013-01-04 -1.151324
1        NaT 2013-01-05  0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07  2.129802
4 2013-01-06 2013-01-08  0.103666

In [11]: expected
Out[11]: 
           A          B         C
0        NaT 2013-01-04 -1.151324
1        NaT 2013-01-05  0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07  2.129802
4 2013-01-06 2013-01-08  0.103666

The full build log showing more arm64 tests failures is available at:
https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=arm64&ver=0.20.3-2&stamp=1506059703&raw=0 and the buildd status page showing all of the failures & successes:
https://buildd.debian.org/status/package.php?p=pandas

Obviously it'd be nice if the tests behaved consistently independent of architecture.

Output of pd.show_versions()

pandas.show_versions() on arm64

pandas.show_versions() (failure)

INSTALLED VERSIONS

commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-3-arm64
machine: aarch64
processor:
byteorder: little
LC_ALL: C
LANG: C
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: None
setuptools: 36.2.7
Cython: 0.26.1
numpy: 1.13.1
scipy: 0.19.1
xarray: None
IPython: None
sphinx: 1.6.4
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

From a working amd64 system:

INSTALLED VERSIONS

commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.12.0-2-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: C
LANG: C
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: None
setuptools: 36.2.7
Cython: 0.26.1
numpy: 1.13.1
scipy: None
xarray: 0.9.2
IPython: 5.1.0
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

dateutil is 2.6.1 in both environments.
The amd64 test environment is not exactly the same as the buildd. but these tests do pass in the buildd.

[detrout]

slowly poking at this:

On arm64

>>> df[df>stamp]
           A          B         C
0 1970-01-01 2013-01-04 -0.675049
1 1970-01-01 2013-01-05 -0.242239
2 2013-01-04 2013-01-06 -0.785362
3 2013-01-05 2013-01-07 -0.207237
4 2013-01-06 2013-01-08 -0.059387
>>> type(df[df>stamp]['A'][0])
<class 'pandas._libs.tslib.Timestamp'>

on amd64

In [8]: df[df>stamp]
Out[8]: 
           A          B         C
0        NaT 2013-01-04 -0.213889
1        NaT 2013-01-05  1.045750
2 2013-01-04 2013-01-06 -0.148973
3 2013-01-05 2013-01-07 -1.205978
4 2013-01-06 2013-01-08 -0.424786

In [9]: type(df[df>stamp]['A'][0])
Out[9]: pandas.tslib.NaTType

So why NaTType vs Timestamp?

[jreback]

we only support little endian platforms. Really not sure much can do here. We don't have CI support, nor development resources anything but the core platforms.

[llimeht]

Worth noting, then, that arm64, armel, armhf, mips64el, and mipsel are little endian so endianness isn't the issue. Of the list of failing architectures, only mips and s390x are big endian.

arm* are a common enough architectures for which pandas has users, so they are probably worth supporting, even though the others are considerably more exotic; numpy and dateutil test suites pass ok on all of them.

[llimeht]

For CI, would travis-ci/travis-ci#3376 be enough? (pandas has a lot of tests that take a long time to run so perhaps it won't be fast enough to avoid time-outs under emulation)

[jreback]

@llimeht yes that might work, didn't even know that was an option. if you were willing to do a PR to add a (allowed failures) job to travis (and you can make it minimal for the moment, only installed base dependencies and even have a very limited tests suite) would be ok to start. at least seeing issues would be good.
can always add to it and/or have multiple builds.

[detrout]

Getting a CI solution up seems like a good general solution, but I am willing to at least try and find what the problem is. Any suggestions would be appreciated though.

[yarikoptic]

@detrout any luck with trying to get NaT problems (re)solved on ARMs? those failures hinder Debian builds so I would be really excited to get them resolved properly . FWIW those seems to be the only once holding up the build to announce clean in Debian: sample build log where you could find also tests failures/outputs: https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=armel&ver=0.22.0-2&stamp=1519375446&raw=0

[yarikoptic]

Ah -- another issue (still open) is #11282 and points to numpy numpy/numpy#8325 (I did feel that we already investigated it before ;-) )