BUG: NaNs converted to 1970-01-01 instead of NaT

[timcera]

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np                                                                                              

s = pd.Series([1, 2, np.nan, np.nan, np.nan]).astype('M8[ns]')
print(s)

0 1970-01-01 00:00:00.000000001
1 1970-01-01 00:00:00.000000002
2 1970-01-01 00:00:00.000000000
3 1970-01-01 00:00:00.000000000
4 1970-01-01 00:00:00.000000000
dtype: datetime64[ns]

Problem description

This problem is on the master branch as of commit baeb1bf

All 'nan' like entries, when converted to datetime should be NaT.

Several tests are failing, the example above is from pandas/tests/test_algos.py

Expected Output

0 1970-01-01 00:00:00.000000001
1 1970-01-01 00:00:00.000000002
2 NaT
3 NaT
4 NaT
dtype: datetime64[ns]

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]
INSTALLED VERSIONS

commit: baeb1bf
python: 3.7.3.final.0
python-bits: 32
OS: Linux
OS-release: 3.18.0-19095-g86596f58eadf
machine: aarch64
processor:
byteorder: little
LC_ALL: en_US.utf8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.25.0.dev0+761.gbaeb1bf92.dirty
pytest: 4.5.0
pip: 19.1.1
setuptools: 41.0.1
Cython: 0.29.10
numpy: 1.16.3
scipy: 1.3.0
pyarrow: None
xarray: 0.12.1
IPython: 7.5.0
sphinx: 2.0.1
patsy: None
dateutil: 2.8.0
pytz: 2019.1
blosc: 1.8.1
bottleneck: None
tables: 3.5.2
numexpr: 2.6.9
feather: None
matplotlib: 3.1.0
openpyxl: 2.6.2
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.8
lxml.etree: 4.3.4
bs4: 4.7.1
html5lib: 0.9999999
sqlalchemy: 1.3.4
pymysql: None
psycopg2: None
jinja2: 2.10.1
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: 0.2.2

[TomAugspurger]

I don't think baeb1bf is related. Do you need to rebuild your C extensions?

[timcera]

Yes.

I was working on my development branch to finish #26631 and kept on getting these test errors. Couldn't figure out why anything that I was working on would affect this, so...

git checkout master
python setup.py develop

Correctly re-compiled the one pyx file that I am working on in my development branch, and the 'nan' to 'NaT' tests still fail.

I understand the confusion about me mentioning baeb1bf. I don't mean to imply that commit is the source of this problem, in fact I know that it isn't. I have had these failing tests since I began work on #26631. I just wanted to indicate the version of the code that I am using.

[WillAyd]

I cannot reproduce this on master. Here's my pd.show_versions FWIW not sure if 32 bit or macOS vs Linux for OP matters

INSTALLED VERSIONS

commit : a14874f
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 18.6.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.0.dev0+782.ga14874f3e
numpy : 1.16.4
pytz : 2019.1
dateutil : 2.8.0
pip : 19.1.1
setuptools : 41.0.1
Cython : 0.29.8
pytest : 4.5.0
hypothesis : 4.23.6
sphinx : 2.0.1
blosc : None
feather : None
xlsxwriter : 1.1.8
lxml.etree : 4.3.3
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.10.1
IPython : 7.5.0
pandas_datareader: None
bs4 : 4.7.1
bottleneck : 1.2.1
fastparquet : 0.3.0
gcsfs : None
matplotlib : 3.1.0
numexpr : 2.6.9
openpyxl : 2.6.1
pandas_gbq : None
pyarrow : 0.11.1
pytables : None
s3fs : 0.2.1
scipy : 1.2.1
sqlalchemy : 1.3.3
tables : 3.5.1
xarray : 0.12.1
xlrd : None
xlwt : 1.3.0
xlsxwriter : 1.1.8

[timcera]

Seems like a problem with my numpy install...

np.array([1,2,np.nan,np.nan]).astype('M8[ns]')

array(['1970-01-01T00:00:00.000000001', '1970-01-01T00:00:00.000000002',
   '1970-01-01T00:00:00.000000000', '1970-01-01T00:00:00.000000000'],
  dtype='datetime64[ns]')

Closing

[timcera]

Here is the numpy bug. Three years old...

numpy/numpy#8325