Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Datetime cleanup (mostly for Python 3) #161

Closed
wants to merge 12 commits into from

4 participants

@mwiebe
Owner

This set of patches fixes up the datetime functionality to pass all tests in Python 3.1 and 3.2.

mwiebe added some commits
@mwiebe mwiebe BUG: datetime: Make datetime_as_string produce a UNICODE array instea…
…d of STRING array on Python3
2f3c1c9
@mwiebe mwiebe BUG: datetime: Fix str() function of datetime arrays
Done by making the unit given by datetime_data be a string instead of
bytes in Python 3.
05cc9ce
@mwiebe mwiebe BUG: datetime: Fix timedelta / timedelta -> float64 on Python 3 0490a38
@mwiebe mwiebe WRN: Remove many warnings when building at a higher warning level
This includes warnings about shadowed variables, some bad casts,
and others. Fixing these warnings also fixed a memory leak in one instance.
f507e10
@mwiebe mwiebe BUG: Add %lld exception for Python 3.1 as well 4e736e9
@charris
Owner

Looks OK

@charris
Owner

The function is creating ASCII strings, which are being buffered to UNICODE output, hence NO can't work as a casting rule. UNSAFE is fastest, since it skips all checks.

@charris

Another warning fix?

Owner

Yeah, shadowed variable.

@charris
Owner

Looks OK.

@charris
Owner

Looks OK. I wish there was a good way to centralize all this info.

Owner

The *_FMT macros are an attempt to do that, but they have an unfortunate deceptive nature to them as currently done. I think it's better to keep things simple and special cased like this until someone goes in and fixes them in an acceptable way.

@charris
Owner

Looks OK. Did you just need to handle another type of timedelta?

Owner

I missed true_divide and floor_divide when adding the mm_d case before. Python3 caught the error because it defaults to using true_divide.

Owner

Does this need a test then?

Owner

I added a minimal test which calls divide, true_divide, and floor_divide.

@charris
Owner

Looks OK.

@charris
Owner

Looks good to me.

@mwiebe
Owner

Thanks for the review!

@charris

Is floor divide right? Using an equality for 6.0/9.0 might also be a bit problematic because the compiler might send such things to different FPU's, especially on 32 bit Intel hardware. Maybe something that has an exact representation would be better, say 8.0/32.0.

Owner

I suppose floor divide should do a floor - when I added this I just wanted timedelta / timedelta to produce a float, and missed both true_divide and floor_divide. Maybe just removing support for timedelta floor_divide is reasonable?

Owner

There haven't been any issues with the 6.0/9.0 in practice, have there? Even on 32-bit Intel hardware, I would expect both SSE and the x87 FPU to produce the same result for that particular division.

Owner

Not that I know of, but you have assert_equal which I thought a bit risky for floats.

Owner

The reason I asked about floor_divide is that the correct operator mapping is //.

In [1]: 6.0//9.0
Out[1]: 0.0
Owner

I'm pretty sure it's only risky when there are opportunities for things like compiler optimizations, order of operations, discrepencies in parsing the input decimal number, or similar numeric artifacts to get in the way and slightly change the answers, and this isn't one of those cases. The cases where I could imagine a problem are really pushing it, like one divide happening on the CPU, and the other on a GPU.

Owner

Now I would expect the // operator to fail with a type error when given two timedelta64s. Hopefully that's ok?

@cournape cournape commented on the diff
numpy/core/src/multiarray/datetime_strings.c
@@ -1429,12 +1429,31 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
}
}
- /* Create the output string data type with a big enough length */
+ /* Get a string size long enough for any datetimes we're given */
+ strsize = get_datetime_iso_8601_strlen(local, unit);
+#if defined(NPY_PY3K)
@cournape Owner

Would it be possible to factor this out, to avoid defines inside function bodies ? I.e.

#ifdef NPY_PY3K
void foo()
{
// some code here
}
#endif

rather than

some_large_function(...)
{
....
#ifdef NPY_PY3K
// some code here
#endif
}

I know this is not strictly followed in numpy codebase, but it would be nice to avoid too much of this IMO.

@mwiebe Owner
mwiebe added a note

I've thought a bit about how to do this, but splitting this part out feels very artificial and doesn't seem like it would improve the clarity of the code at all. I'd rather keep it inline as it is. It's an artifact of the design choice to make NPY_STRING arrays not return strings in Python 3, something that's probably too late to change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@87
87 commented

I noticed an interesting feature in the np.datetime64 construction (on Linux):
It cannot parse datetimes beyond 2038,

>>> np.datetime64('2038-01-20T11:52')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: Failed to use mktime to convert local time to UTC

but specifying a timezone makes it loop around:

>>> np.datetime64('2038-01-20T11:52+0100')
numpy.datetime64('1901-12-15T04:43+0019')
@mwiebe
Owner

The Y2038 bug seems to be restricted to 32-bit Linux as far as I can tell. On Windows, there is datetime64_t, which I believe all your code changes still use, correct? In searching, it appears as if glibc didn't follow Microsoft in adding 64-bit variants of these important functions, and reading comments on forums reveals some perspectives like "all computers will be 64-bit before 2038 rolls around, so you shouldn't have to worry about it."

The hack I've thought of is to shift years later than 2038 to before 2038, do the OS-specific stuff, then add the year offset back in once it's in NumPy's hands again.

@mwiebe
Owner

Looking at your specific example where you specify the timezone, I wouldn't expect that to fail, since it shouldn't be passing through a time_t in that case. Does it fail with the "Z" time zone as well?

edit: The fail point would be converting to a local time for printing, so hopefully this works with the Y2038 hack I've added.

mwiebe added some commits
@mwiebe mwiebe BUG: Remove floor_divide support for timedelta64 // timedelta64 00c792d
@mwiebe mwiebe TST: Add tests for the Y2038 problem
This only seems to be cropping up on 32-bit Linux, where time_t is 32 bits,
and there appears to be no 64-bit time_t extension like on Windows.
dbec25a
@mwiebe mwiebe BUG: Attempt to fix the Y2038 problem parsing dates >= 2038
Also incorporated part of Han Genuit's pull request #23, to try
and reduce the difficulty dealing with the merge conflict that
will result from these concurrent changes
a7a8c6d
@mwiebe
Owner

What do people think is a reasonable way to deal with the issue on Windows, where the standard library doesn't want to apply timezones to dates before 1970? Should the 'Z' timezone be always used for dates prior to 1970 on that platform, or on all platforms?

@mwiebe mwiebe BUG: Attempt Y2038 fix, and restrict local timezone parsing to 1970 a…
…nd later, because of Win32

On MS Windows, dates earlier than 1970 cause mktime to error. This patch
also applies to parsing the same Y2038 hack that was done for printing.
2e90a29
@mwiebe
Owner

I've restricted local parsing to dates >= 1970 as I suggested, and applied the Y2038 hack I proposed to both printing and parsing. I don't have platforms to test this on, so will need help with that!

@87
87 commented

Thanks for the fixes! It tests OK on Linux (32bit). (Except for an unrelated polyfit issue.)

Hopefully, the 2038 hack wont give problems with calendar conversions, but it might be the best solution on 32bit for now..

@87 87 referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@charris
Owner

I've merged this in aded70c..10ed90c.

@charris charris closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 17, 2011
  1. @mwiebe

    BUG: datetime: Make datetime_as_string produce a UNICODE array instea…

    mwiebe authored
    …d of STRING array on Python3
  2. @mwiebe

    BUG: datetime: Fix str() function of datetime arrays

    mwiebe authored
    Done by making the unit given by datetime_data be a string instead of
    bytes in Python 3.
  3. @mwiebe
  4. @mwiebe

    WRN: Remove many warnings when building at a higher warning level

    mwiebe authored
    This includes warnings about shadowed variables, some bad casts,
    and others. Fixing these warnings also fixed a memory leak in one instance.
  5. @mwiebe
Commits on Sep 18, 2011
  1. @mwiebe
  2. @mwiebe
Commits on Sep 30, 2011
  1. @mwiebe
  2. @mwiebe
  3. @mwiebe

    TST: Add tests for the Y2038 problem

    mwiebe authored
    This only seems to be cropping up on 32-bit Linux, where time_t is 32 bits,
    and there appears to be no 64-bit time_t extension like on Windows.
  4. @mwiebe

    BUG: Attempt to fix the Y2038 problem parsing dates >= 2038

    mwiebe authored
    Also incorporated part of Han Genuit's pull request #23, to try
    and reduce the difficulty dealing with the merge conflict that
    will result from these concurrent changes
  5. @mwiebe

    BUG: Attempt Y2038 fix, and restrict local timezone parsing to 1970 a…

    mwiebe authored
    …nd later, because of Win32
    
    On MS Windows, dates earlier than 1970 cause mktime to error. This patch
    also applies to parsing the same Y2038 hack that was done for printing.
This page is out of date. Refresh to see the latest.
Showing with 514 additions and 323 deletions.
  1. +1 −1  LICENSE.txt
  2. +2 −0  numpy/core/code_generators/generate_umath.py
  3. +1 −1  numpy/core/include/numpy/npy_3kcompat.h
  4. +1 −6 numpy/core/src/multiarray/arrayobject.c
  5. +1 −2  numpy/core/src/multiarray/arraytypes.c.src
  6. +2 −0  numpy/core/src/multiarray/buffer.c
  7. +8 −8 numpy/core/src/multiarray/calculation.c
  8. +2 −0  numpy/core/src/multiarray/common.c
  9. +4 −4 numpy/core/src/multiarray/convert.c
  10. +20 −5 numpy/core/src/multiarray/datetime.c
  11. +2 −2 numpy/core/src/multiarray/datetime_busdaycal.c
  12. +282 −126 numpy/core/src/multiarray/datetime_strings.c
  13. +14 −14 numpy/core/src/multiarray/getset.c
  14. +2 −1  numpy/core/src/multiarray/item_selection.c
  15. +26 −26 numpy/core/src/multiarray/iterators.c
  16. +50 −48 numpy/core/src/multiarray/mapping.c
  17. +8 −27 numpy/core/src/multiarray/methods.c
  18. +2 −2 numpy/core/src/multiarray/nditer_pywrap.c
  19. +21 −21 numpy/core/src/multiarray/number.c
  20. +16 −6 numpy/core/src/multiarray/scalartypes.c.src
  21. +3 −3 numpy/core/src/multiarray/sequence.c
  22. +13 −13 numpy/core/src/multiarray/shape.c
  23. +8 −6 numpy/core/src/umath/loops.h
  24. +2 −0  numpy/core/src/umath/loops.h.src
  25. +23 −1 numpy/core/tests/test_datetime.py
View
2  LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (c) 2005-2009, NumPy Developers.
+Copyright (c) 2005-2011, NumPy Developers.
All rights reserved.
Redistribution and use in source and binary forms, with or without
View
2  numpy/core/code_generators/generate_umath.py
@@ -287,6 +287,7 @@ def english_upper(s):
TD(intfltcmplx),
[TypeDescription('m', FullTypeDescr, 'mq', 'm'),
TypeDescription('m', FullTypeDescr, 'md', 'm'),
+ #TypeDescription('m', FullTypeDescr, 'mm', 'd'),
],
TD(O, f='PyNumber_FloorDivide'),
),
@@ -299,6 +300,7 @@ def english_upper(s):
TD(flts+cmplx),
[TypeDescription('m', FullTypeDescr, 'mq', 'm'),
TypeDescription('m', FullTypeDescr, 'md', 'm'),
+ TypeDescription('m', FullTypeDescr, 'mm', 'd'),
],
TD(O, f='PyNumber_TrueDivide'),
),
View
2  numpy/core/include/numpy/npy_3kcompat.h
@@ -15,7 +15,7 @@
#if PY_VERSION_HEX >= 0x03000000
#ifndef NPY_PY3K
-#define NPY_PY3K
+#define NPY_PY3K 1
#endif
#endif
View
7 numpy/core/src/multiarray/arrayobject.c
@@ -209,15 +209,9 @@ PyArray_CopyObject(PyArrayObject *dest, PyObject *src_object)
/* If there's one dest element and src is a Python scalar */
if (PyArray_IsScalar(src_object, Generic)) {
- PyArray_Descr *dtype;
char *value;
int retcode;
- dtype = PyArray_DescrFromScalar(src_object);
- if (dtype == NULL) {
- Py_DECREF(src_object);
- return -1;
- }
value = scalar_value(src_object, dtype);
if (value == NULL) {
Py_DECREF(dtype);
@@ -235,6 +229,7 @@ PyArray_CopyObject(PyArrayObject *dest, PyObject *src_object)
/* Assigning NA affects the mask if it exists */
else if (na != NULL) {
if (PyArray_AssignNA(dest, na, NULL, 0, NULL) < 0) {
+ Py_DECREF(dtype);
Py_DECREF(na);
Py_DECREF(src_object);
return -1;
View
3  numpy/core/src/multiarray/arraytypes.c.src
@@ -663,7 +663,6 @@ VOID_getitem(char *ip, PyArrayObject *ap)
*/
{
intp dims[1], strides[1];
- PyArray_Descr *descr;
dims[0] = itemsize;
strides[0] = 1;
descr = PyArray_DescrNewFromType(NPY_BYTE);
@@ -672,7 +671,7 @@ VOID_getitem(char *ip, PyArrayObject *ap)
PyArray_ISWRITEABLE(ap) ? NPY_ARRAY_WRITEABLE : 0,
NULL);
Py_INCREF(ap);
- if (PyArray_SetBaseObject(u, ap) < 0) {
+ if (PyArray_SetBaseObject(u, (PyObject *)ap) < 0) {
Py_DECREF(u);
return NULL;
}
View
2  numpy/core/src/multiarray/buffer.c
@@ -21,6 +21,7 @@
/* removed multiple segment interface */
+#if !defined(NPY_PY3K)
static Py_ssize_t
array_getsegcount(PyArrayObject *self, Py_ssize_t *lenp)
{
@@ -72,6 +73,7 @@ array_getcharbuf(PyArrayObject *self, Py_ssize_t segment, constchar **ptrptr)
{
return array_getreadbuf(self, segment, (void **) ptrptr);
}
+#endif /* !defined(NPY_PY3K) */
/*************************************************************************
View
16 numpy/core/src/multiarray/calculation.c
@@ -42,8 +42,8 @@ PyArray_ArgMax(PyArrayObject *op, int axis, PyArrayObject *out)
PyArrayObject *ap = NULL, *rp = NULL;
PyArray_ArgFunc* arg_func;
char *ip;
- intp *rptr;
- intp i, n, m;
+ npy_intp *rptr;
+ npy_intp i, n, m;
int elsize;
NPY_BEGIN_THREADS_DEF;
@@ -56,13 +56,13 @@ PyArray_ArgMax(PyArrayObject *op, int axis, PyArrayObject *out)
*/
if (axis != PyArray_NDIM(ap)-1) {
PyArray_Dims newaxes;
- intp dims[MAX_DIMS];
- int i;
+ npy_intp dims[MAX_DIMS];
+ int j;
newaxes.ptr = dims;
newaxes.len = PyArray_NDIM(ap);
- for (i = 0; i < axis; i++) dims[i] = i;
- for (i = axis; i < PyArray_NDIM(ap) - 1; i++) dims[i] = i + 1;
+ for (j = 0; j < axis; j++) dims[j] = j;
+ for (j = axis; j < PyArray_NDIM(ap) - 1; j++) dims[j] = j + 1;
dims[PyArray_NDIM(ap) - 1] = axis;
op = (PyArrayObject *)PyArray_Transpose(ap, &newaxes);
Py_DECREF(ap);
@@ -120,7 +120,7 @@ PyArray_ArgMax(PyArrayObject *op, int axis, PyArrayObject *out)
NPY_BEGIN_THREADS_DESCR(PyArray_DESCR(ap));
n = PyArray_SIZE(ap)/m;
- rptr = (intp *)PyArray_DATA(rp);
+ rptr = (npy_intp *)PyArray_DATA(rp);
for (ip = PyArray_DATA(ap), i = 0; i < n; i++, ip += elsize*m) {
arg_func(ip, m, rptr, ap);
rptr += 1;
@@ -273,7 +273,7 @@ __New_PyArray_Std(PyArrayObject *self, int axis, int rtype, PyArrayObject *out,
PyArrayObject *arr1 = NULL, *arr2 = NULL, *arrnew = NULL;
PyObject *ret = NULL, *newshape = NULL;
int i, n;
- intp val;
+ npy_intp val;
arrnew = (PyArrayObject *)PyArray_CheckAxis(self, &axis, NPY_ARRAY_ALLOWNA);
if (arrnew == NULL) {
View
2  numpy/core/src/multiarray/common.c
@@ -44,6 +44,7 @@ _array_find_python_scalar_type(PyObject *op)
return NULL;
}
+#if !defined(NPY_PY3K)
static PyArray_Descr *
_use_default_type(PyObject *op)
{
@@ -65,6 +66,7 @@ _use_default_type(PyObject *op)
}
return PyArray_DescrFromType(typenum);
}
+#endif
/*
* Recursively examines the object to determine an appropriate dtype
View
8 numpy/core/src/multiarray/convert.c
@@ -247,8 +247,8 @@ PyArray_ToFile(PyArrayObject *self, FILE *fp, char *sep, char *format)
NPY_NO_EXPORT PyObject *
PyArray_ToString(PyArrayObject *self, NPY_ORDER order)
{
- intp numbytes;
- intp index;
+ npy_intp numbytes;
+ npy_intp i;
char *dptr;
int elsize;
PyObject *ret;
@@ -293,9 +293,9 @@ PyArray_ToString(PyArrayObject *self, NPY_ORDER order)
return NULL;
}
dptr = PyBytes_AS_STRING(ret);
- index = it->size;
+ i = it->size;
elsize = PyArray_DESCR(self)->elsize;
- while (index--) {
+ while (i--) {
memcpy(dptr, it->dataptr, elsize);
dptr += elsize;
PyArray_ITER_NEXT(it);
View
25 numpy/core/src/multiarray/datetime.c
@@ -1929,7 +1929,7 @@ convert_datetime_metadata_to_tuple(PyArray_DatetimeMetaData *meta)
}
PyTuple_SET_ITEM(dt_tuple, 0,
- PyBytes_FromString(_datetime_strings[meta->base]));
+ PyUString_FromString(_datetime_strings[meta->base]));
PyTuple_SET_ITEM(dt_tuple, 1,
PyInt_FromLong(meta->num));
@@ -1948,6 +1948,7 @@ convert_datetime_metadata_tuple_to_datetime_metadata(PyObject *tuple,
char *basestr = NULL;
Py_ssize_t len = 0, tuple_size;
int den = 1;
+ PyObject *unit_str = NULL;
if (!PyTuple_Check(tuple)) {
PyObject_Print(tuple, stderr, 0);
@@ -1965,16 +1966,30 @@ convert_datetime_metadata_tuple_to_datetime_metadata(PyObject *tuple,
return -1;
}
- if (PyBytes_AsStringAndSize(PyTuple_GET_ITEM(tuple, 0),
- &basestr, &len) < 0) {
+ unit_str = PyTuple_GET_ITEM(tuple, 0);
+ Py_INCREF(unit_str);
+ if (PyUnicode_Check(unit_str)) {
+ /* Allow unicode format strings: convert to bytes */
+ PyObject *tmp = PyUnicode_AsASCIIString(unit_str);
+ Py_DECREF(unit_str);
+ if (tmp == NULL) {
+ return -1;
+ }
+ unit_str = tmp;
+ }
+ if (PyBytes_AsStringAndSize(unit_str, &basestr, &len) < 0) {
+ Py_DECREF(unit_str);
return -1;
}
out_meta->base = parse_datetime_unit_from_string(basestr, len, NULL);
if (out_meta->base == -1) {
+ Py_DECREF(unit_str);
return -1;
}
+ Py_DECREF(unit_str);
+
/* Convert the values to longs */
out_meta->num = PyInt_AsLong(PyTuple_GET_ITEM(tuple, 1));
if (out_meta->num == -1 && PyErr_Occurred()) {
@@ -2441,7 +2456,7 @@ convert_pydatetime_to_datetimestruct(PyObject *obj, npy_datetimestruct *out,
* the Python datetime.tzinfo object.
*/
NPY_NO_EXPORT int
-get_tzoffset_from_pytzinfo(PyObject *timezone, npy_datetimestruct *dts)
+get_tzoffset_from_pytzinfo(PyObject *timezone_obj, npy_datetimestruct *dts)
{
PyObject *dt, *loc_dt;
npy_datetimestruct loc_dts;
@@ -2454,7 +2469,7 @@ get_tzoffset_from_pytzinfo(PyObject *timezone, npy_datetimestruct *dts)
}
/* Convert the datetime from UTC to local time */
- loc_dt = PyObject_CallMethod(timezone, "fromutc", "O", dt);
+ loc_dt = PyObject_CallMethod(timezone_obj, "fromutc", "O", dt);
Py_DECREF(dt);
if (loc_dt == NULL) {
return -1;
View
4 numpy/core/src/multiarray/datetime_busdaycal.c
@@ -206,8 +206,8 @@ PyArray_WeekMaskConverter(PyObject *weekmask_in, npy_bool *weekmask)
static int
qsort_datetime_compare(const void *elem1, const void *elem2)
{
- npy_datetime e1 = *(npy_datetime *)elem1;
- npy_datetime e2 = *(npy_datetime *)elem2;
+ npy_datetime e1 = *(const npy_datetime *)elem1;
+ npy_datetime e2 = *(const npy_datetime *)elem2;
return (e1 < e2) ? -1 : (e1 == e2) ? 0 : 1;
}
View
408 numpy/core/src/multiarray/datetime_strings.c
@@ -24,6 +24,241 @@
#include "_datetime.h"
#include "datetime_strings.h"
+/* Platform-specific time_t typedef */
+typedef time_t NPY_TIME_T;
+
+/*
+ * Wraps `localtime` functionality for multiple platforms. This
+ * converts a time value to a time structure in the local timezone.
+ *
+ * Returns 0 on success, -1 on failure.
+ */
+static int
+get_localtime(NPY_TIME_T *ts, struct tm *tms)
+{
+ char *func_name = "<unknown>";
+#if defined(_WIN32)
+ #if defined(_MSC_VER) && (_MSC_VER >= 1400)
+ if (localtime_s(tms, ts) != 0) {
+ func_name = "localtime_s";
+ goto fail;
+ }
+ #elif defined(__GNUC__) && defined(NPY_MINGW_USE_CUSTOM_MSVCR)
+ if (_localtime64_s(tms, ts) != 0) {
+ func_name = "_localtime64_s";
+ goto fail;
+ }
+ #else
+ struct tm *tms_tmp;
+ tms_tmp = localtime(ts);
+ if (tms_tmp == NULL) {
+ func_name = "localtime";
+ goto fail;
+ }
+ memcpy(tms, tms_tmp, sizeof(struct tm));
+ #endif
+#else
+ if (localtime_r(ts, tms) == NULL) {
+ func_name = "localtime_r";
+ goto fail;
+ }
+#endif
+
+ return 0;
+
+fail:
+ PyErr_Format(PyExc_OSError, "Failed to use '%s' to convert "
+ "to a local time", func_name);
+ return -1;
+}
+
+/*
+ * Wraps `gmtime` functionality for multiple platforms. This
+ * converts a time value to a time structure in UTC.
+ *
+ * Returns 0 on success, -1 on failure.
+ */
+static int
+get_gmtime(NPY_TIME_T *ts, struct tm *tms)
+{
+ char *func_name = "<unknown>";
+#if defined(_WIN32)
+ #if defined(_MSC_VER) && (_MSC_VER >= 1400)
+ if (gmtime_s(tms, ts) != 0) {
+ func_name = "gmtime_s";
+ goto fail;
+ }
+ #elif defined(__GNUC__) && defined(NPY_MINGW_USE_CUSTOM_MSVCR)
+ if (_gmtime64_s(tms, ts) != 0) {
+ func_name = "_gmtime64_s";
+ goto fail;
+ }
+ #else
+ struct tm *tms_tmp;
+ tms_tmp = gmtime(ts);
+ if (tms_tmp == NULL) {
+ func_name = "gmtime";
+ goto fail;
+ }
+ memcpy(tms, tms_tmp, sizeof(struct tm));
+ #endif
+#else
+ if (gmtime_r(ts, tms) == NULL) {
+ func_name = "gmtime_r";
+ goto fail;
+ }
+#endif
+
+ return 0;
+
+fail:
+ PyErr_Format(PyExc_OSError, "Failed to use '%s' to convert "
+ "to a UTC time", func_name);
+ return -1;
+}
+
+/*
+ * Converts a datetimestruct in UTC to a datetimestruct in local time,
+ * also returning the timezone offset applied.
+ *
+ * Returns 0 on success, -1 on failure.
+ */
+static int
+convert_datetimestruct_utc_to_local(npy_datetimestruct *out_dts_local,
+ const npy_datetimestruct *dts_utc, int *out_timezone_offset)
+{
+ NPY_TIME_T rawtime = 0, localrawtime;
+ struct tm tm_;
+ npy_int64 year_correction = 0;
+
+ /* Make a copy of the input 'dts' to modify */
+ *out_dts_local = *dts_utc;
+
+ /* HACK: Use a year < 2038 for later years for small time_t */
+ if (sizeof(NPY_TIME_T) == 4 && out_dts_local->year >= 2038) {
+ if (is_leapyear(out_dts_local->year)) {
+ /* 2036 is a leap year */
+ year_correction = out_dts_local->year - 2036;
+ out_dts_local->year -= year_correction;
+ }
+ else {
+ /* 2037 is not a leap year */
+ year_correction = out_dts_local->year - 2037;
+ out_dts_local->year -= year_correction;
+ }
+ }
+
+ /*
+ * Convert everything in 'dts' to a time_t, to minutes precision.
+ * This is POSIX time, which skips leap-seconds, but because
+ * we drop the seconds value from the npy_datetimestruct, everything
+ * is ok for this operation.
+ */
+ rawtime = (time_t)get_datetimestruct_days(out_dts_local) * 24 * 60 * 60;
+ rawtime += dts_utc->hour * 60 * 60;
+ rawtime += dts_utc->min * 60;
+
+ /* localtime converts a 'time_t' into a local 'struct tm' */
+ if (get_localtime(&rawtime, &tm_) < 0) {
+ return -1;
+ }
+
+ /* Copy back all the values except seconds */
+ out_dts_local->min = tm_.tm_min;
+ out_dts_local->hour = tm_.tm_hour;
+ out_dts_local->day = tm_.tm_mday;
+ out_dts_local->month = tm_.tm_mon + 1;
+ out_dts_local->year = tm_.tm_year + 1900;
+
+ /* Extract the timezone offset that was applied */
+ rawtime /= 60;
+ localrawtime = (time_t)get_datetimestruct_days(out_dts_local) * 24 * 60;
+ localrawtime += out_dts_local->hour * 60;
+ localrawtime += out_dts_local->min;
+
+ *out_timezone_offset = localrawtime - rawtime;
+
+ /* Reapply the year 2038 year correction HACK */
+ out_dts_local->year += year_correction;
+
+ return 0;
+}
+
+/*
+ * Converts a datetimestruct in local time to a datetimestruct in UTC.
+ *
+ * Returns 0 on success, -1 on failure.
+ */
+static int
+convert_datetimestruct_local_to_utc(npy_datetimestruct *out_dts_utc,
+ const npy_datetimestruct *dts_local)
+{
+ npy_int64 year_correction = 0;
+
+ /* Make a copy of the input 'dts' to modify */
+ *out_dts_utc = *dts_local;
+
+ /* HACK: Use a year < 2038 for later years for small time_t */
+ if (sizeof(NPY_TIME_T) == 4 && out_dts_utc->year >= 2038) {
+ if (is_leapyear(out_dts_utc->year)) {
+ /* 2036 is a leap year */
+ year_correction = out_dts_utc->year - 2036;
+ out_dts_utc->year -= year_correction;
+ }
+ else {
+ /* 2037 is not a leap year */
+ year_correction = out_dts_utc->year - 2037;
+ out_dts_utc->year -= year_correction;
+ }
+ }
+
+ /*
+ * ISO 8601 states to treat date-times without a timezone offset
+ * or 'Z' for UTC as local time. The C standard libary functions
+ * mktime and gmtime allow us to do this conversion.
+ *
+ * Only do this timezone adjustment for recent and future years.
+ * In this case, "recent" is defined to be 1970 and later, because
+ * on MS Windows, mktime raises an error when given an earlier date.
+ */
+ if (out_dts_utc->year >= 1970) {
+ NPY_TIME_T rawtime = 0;
+ struct tm tm_;
+
+ tm_.tm_sec = out_dts_utc->sec;
+ tm_.tm_min = out_dts_utc->min;
+ tm_.tm_hour = out_dts_utc->hour;
+ tm_.tm_mday = out_dts_utc->day;
+ tm_.tm_mon = out_dts_utc->month - 1;
+ tm_.tm_year = out_dts_utc->year - 1900;
+ tm_.tm_isdst = -1;
+
+ /* mktime converts a local 'struct tm' into a time_t */
+ rawtime = mktime(&tm_);
+ if (rawtime == -1) {
+ PyErr_SetString(PyExc_OSError, "Failed to use mktime to "
+ "convert local time to UTC");
+ return -1;
+ }
+
+ /* gmtime converts a 'time_t' into a UTC 'struct tm' */
+ if (get_gmtime(&rawtime, &tm_) < 0) {
+ return -1;
+ }
+ out_dts_utc->sec = tm_.tm_sec;
+ out_dts_utc->min = tm_.tm_min;
+ out_dts_utc->hour = tm_.tm_hour;
+ out_dts_utc->day = tm_.tm_mday;
+ out_dts_utc->month = tm_.tm_mon + 1;
+ out_dts_utc->year = tm_.tm_year + 1900;
+ }
+
+ /* Reapply the year 2038 year correction HACK */
+ out_dts_utc->year += year_correction;
+
+ return 0;
+}
+
/*
* Parses (almost) standard ISO 8601 date strings. The differences are:
*
@@ -127,24 +362,13 @@ parse_iso_8601_datetime(char *str, int len,
tolower(str[2]) == 'd' &&
tolower(str[3]) == 'a' &&
tolower(str[4]) == 'y') {
- time_t rawtime = 0;
+ NPY_TIME_T rawtime = 0;
struct tm tm_;
time(&rawtime);
-#if defined(_WIN32)
- if (localtime_s(&tm_, &rawtime) != 0) {
- PyErr_SetString(PyExc_OSError, "Failed to obtain local time "
- "from localtime_s");
- return -1;
- }
-#else
- /* Other platforms may require something else */
- if (localtime_r(&rawtime, &tm_) == NULL) {
- PyErr_SetString(PyExc_OSError, "Failed obtain local time "
- "from localtime_r");
+ if (get_localtime(&rawtime, &tm_) < 0) {
return -1;
}
-#endif
out->year = tm_.tm_year + 1900;
out->month = tm_.tm_mon + 1;
out->day = tm_.tm_mday;
@@ -182,7 +406,7 @@ parse_iso_8601_datetime(char *str, int len,
if (len == 3 && tolower(str[0]) == 'n' &&
tolower(str[1]) == 'o' &&
tolower(str[2]) == 'w') {
- time_t rawtime = 0;
+ NPY_TIME_T rawtime = 0;
PyArray_DatetimeMetaData meta;
time(&rawtime);
@@ -504,54 +728,8 @@ parse_iso_8601_datetime(char *str, int len,
parse_timezone:
if (sublen == 0) {
- /*
- * ISO 8601 states to treat date-times without a timezone offset
- * or 'Z' for UTC as local time. The C standard libary functions
- * mktime and gmtime allow us to do this conversion.
- *
- * Only do this timezone adjustment for recent and future years.
- */
- if (out->year > 1900 && out->year < 10000) {
- time_t rawtime = 0;
- struct tm tm_;
-
- tm_.tm_sec = out->sec;
- tm_.tm_min = out->min;
- tm_.tm_hour = out->hour;
- tm_.tm_mday = out->day;
- tm_.tm_mon = out->month - 1;
- tm_.tm_year = out->year - 1900;
- tm_.tm_isdst = -1;
-
- /* mktime converts a local 'struct tm' into a time_t */
- rawtime = mktime(&tm_);
- if (rawtime == -1) {
- PyErr_SetString(PyExc_OSError, "Failed to use mktime to "
- "convert local time to UTC");
- goto error;
- }
-
- /* gmtime converts a 'time_t' into a UTC 'struct tm' */
-#if defined(_WIN32)
- if (gmtime_s(&tm_, &rawtime) != 0) {
- PyErr_SetString(PyExc_OSError, "Failed to use gmtime_s to "
- "get a UTC time");
- goto error;
- }
-#else
- /* Other platforms may require something else */
- if (gmtime_r(&rawtime, &tm_) == NULL) {
- PyErr_SetString(PyExc_OSError, "Failed to use gmtime_r to "
- "get a UTC time");
- goto error;
- }
-#endif
- out->sec = tm_.tm_sec;
- out->min = tm_.tm_min;
- out->hour = tm_.tm_hour;
- out->day = tm_.tm_mday;
- out->month = tm_.tm_mon + 1;
- out->year = tm_.tm_year + 1900;
+ if (convert_datetimestruct_local_to_utc(out, out) < 0) {
+ goto error;
}
/* Since neither "Z" nor a time-zone was specified, it's local */
@@ -870,51 +1048,10 @@ make_iso_8601_datetime(npy_datetimestruct *dts, char *outstr, int outlen,
/* Use the C API to convert from UTC to local time */
if (local && tzoffset == -1) {
- time_t rawtime = 0, localrawtime;
- struct tm tm_;
-
- /*
- * Convert everything in 'dts' to a time_t, to minutes precision.
- * This is POSIX time, which skips leap-seconds, but because
- * we drop the seconds value from the npy_datetimestruct, everything
- * is ok for this operation.
- */
- rawtime = (time_t)get_datetimestruct_days(dts) * 24 * 60 * 60;
- rawtime += dts->hour * 60 * 60;
- rawtime += dts->min * 60;
-
- /* localtime converts a 'time_t' into a local 'struct tm' */
-#if defined(_WIN32)
- if (localtime_s(&tm_, &rawtime) != 0) {
- PyErr_SetString(PyExc_OSError, "Failed to use localtime_s to "
- "get a local time");
+ if (convert_datetimestruct_utc_to_local(&dts_local, dts,
+ &timezone_offset) < 0) {
return -1;
}
-#else
- /* Other platforms may require something else */
- if (localtime_r(&rawtime, &tm_) == NULL) {
- PyErr_SetString(PyExc_OSError, "Failed to use localtime_r to "
- "get a local time");
- return -1;
- }
-#endif
- /* Make a copy of the npy_datetimestruct we can modify */
- dts_local = *dts;
-
- /* Copy back all the values except seconds */
- dts_local.min = tm_.tm_min;
- dts_local.hour = tm_.tm_hour;
- dts_local.day = tm_.tm_mday;
- dts_local.month = tm_.tm_mon + 1;
- dts_local.year = tm_.tm_year + 1900;
-
- /* Extract the timezone offset that was applied */
- rawtime /= 60;
- localrawtime = (time_t)get_datetimestruct_days(&dts_local) * 24 * 60;
- localrawtime += dts_local.hour * 60;
- localrawtime += dts_local.min;
-
- timezone_offset = localrawtime - rawtime;
/* Set dts to point to our local time instead of the UTC time */
dts = &dts_local;
@@ -1292,7 +1429,7 @@ NPY_NO_EXPORT PyObject *
array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
PyObject *kwds)
{
- PyObject *arr_in = NULL, *unit_in = NULL, *timezone = NULL;
+ PyObject *arr_in = NULL, *unit_in = NULL, *timezone_obj = NULL;
NPY_DATETIMEUNIT unit;
NPY_CASTING casting = NPY_SAME_KIND_CASTING;
@@ -1313,13 +1450,13 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
"O|OOO&:datetime_as_string", kwlist,
&arr_in,
&unit_in,
- &timezone,
+ &timezone_obj,
&PyArray_CastingConverter, &casting)) {
return NULL;
}
/* Claim a reference to timezone for later */
- Py_XINCREF(timezone);
+ Py_XINCREF(timezone_obj);
op[0] = (PyArrayObject *)PyArray_FromAny(arr_in,
NULL, 0, 0, 0, NULL);
@@ -1385,37 +1522,37 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
}
/* Get the input time zone */
- if (timezone != NULL) {
+ if (timezone_obj != NULL) {
/* Convert to ASCII if it's unicode */
- if (PyUnicode_Check(timezone)) {
+ if (PyUnicode_Check(timezone_obj)) {
/* accept unicode input */
PyObject *obj_str;
- obj_str = PyUnicode_AsASCIIString(timezone);
+ obj_str = PyUnicode_AsASCIIString(timezone_obj);
if (obj_str == NULL) {
goto fail;
}
- Py_DECREF(timezone);
- timezone = obj_str;
+ Py_DECREF(timezone_obj);
+ timezone_obj = obj_str;
}
/* Check for the supported string inputs */
- if (PyBytes_Check(timezone)) {
+ if (PyBytes_Check(timezone_obj)) {
char *str;
Py_ssize_t len;
- if (PyBytes_AsStringAndSize(timezone, &str, &len) < 0) {
+ if (PyBytes_AsStringAndSize(timezone_obj, &str, &len) < 0) {
goto fail;
}
if (strcmp(str, "local") == 0) {
local = 1;
- Py_DECREF(timezone);
- timezone = NULL;
+ Py_DECREF(timezone_obj);
+ timezone_obj = NULL;
}
else if (strcmp(str, "UTC") == 0) {
local = 0;
- Py_DECREF(timezone);
- timezone = NULL;
+ Py_DECREF(timezone_obj);
+ timezone_obj = NULL;
}
else {
PyErr_Format(PyExc_ValueError, "Unsupported timezone "
@@ -1429,12 +1566,31 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
}
}
- /* Create the output string data type with a big enough length */
+ /* Get a string size long enough for any datetimes we're given */
+ strsize = get_datetime_iso_8601_strlen(local, unit);
+#if defined(NPY_PY3K)
@cournape Owner

Would it be possible to factor this out, to avoid defines inside function bodies ? I.e.

#ifdef NPY_PY3K
void foo()
{
// some code here
}
#endif

rather than

some_large_function(...)
{
....
#ifdef NPY_PY3K
// some code here
#endif
}

I know this is not strictly followed in numpy codebase, but it would be nice to avoid too much of this IMO.

@mwiebe Owner
mwiebe added a note

I've thought a bit about how to do this, but splitting this part out feels very artificial and doesn't seem like it would improve the clarity of the code at all. I'd rather keep it inline as it is. It's an artifact of the design choice to make NPY_STRING arrays not return strings in Python 3, something that's probably too late to change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ /*
+ * For Python3, allocate the output array as a UNICODE array, so
+ * that it will behave as strings properly
+ */
+ op_dtypes[1] = PyArray_DescrNewFromType(NPY_UNICODE);
+ if (op_dtypes[1] == NULL) {
+ goto fail;
+ }
+ op_dtypes[1]->elsize = strsize * 4;
+ /* This steals the UNICODE dtype reference in op_dtypes[1] */
+ op[1] = (PyArrayObject *)PyArray_NewLikeArray(op[0],
+ NPY_KEEPORDER, op_dtypes[1], 1);
+ if (op[1] == NULL) {
+ op_dtypes[1] = NULL;
+ goto fail;
+ }
+#endif
+ /* Create the iteration string data type (always ASCII string) */
op_dtypes[1] = PyArray_DescrNewFromType(NPY_STRING);
if (op_dtypes[1] == NULL) {
goto fail;
}
- strsize = get_datetime_iso_8601_strlen(local, unit);
op_dtypes[1]->elsize = strsize;
flags = NPY_ITER_ZEROSIZE_OK|
@@ -1444,7 +1600,7 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
op_flags[1] = NPY_ITER_WRITEONLY|
NPY_ITER_ALLOCATE;
- iter = NpyIter_MultiNew(2, op, flags, NPY_KEEPORDER, NPY_NO_CASTING,
+ iter = NpyIter_MultiNew(2, op, flags, NPY_KEEPORDER, NPY_UNSAFE_CASTING,
op_flags, op_dtypes);
if (iter == NULL) {
goto fail;
@@ -1474,8 +1630,8 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
}
/* Get the tzoffset from the timezone if provided */
- if (local && timezone != NULL) {
- tzoffset = get_tzoffset_from_pytzinfo(timezone, &dts);
+ if (local && timezone_obj != NULL) {
+ tzoffset = get_tzoffset_from_pytzinfo(timezone_obj, &dts);
if (tzoffset == -1) {
goto fail;
}
@@ -1494,7 +1650,7 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
ret = NpyIter_GetOperandArray(iter)[1];
Py_INCREF(ret);
- Py_XDECREF(timezone);
+ Py_XDECREF(timezone_obj);
Py_XDECREF(op[0]);
Py_XDECREF(op[1]);
Py_XDECREF(op_dtypes[0]);
@@ -1506,7 +1662,7 @@ array_datetime_as_string(PyObject *NPY_UNUSED(self), PyObject *args,
return PyArray_Return(ret);
fail:
- Py_XDECREF(timezone);
+ Py_XDECREF(timezone_obj);
Py_XDECREF(op[0]);
Py_XDECREF(op[1]);
Py_XDECREF(op_dtypes[0]);
View
28 numpy/core/src/multiarray/getset.c
@@ -406,7 +406,7 @@ array_descr_set(PyArrayObject *self, PyObject *arg)
{
PyArray_Descr *newtype = NULL;
intp newdim;
- int index;
+ int i;
char *msg = "new type not compatible with array.";
if (!(PyArray_DescrConverter(arg, &newtype)) ||
@@ -450,10 +450,10 @@ array_descr_set(PyArrayObject *self, PyObject *arg)
goto fail;
}
if (PyArray_ISCONTIGUOUS(self)) {
- index = PyArray_NDIM(self) - 1;
+ i = PyArray_NDIM(self) - 1;
}
else {
- index = 0;
+ i = 0;
}
if (newtype->elsize < PyArray_DESCR(self)->elsize) {
/*
@@ -464,20 +464,20 @@ array_descr_set(PyArrayObject *self, PyObject *arg)
goto fail;
}
newdim = PyArray_DESCR(self)->elsize / newtype->elsize;
- PyArray_DIMS(self)[index] *= newdim;
- PyArray_STRIDES(self)[index] = newtype->elsize;
+ PyArray_DIMS(self)[i] *= newdim;
+ PyArray_STRIDES(self)[i] = newtype->elsize;
}
else if (newtype->elsize > PyArray_DESCR(self)->elsize) {
/*
* Determine if last (or first if NPY_ARRAY_F_CONTIGUOUS) dimension
* is compatible
*/
- newdim = PyArray_DIMS(self)[index] * PyArray_DESCR(self)->elsize;
+ newdim = PyArray_DIMS(self)[i] * PyArray_DESCR(self)->elsize;
if ((newdim % newtype->elsize) != 0) {
goto fail;
}
- PyArray_DIMS(self)[index] = newdim / newtype->elsize;
- PyArray_STRIDES(self)[index] = newtype->elsize;
+ PyArray_DIMS(self)[i] = newdim / newtype->elsize;
+ PyArray_STRIDES(self)[i] = newtype->elsize;
}
/* fall through -- adjust type*/
@@ -673,7 +673,7 @@ array_real_set(PyArrayObject *self, PyObject *val)
{
PyArrayObject *ret;
PyArrayObject *new;
- int rint;
+ int retcode;
if (PyArray_ISCOMPLEX(self)) {
ret = _get_part(self, 0);
@@ -690,10 +690,10 @@ array_real_set(PyArrayObject *self, PyObject *val)
Py_DECREF(ret);
return -1;
}
- rint = PyArray_MoveInto(ret, new);
+ retcode = PyArray_MoveInto(ret, new);
Py_DECREF(ret);
Py_DECREF(new);
- return rint;
+ return retcode;
}
/* For Object arrays we need to get
@@ -735,7 +735,7 @@ array_imag_set(PyArrayObject *self, PyObject *val)
if (PyArray_ISCOMPLEX(self)) {
PyArrayObject *ret;
PyArrayObject *new;
- int rint;
+ int retcode;
ret = _get_part(self, 1);
if (ret == NULL) {
@@ -746,10 +746,10 @@ array_imag_set(PyArrayObject *self, PyObject *val)
Py_DECREF(ret);
return -1;
}
- rint = PyArray_MoveInto(ret, new);
+ retcode = PyArray_MoveInto(ret, new);
Py_DECREF(ret);
Py_DECREF(new);
- return rint;
+ return retcode;
}
else {
PyErr_SetString(PyExc_TypeError, "array does not have "\
View
3  numpy/core/src/multiarray/item_selection.c
@@ -1782,7 +1782,8 @@ PyArray_Diagonal(PyArrayObject *self, int offset, int axis1, int axis2)
/* Take a view of the mask if it exists */
if (self_has_maskna) {
PyArrayObject_fields *fret = (PyArrayObject_fields *)ret;
- npy_intp *maskna_strides = PyArray_MASKNA_STRIDES(self);
+
+ maskna_strides = PyArray_MASKNA_STRIDES(self);
fret->maskna_dtype = PyArray_MASKNA_DTYPE(self);
Py_INCREF(fret->maskna_dtype);
View
52 numpy/core/src/multiarray/iterators.c
@@ -34,20 +34,20 @@ NPY_NO_EXPORT npy_intp
parse_index_entry(PyObject *op, npy_intp *step_size,
npy_intp *n_steps, npy_intp max)
{
- npy_intp index;
+ npy_intp i;
if (op == Py_None) {
*n_steps = NEWAXIS_INDEX;
- index = 0;
+ i = 0;
}
else if (op == Py_Ellipsis) {
*n_steps = ELLIPSIS_INDEX;
- index = 0;
+ i = 0;
}
else if (PySlice_Check(op)) {
npy_intp stop;
if (slice_GetIndices((PySliceObject *)op, max,
- &index, &stop, step_size, n_steps) < 0) {
+ &i, &stop, step_size, n_steps) < 0) {
if (!PyErr_Occurred()) {
PyErr_SetString(PyExc_IndexError,
"invalid slice");
@@ -57,11 +57,11 @@ parse_index_entry(PyObject *op, npy_intp *step_size,
if (*n_steps <= 0) {
*n_steps = 0;
*step_size = 1;
- index = 0;
+ i = 0;
}
}
else {
- if (!slice_coerce_index(op, &index)) {
+ if (!slice_coerce_index(op, &i)) {
PyErr_SetString(PyExc_IndexError,
"each index entry must be either a "
"slice, an integer, Ellipsis, or "
@@ -70,15 +70,15 @@ parse_index_entry(PyObject *op, npy_intp *step_size,
}
*n_steps = SINGLE_INDEX;
*step_size = 0;
- if (index < 0) {
- index += max;
+ if (i < 0) {
+ i += max;
}
- if (index >= max || index < 0) {
+ if (i >= max || i < 0) {
PyErr_SetString(PyExc_IndexError, "invalid index");
goto fail;
}
}
- return index;
+ return i;
fail:
return -1;
@@ -645,7 +645,7 @@ iter_length(PyArrayIterObject *self)
static PyArrayObject *
iter_subscript_Bool(PyArrayIterObject *self, PyArrayObject *ind)
{
- npy_intp index, strides;
+ npy_intp counter, strides;
int itemsize;
npy_intp count = 0;
char *dptr, *optr;
@@ -659,8 +659,8 @@ iter_subscript_Bool(PyArrayIterObject *self, PyArrayObject *ind)
"boolean index array should have 1 dimension");
return NULL;
}
- index = PyArray_DIMS(ind)[0];
- if (index > self->size) {
+ counter = PyArray_DIMS(ind)[0];
+ if (counter > self->size) {
PyErr_SetString(PyExc_ValueError,
"too many boolean indices");
return NULL;
@@ -669,7 +669,7 @@ iter_subscript_Bool(PyArrayIterObject *self, PyArrayObject *ind)
strides = PyArray_STRIDES(ind)[0];
dptr = PyArray_DATA(ind);
/* Get size of return array */
- while (index--) {
+ while (counter--) {
if (*((Bool *)dptr) != 0) {
count++;
}
@@ -686,12 +686,12 @@ iter_subscript_Bool(PyArrayIterObject *self, PyArrayObject *ind)
}
/* Set up loop */
optr = PyArray_DATA(ret);
- index = PyArray_DIMS(ind)[0];
+ counter = PyArray_DIMS(ind)[0];
dptr = PyArray_DATA(ind);
copyswap = PyArray_DESCR(self->ao)->f->copyswap;
/* Loop over Boolean array */
swap = (PyArray_ISNOTSWAPPED(self->ao) != PyArray_ISNOTSWAPPED(ret));
- while (index--) {
+ while (counter--) {
if (*((Bool *)dptr) != 0) {
copyswap(optr, self->dataptr, swap, self->ao);
optr += itemsize;
@@ -712,7 +712,7 @@ iter_subscript_int(PyArrayIterObject *self, PyArrayObject *ind)
int itemsize;
int swap;
char *optr;
- npy_intp index;
+ npy_intp counter;
PyArray_CopySwapFunc *copyswap;
itemsize = PyArray_DESCR(self->ao)->elsize;
@@ -754,10 +754,10 @@ iter_subscript_int(PyArrayIterObject *self, PyArrayObject *ind)
Py_DECREF(ret);
return NULL;
}
- index = ind_it->size;
+ counter = ind_it->size;
copyswap = PyArray_DESCR(ret)->f->copyswap;
swap = (PyArray_ISNOTSWAPPED(ret) != PyArray_ISNOTSWAPPED(self->ao));
- while (index--) {
+ while (counter--) {
num = *((npy_intp *)(ind_it->dataptr));
if (num < 0) {
num += self->size;
@@ -941,7 +941,7 @@ static int
iter_ass_sub_Bool(PyArrayIterObject *self, PyArrayObject *ind,
PyArrayIterObject *val, int swap)
{
- npy_intp index, strides;
+ npy_intp counter, strides;
char *dptr;
PyArray_CopySwapFunc *copyswap;
@@ -951,8 +951,8 @@ iter_ass_sub_Bool(PyArrayIterObject *self, PyArrayObject *ind,
return -1;
}
- index = PyArray_DIMS(ind)[0];
- if (index > self->size) {
+ counter = PyArray_DIMS(ind)[0];
+ if (counter > self->size) {
PyErr_SetString(PyExc_ValueError,
"boolean index array has too many values");
return -1;
@@ -963,7 +963,7 @@ iter_ass_sub_Bool(PyArrayIterObject *self, PyArrayObject *ind,
PyArray_ITER_RESET(self);
/* Loop over Boolean array */
copyswap = PyArray_DESCR(self->ao)->f->copyswap;
- while (index--) {
+ while (counter--) {
if (*((Bool *)dptr) != 0) {
copyswap(self->dataptr, val->dataptr, swap, self->ao);
PyArray_ITER_NEXT(val);
@@ -984,7 +984,7 @@ iter_ass_sub_int(PyArrayIterObject *self, PyArrayObject *ind,
{
npy_intp num;
PyArrayIterObject *ind_it;
- npy_intp index;
+ npy_intp counter;
PyArray_CopySwapFunc *copyswap;
copyswap = PyArray_DESCR(self->ao)->f->copyswap;
@@ -998,8 +998,8 @@ iter_ass_sub_int(PyArrayIterObject *self, PyArrayObject *ind,
if (ind_it == NULL) {
return -1;
}
- index = ind_it->size;
- while (index--) {
+ counter = ind_it->size;
+ while (counter--) {
num = *((npy_intp *)(ind_it->dataptr));
if (num < 0) {
num += self->size;
View
98 numpy/core/src/multiarray/mapping.c
@@ -299,7 +299,7 @@ PyArray_GetMap(PyArrayMapIterObject *mit)
PyArrayObject *ret, *temp;
PyArrayIterObject *it;
- int index;
+ int counter;
int swap;
PyArray_CopySwapFunc *copyswap;
@@ -334,11 +334,11 @@ PyArray_GetMap(PyArrayMapIterObject *mit)
Py_DECREF(ret);
return NULL;
}
- index = it->size;
+ counter = it->size;
swap = (PyArray_ISNOTSWAPPED(temp) != PyArray_ISNOTSWAPPED(ret));
copyswap = PyArray_DESCR(ret)->f->copyswap;
PyArray_MapIterReset(mit);
- while (index--) {
+ while (counter--) {
copyswap(it->dataptr, mit->dataptr, swap, ret);
PyArray_MapIterNext(mit);
PyArray_ITER_NEXT(it);
@@ -359,7 +359,7 @@ PyArray_SetMap(PyArrayMapIterObject *mit, PyObject *op)
{
PyArrayObject *arr = NULL;
PyArrayIterObject *it;
- int index;
+ int counter;
int swap;
PyArray_CopySwapFunc *copyswap;
PyArray_Descr *descr;
@@ -394,14 +394,14 @@ PyArray_SetMap(PyArrayMapIterObject *mit, PyObject *op)
return -1;
}
- index = mit->size;
+ counter = mit->size;
swap = (PyArray_ISNOTSWAPPED(mit->ait->ao) !=
(PyArray_ISNOTSWAPPED(arr)));
copyswap = PyArray_DESCR(arr)->f->copyswap;
PyArray_MapIterReset(mit);
/* Need to decref arrays with objects in them */
if (PyDataType_FLAGCHK(descr, NPY_ITEM_HASOBJECT)) {
- while (index--) {
+ while (counter--) {
PyArray_Item_INCREF(it->dataptr, PyArray_DESCR(arr));
PyArray_Item_XDECREF(mit->dataptr, PyArray_DESCR(arr));
memmove(mit->dataptr, it->dataptr, PyArray_ITEMSIZE(arr));
@@ -416,17 +416,19 @@ PyArray_SetMap(PyArrayMapIterObject *mit, PyObject *op)
Py_DECREF(it);
return 0;
}
- while(index--) {
- memmove(mit->dataptr, it->dataptr, PyArray_ITEMSIZE(arr));
- if (swap) {
- copyswap(mit->dataptr, NULL, swap, arr);
+ else {
+ while(counter--) {
+ memmove(mit->dataptr, it->dataptr, PyArray_ITEMSIZE(arr));
+ if (swap) {
+ copyswap(mit->dataptr, NULL, swap, arr);
+ }
+ PyArray_MapIterNext(mit);
+ PyArray_ITER_NEXT(it);
}
- PyArray_MapIterNext(mit);
- PyArray_ITER_NEXT(it);
+ Py_DECREF(arr);
+ Py_DECREF(it);
+ return 0;
}
- Py_DECREF(arr);
- Py_DECREF(it);
- return 0;
}
NPY_NO_EXPORT int
@@ -1416,13 +1418,13 @@ array_subscript(PyArrayObject *self, PyObject *op)
*/
static int
-array_ass_sub_simple(PyArrayObject *self, PyObject *index, PyObject *op)
+array_ass_sub_simple(PyArrayObject *self, PyObject *ind, PyObject *op)
{
int ret;
PyArrayObject *tmp;
npy_intp value;
- value = PyArray_PyIntAsIntp(index);
+ value = PyArray_PyIntAsIntp(ind);
if (!error_converting(value)) {
return array_ass_big_item(self, value, op);
}
@@ -1431,7 +1433,7 @@ array_ass_sub_simple(PyArrayObject *self, PyObject *index, PyObject *op)
/* Rest of standard (view-based) indexing */
if (PyArray_CheckExact(self)) {
- tmp = (PyArrayObject *)array_subscript_simple(self, index);
+ tmp = (PyArrayObject *)array_subscript_simple(self, ind);
if (tmp == NULL) {
return -1;
}
@@ -1444,7 +1446,7 @@ array_ass_sub_simple(PyArrayObject *self, PyObject *index, PyObject *op)
* produces scalars -- those are handled earlier in array_ass_sub
*/
- tmp0 = PyObject_GetItem((PyObject *)self, index);
+ tmp0 = PyObject_GetItem((PyObject *)self, ind);
if (tmp0 == NULL) {
return -1;
}
@@ -1490,7 +1492,7 @@ _tuple_of_integers(PyObject *seq, npy_intp *vals, int maxvals)
static int
-array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
+array_ass_sub(PyArrayObject *self, PyObject *ind, PyObject *op)
{
int ret, oned, fancy;
PyArrayMapIterObject *mit;
@@ -1507,11 +1509,11 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
return -1;
}
- if (PyInt_Check(index) || PyArray_IsScalar(index, Integer) ||
- PyLong_Check(index) || (PyIndex_Check(index) &&
- !PySequence_Check(index))) {
+ if (PyInt_Check(ind) || PyArray_IsScalar(ind, Integer) ||
+ PyLong_Check(ind) || (PyIndex_Check(ind) &&
+ !PySequence_Check(ind))) {
npy_intp value;
- value = PyArray_PyIntAsIntp(index);
+ value = PyArray_PyIntAsIntp(ind);
if (PyErr_Occurred()) {
PyErr_Clear();
}
@@ -1520,11 +1522,11 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
}
}
- if (PyString_Check(index) || PyUnicode_Check(index)) {
+ if (PyString_Check(ind) || PyUnicode_Check(ind)) {
if (PyDataType_HASFIELDS(PyArray_DESCR(self))) {
PyObject *obj;
- obj = PyDict_GetItem(PyArray_DESCR(self)->fields, index);
+ obj = PyDict_GetItem(PyArray_DESCR(self)->fields, ind);
if (obj != NULL) {
PyArray_Descr *descr;
int offset;
@@ -1540,16 +1542,16 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
#if defined(NPY_PY3K)
PyErr_Format(PyExc_ValueError,
"field named %S not found.",
- index);
+ ind);
#else
PyErr_Format(PyExc_ValueError,
"field named %s not found.",
- PyString_AsString(index));
+ PyString_AsString(ind));
#endif
return -1;
}
- if (index == Py_Ellipsis) {
+ if (ind == Py_Ellipsis) {
/*
* Doing "a[...] += 1" triggers assigning an array to itself,
* so this check is needed.
@@ -1571,16 +1573,16 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
* 3) Using newaxis (None)
* 4) Boolean mask indexing
*/
- if (index == Py_None || (PyTuple_Check(index) &&
- (0 == PyTuple_GET_SIZE(index) ||
- count_new_axes_0d(index) > 0))) {
+ if (ind == Py_None || (PyTuple_Check(ind) &&
+ (0 == PyTuple_GET_SIZE(ind) ||
+ count_new_axes_0d(ind) > 0))) {
return PyArray_DESCR(self)->f->setitem(op, PyArray_DATA(self), self);
}
- if (PyBool_Check(index) || PyArray_IsScalar(index, Bool) ||
- (PyArray_Check(index) &&
- (PyArray_DIMS((PyArrayObject *)index)==0) &&
- PyArray_ISBOOL((PyArrayObject *)index))) {
- if (PyObject_IsTrue(index)) {
+ if (PyBool_Check(ind) || PyArray_IsScalar(ind, Bool) ||
+ (PyArray_Check(ind) &&
+ (PyArray_DIMS((PyArrayObject *)ind)==0) &&
+ PyArray_ISBOOL((PyArrayObject *)ind))) {
+ if (PyObject_IsTrue(ind)) {
return PyArray_CopyObject(self, op);
}
else { /* don't do anything */
@@ -1592,9 +1594,9 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
}
/* Integer-tuple */
- if (PyTuple_Check(index) &&
- (PyTuple_GET_SIZE(index) == PyArray_NDIM(self)) &&
- (_tuple_of_integers(index, vals, PyArray_NDIM(self)) >= 0)) {
+ if (PyTuple_Check(ind) &&
+ (PyTuple_GET_SIZE(ind) == PyArray_NDIM(self)) &&
+ (_tuple_of_integers(ind, vals, PyArray_NDIM(self)) >= 0)) {
int idim, ndim = PyArray_NDIM(self);
npy_intp *shape = PyArray_DIMS(self);
npy_intp *strides = PyArray_STRIDES(self);
@@ -1656,9 +1658,9 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
PyErr_Clear();
/* Boolean indexing special case with NA mask support */
- if (PyArray_Check(index) &&
- (PyArray_TYPE((PyArrayObject *)index) == NPY_BOOL) &&
- (PyArray_NDIM(self) == PyArray_NDIM((PyArrayObject *)index))) {
+ if (PyArray_Check(ind) &&
+ (PyArray_TYPE((PyArrayObject *)ind) == NPY_BOOL) &&
+ (PyArray_NDIM(self) == PyArray_NDIM((PyArrayObject *)ind))) {
int retcode;
PyArrayObject *op_arr;
PyArray_Descr *dtype = NULL;
@@ -1676,7 +1678,7 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
if (PyArray_NDIM(op_arr) < 2) {
retcode = array_ass_boolean_subscript(self,
- (PyArrayObject *)index,
+ (PyArrayObject *)ind,
op_arr, NPY_CORDER);
Py_DECREF(op_arr);
return retcode;
@@ -1689,12 +1691,12 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
Py_DECREF(op_arr);
}
- fancy = fancy_indexing_check(index);
+ fancy = fancy_indexing_check(ind);
if (fancy != SOBJ_NOTFANCY) {
oned = ((PyArray_NDIM(self) == 1) &&
- !(PyTuple_Check(index) && PyTuple_GET_SIZE(index) > 1));
- mit = (PyArrayMapIterObject *) PyArray_MapIterNew(index, oned, fancy);
+ !(PyTuple_Check(ind) && PyTuple_GET_SIZE(ind) > 1));
+ mit = (PyArrayMapIterObject *) PyArray_MapIterNew(ind, oned, fancy);
if (mit == NULL) {
return -1;
}
@@ -1718,7 +1720,7 @@ array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op)
return ret;
}
- return array_ass_sub_simple(self, index, op);
+ return array_ass_sub_simple(self, ind, op);
}
View
35 numpy/core/src/multiarray/methods.c
@@ -1885,25 +1885,6 @@ array_transpose(PyArrayObject *self, PyObject *args)
return ret;
}
-/* Return typenumber from dtype2 unless it is NULL, then return
- NPY_DOUBLE if dtype1->type_num is integer or bool
- and dtype1->type_num otherwise.
-*/
-static int
-_get_type_num_double(PyArray_Descr *dtype1, PyArray_Descr *dtype2)
-{
- if (dtype2 != NULL) {
- return dtype2->type_num;
- }
- /* For integer or bool data-types */
- if (dtype1->type_num < NPY_FLOAT) {
- return NPY_DOUBLE;
- }
- else {
- return dtype1->type_num;
- }
-}
-
#define _CHKTYPENUM(typ) ((typ) ? (typ)->type_num : PyArray_NOTYPE)
static PyObject *
@@ -2235,21 +2216,21 @@ static PyObject *
array_setflags(PyArrayObject *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {"write", "align", "uic", NULL};
- PyObject *write = Py_None;
- PyObject *align = Py_None;
+ PyObject *write_flag = Py_None;
+ PyObject *align_flag = Py_None;
PyObject *uic = Py_None;
int flagback = PyArray_FLAGS(self);
PyArrayObject_fields *fa = (PyArrayObject_fields *)self;
if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OOO", kwlist,
- &write,
- &align,
+ &write_flag,
+ &align_flag,
&uic))
return NULL;
- if (align != Py_None) {
- if (PyObject_Not(align)) {
+ if (align_flag != Py_None) {
+ if (PyObject_Not(align_flag)) {
PyArray_CLEARFLAGS(self, NPY_ARRAY_ALIGNED);
}
else if (_IsAligned(self)) {
@@ -2278,8 +2259,8 @@ array_setflags(PyArrayObject *self, PyObject *args, PyObject *kwds)
}
}
- if (write != Py_None) {
- if (PyObject_IsTrue(write)) {
+ if (write_flag != Py_None) {
+ if (PyObject_IsTrue(write_flag)) {
if (_IsWriteable(self)) {
PyArray_ENABLEFLAGS(self, NPY_ARRAY_WRITEABLE);
}
View
4 numpy/core/src/multiarray/nditer_pywrap.c
@@ -2349,7 +2349,7 @@ npyiter_subscript(NewNpyArrayIterObject *self, PyObject *op)
}
else if (PySlice_Check(op)) {
Py_ssize_t istart = 0, iend = 0, istep = 0;
- if (PySlice_GetIndices((PySliceObject *)op,
+ if (PySlice_GetIndices(op,
NpyIter_GetNOp(self->iter),
&istart, &iend, &istep) < 0) {
return NULL;
@@ -2394,7 +2394,7 @@ npyiter_ass_subscript(NewNpyArrayIterObject *self, PyObject *op,
}
else if (PySlice_Check(op)) {
Py_ssize_t istart = 0, iend = 0, istep = 0;
- if (PySlice_GetIndices((PySliceObject *)op,
+ if (PySlice_GetIndices(op,
NpyIter_GetNOp(self->iter),
&istart, &iend, &istep) < 0) {
return -1;
View
42 numpy/core/src/multiarray/number.c
@@ -262,11 +262,13 @@ array_multiply(PyArrayObject *m1, PyObject *m2)
return PyArray_GenericBinaryFunction(m1, m2, n_ops.multiply);
}
+#if !defined(NPY_PY3K)
static PyObject *
array_divide(PyArrayObject *m1, PyObject *m2)
{
return PyArray_GenericBinaryFunction(m1, m2, n_ops.divide);
}
+#endif
static PyObject *
array_remainder(PyArrayObject *m1, PyObject *m2)
@@ -275,17 +277,17 @@ array_remainder(PyArrayObject *m1, PyObject *m2)
}
static int
-array_power_is_scalar(PyObject *o2, double* exp)
+array_power_is_scalar(PyObject *o2, double* out_exponent)
{
PyObject *temp;
const int optimize_fpexps = 1;
if (PyInt_Check(o2)) {
- *exp = (double)PyInt_AsLong(o2);
+ *out_exponent = (double)PyInt_AsLong(o2);
return 1;
}
if (optimize_fpexps && PyFloat_Check(o2)) {
- *exp = PyFloat_AsDouble(o2);
+ *out_exponent = PyFloat_AsDouble(o2);
return 1;
}
if ((PyArray_IsZeroDim(o2) &&
@@ -295,7 +297,7 @@ array_power_is_scalar(PyObject *o2, double* exp)
(optimize_fpexps && PyArray_IsScalar(o2, Floating))) {
temp = Py_TYPE(o2)->tp_as_number->nb_float(o2);
if (temp != NULL) {
- *exp = PyFloat_AsDouble(o2);
+ *out_exponent = PyFloat_AsDouble(o2);
Py_DECREF(temp);
return 1;
}
@@ -315,7 +317,7 @@ array_power_is_scalar(PyObject *o2, double* exp)
PyErr_Clear();
return 0;
}
- *exp = (double) val;
+ *out_exponent = (double) val;
return 1;
}
#endif
@@ -326,12 +328,12 @@ array_power_is_scalar(PyObject *o2, double* exp)
static PyObject *
fast_scalar_power(PyArrayObject *a1, PyObject *o2, int inplace)
{
- double exp;
+ double exponent;
- if (PyArray_Check(a1) && array_power_is_scalar(o2, &exp)) {
+ if (PyArray_Check(a1) && array_power_is_scalar(o2, &exponent)) {
PyObject *fastop = NULL;
if (PyArray_ISFLOAT(a1) || PyArray_ISCOMPLEX(a1)) {
- if (exp == 1.0) {
+ if (exponent == 1.0) {
/* we have to do this one special, as the
"copy" method of array objects isn't set
up early enough to be added
@@ -344,16 +346,16 @@ fast_scalar_power(PyArrayObject *a1, PyObject *o2, int inplace)
return PyArray_Copy(a1);
}
}
- else if (exp == -1.0) {
+ else if (exponent == -1.0) {
fastop = n_ops.reciprocal;
}
- else if (exp == 0.0) {
+ else if (exponent == 0.0) {
fastop = n_ops._ones_like;
}
- else if (exp == 0.5) {
+ else if (exponent == 0.5) {
fastop = n_ops.sqrt;
}
- else if (exp == 2.0) {
+ else if (exponent == 2.0) {
fastop = n_ops.square;
}
else {
@@ -366,7 +368,7 @@ fast_scalar_power(PyArrayObject *a1, PyObject *o2, int inplace)
return PyArray_GenericUnaryFunction(a1, fastop);
}
}
- else if (exp==2.0) {
+ else if (exponent == 2.0) {
fastop = n_ops.multiply;
if (inplace) {
return PyArray_GenericInplaceBinaryFunction
@@ -460,11 +462,13 @@ array_inplace_multiply(PyArrayObject *m1, PyObject *m2)
return PyArray_GenericInplaceBinaryFunction(m1, m2, n_ops.multiply);
}
+#if !defined(NPY_PY3K)
static PyObject *
array_inplace_divide(PyArrayObject *m1, PyObject *m2)
{
return PyArray_GenericInplaceBinaryFunction(m1, m2, n_ops.divide);
}
+#endif
static PyObject *
array_inplace_remainder(PyArrayObject *m1, PyObject *m2)
@@ -798,8 +802,7 @@ NPY_NO_EXPORT PyNumberMethods array_as_number = {
(binaryfunc)array_add, /*nb_add*/
(binaryfunc)array_subtract, /*nb_subtract*/
(binaryfunc)array_multiply, /*nb_multiply*/
-#if defined(NPY_PY3K)
-#else
+#if !defined(NPY_PY3K)
(binaryfunc)array_divide, /*nb_divide*/
#endif
(binaryfunc)array_remainder, /*nb_remainder*/
@@ -815,8 +818,7 @@ NPY_NO_EXPORT PyNumberMethods array_as_number = {
(binaryfunc)array_bitwise_and, /*nb_and*/
(binaryfunc)array_bitwise_xor, /*nb_xor*/
(binaryfunc)array_bitwise_or, /*nb_or*/
-#if defined(NPY_PY3K)
-#else
+#if !defined(NPY_PY3K)
0, /*nb_coerce*/
#endif
(unaryfunc)array_int, /*nb_int*/
@@ -826,8 +828,7 @@ NPY_NO_EXPORT PyNumberMethods array_as_number = {
(unaryfunc)array_long, /*nb_long*/
#endif
(unaryfunc)array_float, /*nb_float*/
-#if defined(NPY_PY3K)
-#else
+#if !defined(NPY_PY3K)
(unaryfunc)array_oct, /*nb_oct*/
(unaryfunc)array_hex, /*nb_hex*/
#endif
@@ -839,8 +840,7 @@ NPY_NO_EXPORT PyNumberMethods array_as_number = {
(binaryfunc)array_inplace_add, /*inplace_add*/
(binaryfunc)array_inplace_subtract, /*inplace_subtract*/
(binaryfunc)array_inplace_multiply, /*inplace_multiply*/
-#if defined(NPY_PY3K)
-#else
+#if !defined(NPY_PY3K)
(binaryfunc)array_inplace_divide, /*inplace_divide*/
#endif
(binaryfunc)array_inplace_remainder, /*inplace_remainder*/
View
22 numpy/core/src/multiarray/scalartypes.c.src
@@ -681,9 +681,13 @@ timedeltatype_repr(PyObject *self)
ret = PyUString_FromString("numpy.timedelta64('NaT'");
}
else {
- /* Can't use "%lld" in Python < 2.7, or if HAVE_LONG_LONG is not
- * defined */
-#if defined(HAVE_LONG_LONG) && (PY_VERSION_HEX >= 0x02070000)
+ /*
+ * Can't use "%lld" in Python < 2.7, Python3 < 3.2,
+ * or if HAVE_LONG_LONG is not defined
+ */
+#if defined(HAVE_LONG_LONG) && \
+ ((PY_VERSION_HEX >= 0x02070000 && PY_VERSION_HEX < 0x03000000) || \
+ (PY_VERSION_HEX >= 0x03020000))
ret = PyUString_FromFormat("numpy.timedelta64(%lld",
(long long)scal->obval);
#else
@@ -791,9 +795,13 @@ timedeltatype_str(PyObject *self)
ret = PyUString_FromString("NaT");
}
else {
- /* Can't use "%lld" in Python < 2.7, or if HAVE_LONG_LONG is not
- * defined */
-#if defined(HAVE_LONG_LONG) && (PY_VERSION_HEX >= 0x02070000)
+ /*
+ * Can't use "%lld" in Python < 2.7, Python3 < 3.2,
+ * or if HAVE_LONG_LONG is not defined
+ */
+#if defined(HAVE_LONG_LONG) && \
+ ((PY_VERSION_HEX >= 0x02070000 && PY_VERSION_HEX < 0x03000000) || \
+ (PY_VERSION_HEX >= 0x03020000))
ret = PyUString_FromFormat("%lld ",
(long long)(scal->obval * scal->obmeta.num));
#else
@@ -2247,6 +2255,7 @@ gentype_getreadbuf(PyObject *self, Py_ssize_t segment, void **ptrptr)
return numbytes;
}
+#if !defined(NPY_PY3K)
static Py_ssize_t
gentype_getsegcount(PyObject *self, Py_ssize_t *lenp)
{
@@ -2279,6 +2288,7 @@ gentype_getcharbuf(PyObject *self, Py_ssize_t segment, constchar **ptrptr)
return -1;
}
}
+#endif /* !defined(NPY_PY3K) */
#if PY_VERSION_HEX >= 0x02060000
View
6 numpy/core/src/multiarray/sequence.c
@@ -190,7 +190,7 @@ NPY_NO_EXPORT PySequenceMethods array_as_sequence = {
static int
array_any_nonzero(PyArrayObject *arr)
{
- intp index;
+ npy_intp counter;
PyArrayIterObject *it;
Bool anyTRUE = FALSE;
@@ -198,8 +198,8 @@ array_any_nonzero(PyArrayObject *arr)
if (it == NULL) {
return anyTRUE;
}
- index = it->size;
- while(index--) {
+ counter = it->size;
+ while (counter--) {
if (PyArray_DESCR(arr)->f->nonzero(it->dataptr, arr)) {
anyTRUE = TRUE;
break;
View
26 numpy/core/src/multiarray/shape.c
@@ -922,8 +922,8 @@ PyArray_Transpose(PyArrayObject *ap, PyArray_Dims *permute)
*/
int _npy_stride_sort_item_comparator(const void *a, const void *b)
{
- npy_intp astride = ((npy_stride_sort_item *)a)->stride,
- bstride = ((npy_stride_sort_item *)b)->stride;
+ npy_intp astride = ((const npy_stride_sort_item *)a)->stride,
+ bstride = ((const npy_stride_sort_item *)b)->stride;
/* Sort the absolute value of the strides */
if (astride < 0) {
@@ -938,8 +938,8 @@ int _npy_stride_sort_item_comparator(const void *a, const void *b)
* Make the qsort stable by next comparing the perm order.
* (Note that two perm entries will never be equal)
*/
- npy_intp aperm = ((npy_stride_sort_item *)a)->perm,
- bperm = ((npy_stride_sort_item *)b)->perm;
+ npy_intp aperm = ((const npy_stride_sort_item *)a)->perm,
+ bperm = ((const npy_stride_sort_item *)b)->perm;
return (aperm < bperm) ? -1 : 1;
}
if (astride > bstride) {
@@ -1001,7 +1001,7 @@ NPY_NO_EXPORT void
PyArray_CreateMultiSortedStridePerm(int narrays, PyArrayObject **arrays,
int ndim, int *out_strideperm)
{
- int i0, i1, ipos, j0, j1, iarrays;
+ int i0, i1, ipos, ax_j0, ax_j1, iarrays;
/* Initialize the strideperm values to the identity. */
for (i0 = 0; i0 < ndim; ++i0) {
@@ -1018,18 +1018,18 @@ PyArray_CreateMultiSortedStridePerm(int narrays, PyArrayObject **arrays,
for (i0 = 1; i0 < ndim; ++i0) {
ipos = i0;
- j0 = out_strideperm[i0];
+ ax_j0 = out_strideperm[i0];
for (i1 = i0 - 1; i1 >= 0; --i1) {
int ambig = 1, shouldswap = 0;
- j1 = out_strideperm[i1];
+ ax_j1 = out_strideperm[i1];
for (iarrays = 0; iarrays < narrays; ++iarrays) {
- if (PyArray_SHAPE(arrays[iarrays])[j0] != 1 &&
- PyArray_SHAPE(arrays[iarrays])[j1] != 1) {
- if (s_intp_abs(PyArray_STRIDES(arrays[iarrays])[j0]) <=
- s_intp_abs(PyArray_STRIDES(arrays[iarrays])[j1])) {
+ if (PyArray_SHAPE(arrays[iarrays])[ax_j0] != 1 &&
+ PyArray_SHAPE(arrays[iarrays])[ax_j1] != 1) {
+ if (s_intp_abs(PyArray_STRIDES(arrays[iarrays])[ax_j0]) <=
+ s_intp_abs(PyArray_STRIDES(arrays[iarrays])[ax_j1])) {
/*
* Set swap even if it's not ambiguous already,
* because in the case of conflicts between
@@ -1070,7 +1070,7 @@ PyArray_CreateMultiSortedStridePerm(int narrays, PyArrayObject **arrays,
for (i1 = i0; i1 > ipos; --i1) {
out_strideperm[i1] = out_strideperm[i1-1];
}
- out_strideperm[ipos] = j0;
+ out_strideperm[ipos] = ax_j0;
}
}
}
@@ -1125,8 +1125,8 @@ PyArray_Ravel(PyArrayObject *arr, NPY_ORDER order)
/* If all the strides matched a contiguous layout, return a view */
if (i < 0) {
PyArrayObject *ret;
- npy_intp stride = strideperm[ndim-1].stride;
+ stride = strideperm[ndim-1].stride;
val[0] = PyArray_SIZE(arr);
Py_INCREF(PyArray_DESCR(arr));
View
14 numpy/core/src/umath/loops.h
@@ -2668,8 +2668,10 @@ TIMEDELTA_mm_d_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS
#define TIMEDELTA_mq_m_true_divide TIMEDELTA_mq_m_divide
#define TIMEDELTA_md_m_true_divide TIMEDELTA_md_m_divide
+#define TIMEDELTA_mm_d_true_divide TIMEDELTA_mm_d_divide
#define TIMEDELTA_mq_m_floor_divide TIMEDELTA_mq_m_divide
#define TIMEDELTA_md_m_floor_divide TIMEDELTA_md_m_divide
+#define TIMEDELTA_mm_d_floor_divide TIMEDELTA_mm_d_divide
#define TIMEDELTA_fmin TIMEDELTA_minimum
#define TIMEDELTA_fmax TIMEDELTA_maximum
#define DATETIME_fmin DATETIME_minimum
@@ -2681,27 +2683,27 @@ TIMEDELTA_mm_d_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS
*****************************************************************************
*/
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_equal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_not_equal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_greater(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_greater_equal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_less(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
-#line 514
+#line 516
NPY_NO_EXPORT void
OBJECT_less_equal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func));
View
2  numpy/core/src/umath/loops.h.src
@@ -494,8 +494,10 @@ TIMEDELTA_mm_d_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS
#define TIMEDELTA_mq_m_true_divide TIMEDELTA_mq_m_divide
#define TIMEDELTA_md_m_true_divide TIMEDELTA_md_m_divide
+#define TIMEDELTA_mm_d_true_divide TIMEDELTA_mm_d_divide
#define TIMEDELTA_mq_m_floor_divide TIMEDELTA_mq_m_divide
#define TIMEDELTA_md_m_floor_divide TIMEDELTA_md_m_divide
+/* #define TIMEDELTA_mm_d_floor_divide TIMEDELTA_mm_d_divide */
#define TIMEDELTA_fmin TIMEDELTA_minimum
#define TIMEDELTA_fmax TIMEDELTA_maximum
#define DATETIME_fmin DATETIME_minimum
View
24 numpy/core/tests/test_datetime.py
@@ -924,11 +924,15 @@ def test_datetime_divide(self):
assert_equal((tda / 0.5).dtype, np.dtype('m8[h]'))
# m8 / m8
assert_equal(tda / tdb, 6.0 / 9.0)
+ assert_equal(np.divide(tda, tdb), 6.0 / 9.0)
+ assert_equal(np.true_divide(tda, tdb), 6.0 / 9.0)
assert_equal(tdb / tda, 9.0 / 6.0)
assert_equal((tda / tdb).dtype, np.dtype('f8'))
assert_equal(tda / tdd, 60.0)
assert_equal(tdd / tda, 1.0 / 60.0)
+ # m8 // m8
+ assert_raises(TypeError, np.floor_divide, tda, tdb)
# int / m8
assert_raises(TypeError, np.divide, 2, tdb)
# float / m8
@@ -1684,11 +1688,29 @@ def test_datetime_is_busday(self):
assert_equal(np.is_busday(holidays, busdaycal=bdd),
np.zeros(len(holidays), dtype='?'))
+ def test_datetime_y2038(self):
+ # Test parsing on either side of the Y2038 boundary
+ a = np.datetime64('2038-01-19T03:14:07Z')
+ assert_equal(a.view(np.int64), 2**31 - 1)
+ a = np.datetime64('2038-01-19T03:14:08Z')
+ assert_equal(a.view(np.int64), 2**31)
+
+ # Test parsing on either side of the Y2038 boundary with
<