Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyodata 1.8.0 - Error if Edm.DateTimeOffset is not set #195

Closed
metaodi opened this issue Jan 26, 2022 · 4 comments
Closed

pyodata 1.8.0 - Error if Edm.DateTimeOffset is not set #195

metaodi opened this issue Jan 26, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@metaodi
Copy link

metaodi commented Jan 26, 2022

This issue is based on an issue I got on a library that uses pyodata (metaodi/swissparlpy#17)

Some calls fail when pyodata tries to parse dates:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in parse_datetime_literal(value)
    388     try:
--> 389         return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S.%f')
    390     except ValueError:

/opt/tljh/user/lib/python3.7/_strptime.py in _strptime_datetime(cls, data_string, format)
    576     format string."""
--> 577     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
    578     tzname, gmtoff = tt[-2:]

/opt/tljh/user/lib/python3.7/_strptime.py in _strptime(data_string, format)
    358         raise ValueError("time data %r does not match format %r" %
--> 359                          (data_string, format))
    360     if len(data_string) != found.end():

ValueError: time data '0000-00-00T00:00:00' does not match format '%Y-%m-%dT%H:%M:%S.%f'

And the trace is the following:

/opt/tljh/user/lib/python3.7/site-packages/swissparlpy/client.py in __init__(self, entity_request, variables)
     59 class SwissParlResponse(object):
     60     def __init__(self, entity_request, variables):
---> 61         self.entities = entity_request.execute()
     62         self.count = self.entities.total_count
     63         self.variables = variables

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in execute(self)
    326             self._logger.debug('  body: <cannot be decoded>')
    327 
--> 328         return self._handler(response)
    329 
    330     def custom(self, name, value):

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in get_entities_handler(response)
   1407             result = ListWithTotalCount(total_count)
   1408             for props in entities:
-> 1409                 entity = EntityProxy(self._service, self._entity_set, self._entity_set.entity_type, props)
   1410                 result.append(entity)
   1411 

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/service.py in __init__(self, service, entity_set, entity_type, proprties, entity_key, etag)
    766                     else:
    767                         # null value is in literal form for now, convert it to python representation
--> 768                         self._cache[type_proprty.name] = type_proprty.from_literal(type_proprty.typ.null_value)
    769 
    770             # then, assign all navigation properties

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in from_literal(self, value)
    837             return None
    838 
--> 839         return self.typ.traits.from_literal(value)
    840 
    841     def to_literal(self, value):

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in from_literal(self, value)
    554                 tz_sign = -1 if match.group('sign') == '-' else 1
    555                 tz_info = datetime.timezone(tz_sign * tz_offset)
--> 556             return parse_datetime_literal(datetime_part).replace(tzinfo=tz_info)
    557         except (ValueError, AttributeError):
    558             raise PyODataModelError(f'Cannot decode datetimeoffset from value {value}.')

/opt/tljh/user/lib/python3.7/site-packages/pyodata/v2/model.py in parse_datetime_literal(value)
    395                 return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M')
    396             except ValueError:
--> 397                 raise PyODataModelError(f'Cannot decode datetime from value {value}.')
    398 
    399 

PyODataModelError: Cannot decode datetime from value 0000-00-00T00:00:00.

I'm not 100% I understand everything that happens, but to me it seems, that in pyodata/v2/service.py (Line 768) the defined null_value is passed to from_literal and the null_value of Edm.DateTimeOffset is set to 'datetimeoffset\'0000-00-00T00:00:00Z\''

The new code introduced in #184 doesn't seem to check for this case.

My current workaround is this to monkey patch this:

def patched_parse_datetime_literal(value):
    print(value)
    if value == '0000-00-00T00:00:00':
        return datetime.datetime(1970, 1, 1)
    try:
        return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S.%f')
    except ValueError:
        try:
            return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M:%S')
        except ValueError:
            try:
                return datetime.datetime.strptime(value, '%Y-%m-%dT%H:%M')
            except ValueError:
                raise PyODataModelError(f'Cannot decode datetime from value {value}.')
pyodata.v2.model.parse_datetime_literal = patched_parse_datetime_literal

But maybe this is not the best place to fix this issue.

cc @rettichschnidi

@metaodi
Copy link
Author

metaodi commented Jan 26, 2022

Here is the server response that leads to this error: Transcript.txt

@rettichschnidi
Copy link
Contributor

rettichschnidi commented Jan 26, 2022

  1. The default value is poorly chosen (invalid month, invalid day). Will create a PR.
  2. Python does not like the year being 0:
$ ipython3 
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import datetime; datetime.datetime(0,1,1)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-4df0df8009df> in <module>
----> 1 import datetime; datetime.datetime(0,1,1)

ValueError: year 0 is out of rang

I do not know how to work around this and whether it would be needed in any case. Need to look up if OData allows specifying dates in year 0 or before.

For now, I will fix this by defaulting to a suitable default year (1970?).

rettichschnidi added a commit to rettichschnidi/python-pyodata that referenced this issue Jan 26, 2022
Default value was broken in several ways:
- Month 0 does not exist
- Day 0 does not exist
- Year 0 can not be handled by Python datetime

The new default value is copied from Edm.DateTime.
@rettichschnidi
Copy link
Contributor

rettichschnidi commented Jan 26, 2022

Need to look up if OData allows specifying dates in year 0 or before.

Did not dig into this, but it seems like we have never been able to deal with values before 01 Jan 0001 anyway. Went on to create PR #197.

@phanak-sap phanak-sap added the bug Something isn't working label Jan 27, 2022
@phanak-sap phanak-sap changed the title Error if Edm.DateTimeOffset is not set pyodata 1.8.0 - Error if Edm.DateTimeOffset is not set Jan 27, 2022
rettichschnidi added a commit to rettichschnidi/python-pyodata that referenced this issue Jan 28, 2022
Default value was broken in several ways:
- Month 0 does not exist
- Day 0 does not exist
- Year 0 can not be handled by Python datetime
rettichschnidi added a commit to rettichschnidi/python-pyodata that referenced this issue Jan 29, 2022
Default value was broken in several ways:
- Month 0 does not exist
- Day 0 does not exist
- Year 0 can not be handled by Python datetime
phanak-sap pushed a commit that referenced this issue Jan 31, 2022
Default value was broken in several ways:
- Month 0 does not exist
- Day 0 does not exist
- Year 0 can not be handled by Python datetime
@phanak-sap
Copy link
Contributor

This should be fixed in #195 and part of release 1.9.0. Please reopen if still reproducible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants