XSDToPython maps integer to long #217

Closed
salmonito opened this Issue Jun 18, 2012 · 4 comments

Comments

Projects
None yet
4 participants
@salmonito

_XSD_NS[u'integer'] : long, same for int.
Not sure what the rationale was to do that. In my application for example I have integer attribute, I read it as #integer literal, get python value as long, and update to long ending up with two values in my RDF store integer and long.
cheers.

@gjhiggins

This comment has been minimized.

Show comment Hide comment
@gjhiggins

gjhiggins Jun 18, 2012

Owner

Origin is back in the mists of time - 6 years ago, according to github

f05bc81#rdflib/Literal.py

was a chimezie commit - just a typo?

http://code.google.com/p/rdflib/source/browse/rdflib/Literal.py?name=initial_import_from_cvs&r=77b0bdecf766f529303c83746c7f935601e4107f

Owner

gjhiggins commented Jun 18, 2012

Origin is back in the mists of time - 6 years ago, according to github

f05bc81#rdflib/Literal.py

was a chimezie commit - just a typo?

http://code.google.com/p/rdflib/source/browse/rdflib/Literal.py?name=initial_import_from_cvs&r=77b0bdecf766f529303c83746c7f935601e4107f

@joernhees

This comment has been minimized.

Show comment Hide comment
@joernhees

joernhees Jun 19, 2012

Owner

The "problem" of xsd:int being cast to a python long was introduced in 1676f9e . The log msg indicates that there was a problem with equality tests otherwise... maybe eikeon remembers more details, even though it's 5 years ago.

IMO the original chimezie commit was a bit more correct for integer, long and int: f05bc81#L0R44 as it follows the specs more closely: http://www.w3.org/TR/xmlschema-2/#built-in-datatypes .
The http://www.w3.org/TR/xmlschema-2/#integer does not make any assumptions about how big the number is, just that it doesn't have decimals. http://www.w3.org/TR/xmlschema-2/#long and http://www.w3.org/TR/xmlschema-2/#int make assumptions about how big the numbers can be and hence int can be cast to a python int using the additional info that a python int is at least 32 bit --> max 2**31-1 == 2147483647 (http://docs.python.org/library/stdtypes.html#numeric-types-int-float-long-complex).

The original and current version both are inconsistent in various ways, when it comes to xsd:.+Integer .As mentioned above for example the xsd:nonPositiveInteger doesn't make any assumptions about the length of the number. Nevertheless, it currently is interpreted as int in python (which actually isn't too bad, see below):

As python actually deals with int overflows gracefully by using long instead, why not make all the mappings int?

In [1]: int("4242424")
Out[1]: 4242424

In [2]: int("42424242424242424242424242424242")
Out[2]: 42424242424242424242424242424242L

That way python would dynamically decide if the value is small enough to be held in an int or if it needs a long. Only problem would be people relying on int/long to serialize to the correct xsd datatypes (which IMO is a bad idea).

@salmonito :
Maybe as a workaround you can try to downcast the long to an int in python prior to saving it into the database?

Owner

joernhees commented Jun 19, 2012

The "problem" of xsd:int being cast to a python long was introduced in 1676f9e . The log msg indicates that there was a problem with equality tests otherwise... maybe eikeon remembers more details, even though it's 5 years ago.

IMO the original chimezie commit was a bit more correct for integer, long and int: f05bc81#L0R44 as it follows the specs more closely: http://www.w3.org/TR/xmlschema-2/#built-in-datatypes .
The http://www.w3.org/TR/xmlschema-2/#integer does not make any assumptions about how big the number is, just that it doesn't have decimals. http://www.w3.org/TR/xmlschema-2/#long and http://www.w3.org/TR/xmlschema-2/#int make assumptions about how big the numbers can be and hence int can be cast to a python int using the additional info that a python int is at least 32 bit --> max 2**31-1 == 2147483647 (http://docs.python.org/library/stdtypes.html#numeric-types-int-float-long-complex).

The original and current version both are inconsistent in various ways, when it comes to xsd:.+Integer .As mentioned above for example the xsd:nonPositiveInteger doesn't make any assumptions about the length of the number. Nevertheless, it currently is interpreted as int in python (which actually isn't too bad, see below):

As python actually deals with int overflows gracefully by using long instead, why not make all the mappings int?

In [1]: int("4242424")
Out[1]: 4242424

In [2]: int("42424242424242424242424242424242")
Out[2]: 42424242424242424242424242424242L

That way python would dynamically decide if the value is small enough to be held in an int or if it needs a long. Only problem would be people relying on int/long to serialize to the correct xsd datatypes (which IMO is a bad idea).

@salmonito :
Maybe as a workaround you can try to downcast the long to an int in python prior to saving it into the database?

@gjhiggins

This comment has been minimized.

Show comment Hide comment
@gjhiggins

gjhiggins Jun 19, 2012

Owner

++ IMO the original chimezie commit was a bit more correct for integer, long and int

Thanks for the clarification, obviously I'd lost some focus by that stage of my forensics trawl.

Owner

gjhiggins commented Jun 19, 2012

++ IMO the original chimezie commit was a bit more correct for integer, long and int

Thanks for the clarification, obviously I'd lost some focus by that stage of my forensics trawl.

@gromgull

This comment has been minimized.

Show comment Hide comment
@gromgull

gromgull Nov 22, 2012

Owner

In the new literal reworking branch - I've fixed this: 4e6cd68

ints in python transparently promote to longs when the values get to large, and longs have no limit. I find it odd to try to shoe-horn either of them into any particular concrete bit-limited xsd datatype. After discussing with Jörn, both are now mapped to xsd:integer

Owner

gromgull commented Nov 22, 2012

In the new literal reworking branch - I've fixed this: 4e6cd68

ints in python transparently promote to longs when the values get to large, and longs have no limit. I find it odd to try to shoe-horn either of them into any particular concrete bit-limited xsd datatype. After discussing with Jörn, both are now mapped to xsd:integer

@gromgull gromgull closed this Nov 22, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment