Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLite error code not exposed to python #60583

Closed
Bluehorn mannequin opened this issue Nov 1, 2012 · 18 comments
Closed

SQLite error code not exposed to python #60583

Bluehorn mannequin opened this issue Nov 1, 2012 · 18 comments
Labels
3.7 stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@Bluehorn
Copy link
Mannequin

Bluehorn mannequin commented Nov 1, 2012

BPO 16379
Nosy @nedbat, @ezio-melotti, @Bluehorn, @ilevkivskyi, @palaviv, @pablogsal, @miss-islington, @erlend-aasland
PRs
  • bpo-16379: Expose sqlite error code  #1108
  • bpo-16379: expose SQLite error codes and error names in sqlite3 #27786
  • bpo-16379: Fix SQLite version checks in test_module_constants() #28809
  • Files
  • i16379-v1.diff
  • i16379-v2.diff
  • i16379-v3.diff
  • 16379.patch
  • 16379-2.patch
  • 16379-3.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-08-30.19:18:24.765>
    created_at = <Date 2012-11-01.10:14:38.640>
    labels = ['3.7', 'type-feature', 'library']
    title = 'SQLite error code not exposed to python'
    updated_at = <Date 2021-10-07.19:48:25.809>
    user = 'https://github.com/Bluehorn'

    bugs.python.org fields:

    activity = <Date 2021-10-07.19:48:25.809>
    actor = 'miss-islington'
    assignee = 'ghaering'
    closed = True
    closed_date = <Date 2021-08-30.19:18:24.765>
    closer = 'erlendaasland'
    components = ['Library (Lib)']
    creation = <Date 2012-11-01.10:14:38.640>
    creator = 'torsten'
    dependencies = []
    files = ['28470', '28496', '28497', '44213', '44306', '44331']
    hgrepos = []
    issue_num = 16379
    keywords = ['patch']
    message_count = 18.0
    messages = ['174395', '178426', '178429', '178597', '178598', '178601', '245819', '249010', '249044', '257309', '273593', '274064', '359881', '399158', '399667', '400653', '400662', '403440']
    nosy_count = 11.0
    nosy_names = ['ghaering', 'nedbat', 'ezio.melotti', 'torsten', 'sleepycal', 'danielsh', 'levkivskyi', 'palaviv', 'pablogsal', 'miss-islington', 'erlendaasland']
    pr_nums = ['1108', '27786', '28809']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue16379'
    versions = ['Python 3.7']

    @Bluehorn
    Copy link
    Mannequin Author

    Bluehorn mannequin commented Nov 1, 2012

    The sqlite3 module does not expose the sqlite3 error codes to python. This makes it impossible to detect specific error conditions directly.

    Case in point: If a user selects some random file as the database in our application, we can not detect that it is not a valid database file:

    $ /opt/python3/bin/python3
    Python 3.4.0a0 (default:2d6eec5d01f7, Nov  1 2012, 10:47:27) 
    [GCC 4.6.3] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> conn = sqlite3.connect("/etc/passwd")
    >>> from pprint import pprint
    >>> try:
    ...     conn.execute("select * from random_table")
    ... except Exception as e:
    ...     pprint({name: getattr(e, name) for name in dir(e)})
    ...     raise
    ... 
    {'__cause__': None,
     '__class__': <class 'sqlite3.DatabaseError'>,
     '__context__': None,
     '__delattr__': <method-wrapper '__delattr__' of DatabaseError object at 0x7ffc9a13b050>,
     '__dict__': {},
     '__dir__': <built-in method __dir__ of DatabaseError object at 0x7ffc9a13b050>,
     '__doc__': None,
     '__eq__': <method-wrapper '__eq__' of DatabaseError object at 0x7ffc9a13b050>,
     '__format__': <built-in method __format__ of DatabaseError object at 0x7ffc9a13b050>,
     '__ge__': <method-wrapper '__ge__' of DatabaseError object at 0x7ffc9a13b050>,
     '__getattribute__': <method-wrapper '__getattribute__' of DatabaseError object at 0x7ffc9a13b050>,
     '__gt__': <method-wrapper '__gt__' of DatabaseError object at 0x7ffc9a13b050>,
     '__hash__': <method-wrapper '__hash__' of DatabaseError object at 0x7ffc9a13b050>,
     '__init__': <method-wrapper '__init__' of DatabaseError object at 0x7ffc9a13b050>,
     '__le__': <method-wrapper '__le__' of DatabaseError object at 0x7ffc9a13b050>,
     '__lt__': <method-wrapper '__lt__' of DatabaseError object at 0x7ffc9a13b050>,
     '__module__': 'sqlite3',
     '__ne__': <method-wrapper '__ne__' of DatabaseError object at 0x7ffc9a13b050>,
     '__new__': <built-in method __new__ of type object at 0x8267e0>,
     '__reduce__': <built-in method __reduce__ of DatabaseError object at 0x7ffc9a13b050>,
     '__reduce_ex__': <built-in method __reduce_ex__ of DatabaseError object at 0x7ffc9a13b050>,
     '__repr__': <method-wrapper '__repr__' of DatabaseError object at 0x7ffc9a13b050>,
     '__setattr__': <method-wrapper '__setattr__' of DatabaseError object at 0x7ffc9a13b050>,
     '__setstate__': <built-in method __setstate__ of DatabaseError object at 0x7ffc9a13b050>,
     '__sizeof__': <built-in method __sizeof__ of DatabaseError object at 0x7ffc9a13b050>,
     '__str__': <method-wrapper '__str__' of DatabaseError object at 0x7ffc9a13b050>,
     '__subclasshook__': <built-in method __subclasshook__ of type object at 0x1238770>,
     '__suppress_context__': False,
     '__traceback__': <traceback object at 0x7ffc9a138cf8>,
     '__weakref__': None,
     'args': ('file is encrypted or is not a database',),
     'with_traceback': <built-in method with_traceback of DatabaseError object at 0x7ffc9a13b050>}
    Traceback (most recent call last):
      File "<stdin>", line 2, in <module>
    sqlite3.DatabaseError: file is encrypted or is not a database
    >>>

    Currently, one can only match the error message, with is bad programming style.

    The error code for this error is SQLITE_NOTADB, as found in the function sqlite3ErrStr when searching for the message in SQLite's main.c at http://www.sqlite.org/src/artifact/02255cf1da50956c5427c469abddb15bccc4ba09

    Unfortunately, the sqlite3 module does not expose the error code itself (neither the actual error code nor the defined error codes) in any way. Errors are handled in Modules/_sqlite/util.c:

    http://hg.python.org/cpython/file/2d6eec5d01f7/Modules/_sqlite/util.c#l99

    I would like to have the defined error codes available in some mapping inside the sqlite3 module as well as the actual error code inside every sqlite exception e as e.sqlite_errcode

    @Bluehorn Bluehorn mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Nov 1, 2012
    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Dec 28, 2012

    Attached patch that:

    • Adds sqlite3.SQLITE_OK constants
    • Adds sqlite3.errorcode dictionary
    • Adds "sqlite_errcode" attribute to sqlite.Error instances

    The patch does not add support for extended result codes
    (http://www.sqlite.org/c3ref/c_abort_rollback.html).

    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Dec 28, 2012

    I didn't compile-test the Doc/ part of the patch.

    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Dec 30, 2012

    New patch, with better docs and less error leaks, per Ezio's review.

    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Dec 30, 2012

    A couple of random eyebrow-raisers I noticed while working on v2:

    • sqlite3.Warning is a subclass of Exception, rather than sqlite3.Error
      or builtins.Warning. (Also, the docs say "will raise a Warning",
      intending to refer to sqlite3.Warning, but the lack of markup makes
      this ambiguous --- it would be interpreted as a reference to
      builtins.Warning).

    • If _PyUnicode_AsStringAndSize() returns NULL, the code sets a
      sqlite3.Warning exception without first checking what exception is
      already set. (grep for PYSQLITE_SQL_WRONG_TYPE)

    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Dec 30, 2012

    New patch fixing indentation of versionadded markup.

    @sleepycal
    Copy link
    Mannequin

    sleepycal mannequin commented Jun 25, 2015

    Any update on this? Still seems to be a problem as of 3.4.0.

    @ghaering ghaering mannequin self-assigned this Aug 19, 2015
    @ghaering
    Copy link
    Mannequin

    ghaering mannequin commented Aug 23, 2015

    I propose to also set the SQLite extended error code if this is implemented.

    What's the reasoning behind offering a error code to name mapping? This seem problematic to me. In case a newer SQLite version introduces a new error code, this error code cannot be found in the mapping. I propose to leave this out in order to not have this problem.

    Otherwise we will have people depending on any error code being able to be found in this mapping.

    @danielsh
    Copy link
    Mannequin

    danielsh mannequin commented Aug 24, 2015

    What's the reasoning behind offering a error code to name mapping?

    Allowing code that runs into an error to print the error name rather than its
    numeric value. This saves whoever reads the error message having to look it up
    himself.

    his seem problematic to me. In case a newer SQLite version introduces a new
    error code, this error code cannot be found in the mapping. I propose to
    leave this out in order to not have this problem.

    Otherwise we will have people depending on any error code being able to be
    found in this mapping.

    Then people shouldn't depend on the mapping being complete. Let's keep the
    mapping and document that people should only use it as
    sqlite3.errorcode.get(...), never as sqlite3.errorcode[...].

    Or if that's not a good API, we could encapsulate the incompleteness of the
    mapping into a small wrapper function:

       def something(errorcode):
           return sqlite3.errorcode.get(errorcode,
                                        "<sqlite3 error {!d}>".format(errorcode))

    @ezio-melotti
    Copy link
    Member

    ezio-melotti commented Jan 1, 2016

    I think the error message should display both the numeric code and also the error name if available.
    Instead of using a mapping, perhaps an Enum could be used instead.

    @palaviv
    Copy link
    Mannequin

    palaviv mannequin commented Aug 24, 2016

    Attached is a patch based on Daniel last patch with the following changes:

    • There is no errorcode mapping.
    • The exception object has a error_name attribute.

    I think this two changes should solve the API problems raised by Ezio and Gerhard about the error code mapping.

    I propose to also set the SQLite extended error code if this is implemented.

    I think that this should be done in a separate patch. I will start working on the extended error code and will upload a patch to bpo-24139.

    @palaviv
    Copy link
    Mannequin

    palaviv mannequin commented Aug 31, 2016

    Attached is a new patch with the encoding problem fixed.

    @Mariatta Mariatta added the 3.7 label Apr 13, 2017
    @nedbat
    Copy link
    Member

    nedbat commented Jan 12, 2020

    What would it take to get this merged?

    @erlend-aasland
    Copy link
    Contributor

    erlend-aasland commented Aug 6, 2021

    What would it take to get this merged?

    I've rebased Aviv's PR (GH-1108) onto main and resolved the conflicts. If I get his blessing, I'll open a PR try to land this.

    @erlend-aasland
    Copy link
    Contributor

    erlend-aasland commented Aug 16, 2021

    > What would it take to get this merged?

    I've rebased Aviv's PR (GH-1108) onto main and resolved the conflicts.
    If I get his blessing, I'll open a PR try to land this.

    I asked Aviv on GH eight days ago, and I haven't heard anything yet, so I'm going forward with this. For the record, here's a link to my question on the PR:
    #1108 (comment)

    I would also like to see this feature; it makes sqlite3 development slightly more convenient :)

    @pablogsal
    Copy link
    Member

    pablogsal commented Aug 30, 2021

    New changeset 86d8b46 by Erlend Egeberg Aasland in branch 'main':
    bpo-16379: expose SQLite error codes and error names in sqlite3 (GH-27786)
    86d8b46

    @erlend-aasland
    Copy link
    Contributor

    erlend-aasland commented Aug 30, 2021

    Thanks, Torsten for this nice suggestion, Daniel & Aviv for the initial patches, Gerhard & Ezio for helping improving the API, and Pablo, Asif, Hai Shi, & Michael for reviewing and merging!

    @miss-islington
    Copy link
    Contributor

    miss-islington commented Oct 7, 2021

    New changeset 8deb7af by Erlend Egeberg Aasland in branch 'main':
    bpo-16379: Fix SQLite version checks in test_module_constants() (GH-28809)
    8deb7af

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants