A proposed SQLite data access layer (DAL)

[mahmoud]

NB: This is a staged pull request, full of bad code (hence the branch name), so that we can see how some Python antipatterns manifest. I'll be playing the part of both the submitter and the reviewer, but the submitter (starting with the message following the italics) will write in quotes.

"I really like the LineDAL's JSONLines approach, and saw there was a stubbed out version of a SQLite implementation, so I am submitting this proposed implementation. I'm new to Python, so please let me know what you think."

[mahmoud]

Wow! Thank you! Unfortunately, as it stands, I see many issues with this code. The interface is correct, but there are many antipatterns in play that would hurt immediate usability, as well as long-term maintainability. Please see my comments inline.

Overall a good effort, though, and if you make the changes in future PRs, your code is sure to be accepted! Thanks again!

[mahmoud]

This appears to be an unused import, and while built-in, can set the wrong expectation for the following code.

[mahmoud]

Another unused import, but this one is more concerning. Because it's a third-party package, this can really harm the reader's impression of the following code's reusability.

You should add PyLint to your editor. PyLint will tell you about unused imports.

[mahmoud]

Do not use name mangling unless absolutely necessary (I have only used it once in 7 years). Name mangling is not private variables in Python, and is really just a workaround for a very specific inheritance corner case.

If you want to communicate to other developers that a variable is for internal use, a single underscore is preferred, e.g., self._file_path. (I wouldn't even say that is a high priority in this case. Mustn't spend all day navelgazing though.)

[mahmoud]

Parallel to the comment above, the global keyword is a big yellow flag. Uses are rare enough in production code that a code comment is warranted. In this case, total_count should be an attribute on self.

[mahmoud]

This is a very unidiomatic way to convert a bool into a string. Just use an inline 'T' if var else 'F' or a semantically meaningful converter function.

[mahmoud]

Unlike Java and C++, class instance members do not need class-level declarations. extension is a true class-level attribute, but this attribute is destined to be overridden in __init__. This line should be removed.

[mahmoud]

This type of code at the module-level results in the infamous "import-time side effect". While the intentions may be good, this unexpected behavior can have serious implications.

It's best to leave functions like this to the modules closer to the main code entry point. In this case, because there may be other future sqlite usages in the project, I would centralize this sort of behavior in app.py.

[mahmoud]

This catch-all except clause is a major antipattern. Ignoring all exceptions is unpythonic. It's ok to pause a second and be specific about which exceptions you want to handle.

This block of code is not best-effort. This method should allow any exceptions raised here to propagate. The calling function can catch exceptions if it wants.

Furthermore, even though it's only four lines, so many different things happen in this try block, that I would also call this try block too large.

[mahmoud]

This is a good change.

NB: This is an affirmative comment. All other comments inline are critiques, unless otherwise specified. Only the sqlite_dal.py file is of material interest. Because there are so many comments, it may be useful to uncheck the "Show notes" option above to do an initial read through. Alternatively, use the "View" button in the top right of the sqlite_dal.py file below.

[mahmoud]

This block of string construction of SQL has many problems:

• It is incorrect. If one of the values includes a quote, the escaping will break, causing a SQL error at best.
• It is insecure. String construction of SQL queries also opens you up to SQL injection.
• It is brittle. One missing key and the whole function goes up in KeyError smoke.
• It is hard to expand. If the schema or message is updated, keeping the orders consistent throughout will be a major issue.

At the very least use the parameters keyword argument of the execute function. Never allow inputs to modify the SQL query strings.

[mahmoud]

Connecting and executing are common among all the methods of this class. Factor out logic like this to a central method.

[mahmoud]

Apart from problems described in other comments, this simple function commits a crime against usability.

In the success case, this function returns a list. In the failure case, it returns an integer.

• Use exceptions for errors.
• Functions should have one return type (or one type + None). An exception to this rule is when the function allows the caller to pass in its own default or processing function.

In the except case, this function should allow the exception to propagate or return an empty list.

[mahmoud]

At over forty lines, this function is too long, especially for the level of other problems it has. Factor out a less brittle approach. Expect more power out of a line of Python.