Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Unicode error in audit log #1563

Merged
merged 2 commits into from Apr 8, 2019

Conversation

Projects
None yet
2 participants
@plettich
Copy link
Member

commented Apr 8, 2019

When using mysql with the audit log, there is an issue if one of the
columns of an entry contains non ascii symbols.
To avoid writing invalid data to mysql, the column types in the SQLAlchemy
model need to be set to Unicode.
Some possible regressions need to be caught when updating the code.

fixes #1560

Fix Unicode error in audit log
When using mysql with the audit log, there is an issue if one of the
columns of an entry contains non ascii symbols.
To avoid writing invalid data to mysql, the column types in the SQLAlchemy
model need to be set to `Unicode`.
Some possible regressions need to be caught when updating the code.

fixes #1560

@plettich plettich requested a review from privacyidea/core Apr 8, 2019

@codecov

This comment has been minimized.

Copy link

commented Apr 8, 2019

Codecov Report

Merging #1563 into master will decrease coverage by 0.03%.
The diff coverage is 76.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1563      +/-   ##
==========================================
- Coverage   96.97%   96.94%   -0.04%     
==========================================
  Files         148      148              
  Lines       17680    17687       +7     
==========================================
+ Hits        17146    17147       +1     
- Misses        534      540       +6
Impacted Files Coverage Δ
privacyidea/models.py 98.92% <100%> (ø) ⬆️
privacyidea/lib/auditmodules/sqlaudit.py 94.24% <57.14%> (-3.05%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e871f54...7484253. Read the comment docs.

# audit_entry, we will get issues when packing the response.
log.warning('Could not verify log entry! We get invalid values '
'from the database, please check the encoding.')
log.debug('{0!s'.format(traceback.format_exc()))

This comment has been minimized.

Copy link
@cornelinux

cornelinux Apr 8, 2019

Member

There is a curly closing bracked missing.

This comment has been minimized.

Copy link
@plettich

plettich Apr 8, 2019

Author Member

fixed in 7484253

@@ -405,14 +407,22 @@ def search(self, search_dict, page_size=15, page=1, sortorder="asc",
auditIter = self.search_query(search_dict, page_size=page_size,
page=page, sortorder=sortorder,
timelimit=timelimit)
try:
le = next(auditIter)
while le:

This comment has been minimized.

Copy link
@cornelinux

cornelinux Apr 8, 2019

Member

Why did you change the while le loop to a while True loop?
Would there be an issue with having a "double" loop condition of "while le" and "StopIteration"?

This comment has been minimized.

Copy link
@plettich

plettich Apr 8, 2019

Author Member

My first approach was to catch the error for each individual auditIter entry. Unfortunately the first call to next(auditIter) throws the error and stops the iteration.
And while looking through examples to call next(auditIter) only once, the while True: loop seemed to be the most pythonic way.

@cornelinux cornelinux merged commit 993d38d into master Apr 8, 2019

3 of 5 checks passed

codecov/patch 76.92% of diff hit (target 96.97%)
Details
codecov/project 96.94% (-0.04%) compared to e871f54
Details
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@cornelinux cornelinux deleted the 1560/unicode_error_in_audit_log branch Apr 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.