New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

null value in column "details" when parsing SMART error log #1498

Closed
phillxnet opened this Issue Oct 25, 2016 · 5 comments

Comments

Projects
None yet
2 participants
@phillxnet
Member

phillxnet commented Oct 25, 2016

Thanks to forum member alexey in the following thread for reporting this issue. The SMART error log parser is, under specific hardware conditions, producing unexpected and incompatible output resulting in the following Web-UI error message whenever the "Refresh" button in the smart screen for the specific drive is pressed:

Houston, we've had a problem.

null value in column "details" violates not-null constraint DETAIL: Failing row contains ...

with the associated Traceback having the penultimate line:

IntegrityError: null value in column "details" violates not-null constraint

resulting in no SMART info available via the Web-UI for this drive.

As a result of alexey submitting the requested command outputs from the affected drive via the forum thread the reported issue has been reproduced and narrowed down to the error log parsing component of the pre-processing that is required to display the same in the Web-UI.

Please update the following forum thread with significant progress on this issue:
https://forum.rockstor.com/t/smart-not-working/2137

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Oct 25, 2016

enable SMART test mode and add debug logging for issue #1498
By setting TESTMODE to True all smart parsing sources command
dumps instead of local smartctl command output. This allows for
testing of user provided smartctl command output placed in
/root/smartdumps.
@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Oct 25, 2016

Member

The error seems to come from parsing the output of:

smartctl -l error drive-name

and parsing the following line is where our None (null) value comes from:

04 51 01 00 00 00 a0  Error: ABRT

the expected format of this line is as follows (from a real drive whose error log is parsed correctly):

40 51 00 db f7 03 00  Error: UNC at LBA = 0x0003f7db = 260059

Hence we are missing the "at LBA = 0x0003f7db = 260059" which is normally assigned to the "details" column via error_logs() in system/smart.py:

details = ' '.join(e_fields[1:]) if (len(e_fields) > 1) else None
summary[err_num] = list([lifetime_hours, state, etype, details])
Member

phillxnet commented Oct 25, 2016

The error seems to come from parsing the output of:

smartctl -l error drive-name

and parsing the following line is where our None (null) value comes from:

04 51 01 00 00 00 a0  Error: ABRT

the expected format of this line is as follows (from a real drive whose error log is parsed correctly):

40 51 00 db f7 03 00  Error: UNC at LBA = 0x0003f7db = 260059

Hence we are missing the "at LBA = 0x0003f7db = 260059" which is normally assigned to the "details" column via error_logs() in system/smart.py:

details = ' '.join(e_fields[1:]) if (len(e_fields) > 1) else None
summary[err_num] = list([lifetime_hours, state, etype, details])

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Oct 25, 2016

modify SMART summary table details to cope with no sector info #1498
In the case of some SMART error reports there is no sector information:
ie "04 51 01 00 00 00 a0  Error: ABRT" as opposed to the expected:
"40 51 00 db f7 03 00  Error: UNC at LBA = 0x0003f7db = 260059",
in this instance do not return None which breaks db / UI related parsing
but instead return user relevant info ie "No Sector Details Available".

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Oct 25, 2016

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Oct 25, 2016

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Oct 25, 2016

Member

The following is a screen shot of the resulting error log after the commits associated with this issue are applied to the SMART data as provided by forum member alexey.

error-log-after-fix

Member

phillxnet commented Oct 25, 2016

The following is a screen shot of the resulting error log after the commits associated with this issue are applied to the SMART data as provided by forum member alexey.

error-log-after-fix

@schakrava schakrava added the bug label Nov 12, 2016

@schakrava schakrava added this to the Pinnacles milestone Nov 12, 2016

@schakrava schakrava closed this in #1499 Nov 12, 2016

schakrava added a commit that referenced this issue Nov 12, 2016

Merge pull request #1499 from phillxnet/1498_null_value_in_column_"de…
…tails"_when_parsing_SMART_error_log

null value in column "details" when parsing smart error log. Fixes #1498
@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Nov 13, 2016

Member

Please note that this issue was actually a duplicate of a prior reported #1358. Linking for reference purposes. The duplication status was only noticed after this issue was addressed and fixed.

Member

phillxnet commented Nov 13, 2016

Please note that this issue was actually a duplicate of a prior reported #1358. Linking for reference purposes. The duplication status was only noticed after this issue was addressed and fixed.

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Nov 13, 2016

Member

Apologies to the original issue reporter - forum member kziegler.

Member

phillxnet commented Nov 13, 2016

Apologies to the original issue reporter - forum member kziegler.

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Nov 13, 2016

Member

All forum threads referenced have been updated.

Member

phillxnet commented Nov 13, 2016

All forum threads referenced have been updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment