sg_inq+sg_vpd: more updates but not finished

The sg_inq+sg_inq work is mainly JSON additions.
sg_vpd has a new --sinq_inraw=RFN option.
Update and place names for the 64 TapeAlert flags
in the library. This improves TapeAlert reporting
for sg_inq, sg_vpd and sg_logs.
Refine the description of the VPD page merge of
processing for sg_inq and sg_vpd to only include _T10_
defined pages, so the vendor specific VPD page
processings of those utilities are still separate.



git-svn-id: https://svn.bingwo.ca/repos/sg3_utils/trunk@969 6180dd3e-e324-4e3e-922d-17de1ae2f315

ChangeLog

@@ -2,7 +2,7 @@ Each utility has its own version number, date of last change and
 some description at the top of its ".c" file. All utilities in the main
 directory have their own "man" pages. There is also a sg3_utils man page.
 
-Changelog for pre-release sg3_utils-1.48 [20220812] [svn: r968]
+Changelog for pre-release sg3_utils-1.48 [20220818] [svn: r969]
   - some utilities: add experimental --json[=JO] option
   - sg_z_act_query: new utility for sending either a
     Zone activate or Zone query command
@@ -47,12 +47,15 @@ Changelog for pre-release sg3_utils-1.48 [20220812] [svn: r968]
       the left hand side of each line of hex
     - improve 'last_n' log pages; supply VPD and mode pages
       with their name (if T10 defined)
+    - update names for TapeAlert lpage
   - sg_modes: improve handling of zbc disks with pdt=0x14
-  - sg_inq, sg_vpd: merge VPD page processing
+  - sg_inq, sg_vpd: merge VPD page processing for T10
+    defined pages, VS pages still differ
     - Device Identication VPD page, change
       "IEEE Company_id" to "AOI" as per spc6r06.pdf
     - add support for Hitachi/HP open-v ldev names
   - sg_vpd: apply github pull 18 (missing LF)
+    - add --sinq_inraw=RFN option
   - sg_opcodes: cleanup error reporting
     - add --inhex=FN to process earlier -HHH
   - sg_format: allow disk formats on ZBC (zoned) disks

README.details

@@ -170,7 +170,13 @@ favours POSIX system and library calls over OS specific calls.
 
 The C code is written in a C++ friendly way and is checked from time to
 time that it compiles clean with C++. To accommodate C++ certain C99
-constructs such as designated initializers cannot be used.
+constructs such as designated initializers cannot be used. To build
+with C++, C++11 (i.e. the C++ standard from 2011) or later is required.
+Finding a common C and C++ syntax for zeroing stack variables (including
+aggregates) may need to wait until C23 allows this syntax:
+   struct example_t ex1 {};
+which C++ introduced in C++11.  In the meantime the SG_C_CPP_ZERO_INIT
+define (hack) does this.
 
 The author has not seriously attempted to build this code on MSVC (aka
 Visual Studio). There are a few roadblocks (that may be overcome in the
@@ -542,4 +548,4 @@ See https://sg.danny.cz/sg/tools.html
 
 
 Douglas Gilbert		dgilbert@interlog.com
-10th June 2022
+12th August 2022

doc/sg3_utils_json.8

@@ -14,9 +14,10 @@ returned by SCSI commands (e.g. sg_vpd) can optionally provide JSON
 output, rather than simple, human-readable output. The default remains
 human-readable output.
 .PP
-JSON is an open standard file format that can be used for data exchange
-between systems. See https://en.wikipedia.org/wiki/JSON . JSON comes in many
-flavours and this one uses the json-builder C implementation found at
+JavaScript Object Notation (JSON) is an open standard file format that can be
+used for data exchange between programs including across a network. See
+https://en.wikipedia.org/wiki/JSON . JSON comes in many flavours and this one
+uses the json-builder C implementation found at
 https://github.com/json-parser/json-builder which implements four simple JSON
 data types: string, integer, boolean and null. Its other data types are JSON
 object and JSON array.
@@ -210,11 +211,17 @@ human readable form. Errors are reported to stderr and may cause the early
 termination of a utility (e.g. command line option syntax error).
 .PP
 When the \fI\-\-json\fR option is given and no errors are detected, then
-only JSON is sent to stdout. If the 'o' control character is in the \fIJO\fR
-argument to the \fI\-\-json\fR option, then the former "human readable"
-output is placed in a JSON array named "output" within a JSON object
-named "utility_invoked". Each line of the former "human readable" output
-is placed in its own element of the JSON array named "output".
+only JSON is normally sent to stdout. As the SCSI response is parsed, a JSON
+representation is built as a tree in memory. After all other actions (perhaps
+apart from the final exit status report) that JSON tree is "dumped" to
+stdout. This means if there is any non-JSON output sent to stdout that
+it will appear _before_ the JSON output.
+.PP
+If the 'o' control character is in the \fIJO\fR argument to the
+\fI\-\-json\fR option, then the former "human readable" output is placed in
+a JSON array named "output" within a JSON object named "utility_invoked".
+Each line of the former "human readable" output is placed in its own
+element of the JSON array named "output".
 .PP
 A JSON tree is built in memory as the utility parses the data returned
 from the SCSI device (e.g. sg_vpd parsing a VPD page returned from a
@@ -236,7 +243,7 @@ reason then no JSON output will appear. With the normal, human readable output
 processing, some output may appear before the utility aborts in such bad
 situations.
 .SH ERRORS
-No attempts has been made to translate errors into JSON form, apart from the
+No attempts have been made to translate errors into JSON form, apart from the
 final "exit_status" JSON object where a value of 0 means "no errors". Exit
 status values indicating a problem range from 1 to 255.
 .PP

doc/sg_decode_sense.8

@@ -1,13 +1,13 @@
-.TH SG_DECODE_SENSE "8" "July 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_DECODE_SENSE "8" "August 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_decode_sense \- decode SCSI sense and related data
 .SH SYNOPSIS
 .B sg_decode_sense
 [\fI\-\-binary=BFN\fR] [\fI\-\-cdb\fR] [\fI\-\-err=ES\fR] [\fI\-\-file=HFN\fR]
 [\fI\-\-help\fR] [\fI\-\-hex\fR] [\fI\-\-inhex=HFN\fR]
-[\fI\-\-ignore\-first\fR] [\fI\-\-nodecode\fR] [\fI\-\-nospace\fR]
-[\fI\-\-status=SS\fR] [\fI\-\-verbose\fR] [\fI\-\-version\fR]
-[\fI\-\-write=WFN\fR] [H1 H2 H3 ...]
+[\fI\-\-ignore\-first\fR] [\fI\-\-json[=JO]\fR] [\fI\-\-nodecode\fR]
+[\fI\-\-nospace\fR] [\fI\-\-status=SS\fR] [\fI\-\-verbose\fR]
+[\fI\-\-version\fR] [\fI\-\-write=WFN\fR] [H1 H2 H3 ...]
 .SH DESCRIPTION
 .\" Add any additional description here
 This utility takes SCSI sense data in binary or as a sequence of ASCII
@@ -106,6 +106,12 @@ first hexadecimal value on each line. This option has no effect if
 character from and after "#" on a line are ignored. Useful with the
 \fI\-\-file=HFN\fR and \fI\-\-nodecode\fR options.
 .TP
+\fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
+.br
+This option is designed to parse sense data into JSON output.
+.TP
 \fB\-N\fR, \fB\-\-nodecode\fR
 Do not decode the given data as sense or a cdb. Useful when arbitrary data
 is given (e.g. when converting hex to binary or vice versa).

doc/sg_get_elem_status.8

@@ -1,4 +1,4 @@
-.TH SG_GET_ELEM_STATUS "8" "July 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_GET_ELEM_STATUS "8" "August 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_get_elem_status \- send SCSI GET PHYSICAL ELEMENT STATUS command
 .SH SYNOPSIS
@@ -72,8 +72,8 @@ sg3_utils manpage for more information. If the \fI\-\-raw\fR option is
 also given then the contents of \fIFN\fR are treated as binary.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see the sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-m\fR, \fB\-\-maxlen\fR=\fILEN\fR
 where \fILEN\fR is the (maximum) response length in bytes. It is placed in

doc/sg_get_lba_status.8

@@ -1,4 +1,4 @@
-.TH SG_GET_LBA_STATUS "8" "July 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_GET_LBA_STATUS "8" "August 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_get_lba_status \- send SCSI GET LBA STATUS(16 or 32) command
 .SH SYNOPSIS
@@ -87,8 +87,8 @@ given then it is ignored. If the \fI\-\-raw\fR option is also given then
 the contents of \fIFN\fR are treated as binary.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-l\fR, \fB\-\-lba\fR=\fILBA\fR
 where \fILBA\fR is the starting Logical Block Address (LBA) to check the

doc/sg_inq.8

@@ -1,4 +1,4 @@
-.TH SG_INQ "8" "July 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_INQ "8" "August 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_inq \- issue SCSI INQUIRY command and/or decode its response
 .SH SYNOPSIS
@@ -13,8 +13,8 @@ sg_inq \- issue SCSI INQUIRY command and/or decode its response
 .PP
 .B sg_inq
 [\fI\-36\fR] [\fI\-a\fR] [\fI\-A\fR] [\fI\-b\fR] [\fI\-\-B=0|1\fR]
-[\fI\-c\fR] [\fI\-cl\fR] [\fI\-d\fR] [\fI\-e\fR] [\fI\-f\fR]
-[\fI\-h\fR] [\fI\-H\fR] [\fI\-i\fR] [\fI\-I=FN\fR] [\fI\-l=LEN\fR]
+[\fI\-c\fR] [\fI\-cl\fR] [\fI\-d\fR] [\fI\-e\fR] [\fI\-f\fR] [\fI\-h\fR]
+[\fI\-H\fR] [\fI\-i\fR] [\fI\-I=FN\fR] [\fI\-j[=LEN]\fR] [\fI\-l=LEN\fR]
 [\fI\-L\fR] [\fI\-m\fR] [\fI\-M\fR] [\fI\-o\fR] [\fI\-p=VPD_PG\fR]
 [\fI\-P\fR] [\fI\-r\fR] [\fI\-s\fR] [\fI\-u\fR] [\fI\-v\fR]
 [\fI\-V\fR] [\fI\-x\fR] [\fI\-36\fR] [\fI\-?\fR] \fIDEVICE\fR
@@ -166,8 +166,8 @@ including a hash mark to the end of a line is ignored. If the \fI\-\-raw\fR
 option is also given then \fIFN\fR is treated as binary.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-l\fR, \fB\-\-len\fR=\fILEN\fR
 the number \fILEN\fR is the "allocation length" field in the INQUIRY cdb.
@@ -433,6 +433,9 @@ response in hex use '\-p=83 \-h'.
 \fB\-I\fR=\fIFN\fR
 equivalent to \fI\-\-inhex=FN\fR in the OPTIONS section.
 .TP
+\fB\-j[\fR=\fIJO]\fR
+equivalent to \fI\-\-json[=JO]\fR in the OPTIONS section.
+.TP
 \fB\-l\fR=\fILEN\fR
 equivalent to \fI\-\-len=LEN\fR in the OPTIONS section.
 .TP

doc/sg_opcodes.8

@@ -1,4 +1,4 @@
-.TH SG_OPCODES "8" "April 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_OPCODES "8" "August 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_opcodes \- report supported SCSI commands or task management functions
 .SH SYNOPSIS
@@ -94,8 +94,8 @@ sg3_utils manpage for more information. If the \fI\-\-raw\fR option is
 also given then the contents of \fIFN\fR are treated as binary.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see the sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-m\fR, \fB\-\-mask\fR
 additionally prints out the cdb mask in hex. So a 12 byte cdb will have

doc/sg_rep_zones.8

@@ -1,4 +1,4 @@
-.TH SG_REP_ZONES "8" "July 2022" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_REP_ZONES "8" "AUGUST 2022" "sg3_utils\-1.48" SG3_UTILS
 .SH NAME
 sg_rep_zones \- send SCSI REPORT ZONES, REALMS or ZONE DOMAINS command
 .SH SYNOPSIS
@@ -103,8 +103,8 @@ from a REPORT ZONES command. Use the \fI\-\-domain\fR or \fI\-\-realm\fR
 option for decoding the other two commands.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see the sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-l\fR, \fB\-\-locator\fR=\fILBA\fR
 where \fILBA\fR plays a similar role as it does in \fI\-\-start=LBA\fR.

doc/sg_vpd.8

@@ -6,8 +6,9 @@ sg_vpd \- fetch SCSI VPD page and/or decode its response
 [\fI\-\-all\fR] [\fI\-\-enumerate\fR] [\fI\-\-examine\fR] [\fI\-\-force\fR]
 [\fI\-\-help\fR] [\fI\-\-hex\fR] [\fI\-\-ident\fR] [\fI\-\-inhex=FN\fR]
 [\fI\-\-json[=JO]\fR] [\fI\-\-long\fR] [\fI\-\-maxlen=LEN\fR]
-[\fI\-\-page=PG\fR] [\fI\-\-quiet\fR] [\fI\-\-raw\fR] [\fI\-\-vendor=VP\fR]
-[\fI\-\-verbose\fR] [\fI\-\-version\fR] [\fIDEVICE\fR]
+[\fI\-\-page=PG\fR] [\fI\-\-quiet\fR] [\fI\-\-raw\fR]
+[\fI\-\-sinq_inraw=RFN\fR] [\fI\-\-vendor=VP\fR] [\fI\-\-verbose\fR]
+[\fI\-\-version\fR] [\fIDEVICE\fR]
 .SH DESCRIPTION
 .\" Add any additional description here
 .PP
@@ -62,20 +63,24 @@ with \fI\-\-vendor=VP\fR to restrict output to known vendor specific pages
 for vendor/product \fIVP\fR.
 .TP
 \fB\-E\fR, \fB\-\-examine\fR
-scan part of all of the VPD space (from 0x0 to 0xff) and output any pages
-found. This option ignores the contents of VPD page 0x0 which should contain
-a list of all supported VPD pages. However some vendors either forget to
-list some standard pages or perhaps purposely don't list vendor specific
-pages which are in the range 0xc0 to 0xff.
-.br
-If the \fI\-\-page=PG\fR option is not given and this option is given once
-then the scan is from VPD page number 0x80 to 0xff inclusive. If the
-\fI\-\-page=PG\fR option is given then the scan is from 0x80 to
-\fIPG\fR inclusive. If this option is given twice then the scan starts at
-VPD page 0x0.
+scan part of all of the VPD space (page numbers 0x0 to 0xff) and output any
+pages found. If this option is given once, the scan starts at page 0x80;
+if it is given twice, the scan starts at 0x0; and if given three times the
+scan starts at 0xc0. This option takes no notice of the contents of VPD page
+0x0 which should contain a list of all supported VPD pages. Some vendors
+either forget to list some standard pages or perhaps purposely don't list
+vendor specific pages which are in the range 0xc0 to 0xff.
+.br
+If the \fI\-\-page=PG\fR option is not given then the scan finishes at page
+0xff. if the \fI\-\-page=PG\fR option is given then the scan finishes at
+page \fIPG\fR. A check is made before the scan to make sure the start page
+is less than or equal to the finish page; if not the start and finish page
+numbers are swapped.
 .br
 The sdparm utility which lists mode and VPD pages also has a \fB\-\-examine\fR
-option will similar functionility.
+option will similar functionility. Note that T10 has changed most of the
+pages that list supported pages (e.g. VPD, mode and log pages; supported
+commands) to add the weasel words "may or may not list all ...".
 .TP
 \fB\-f\fR, \fB\-\-force\fR
 As a sanity check, the normal action when fetching VPD pages other than
@@ -119,8 +124,8 @@ including a hash mark to the end of line is ignored. If the \fI\-\-raw\fR
 option is also given then \fIFN\fR is treated as binary.
 .TP
 \fB\-j\fR, \fB\-\-json[\fR=\fIJO\fR]
-output is in JSON format instead of human readable form. This is an
-EXPERIMENTAL feature; see sg3_utils_json manpage.
+output is in JSON format instead of human readable form. See sg3_utils_json
+manpage or use '?' for \fIJO\fR for a summary.
 .TP
 \fB\-l\fR, \fB\-\-long\fR
 when decoding some VPD pages, give a little more output. For example the ATA
@@ -166,6 +171,23 @@ used. The binary is sent to stdout, and errors are sent to stderr.
 if used with \fI\-\-inhex=FN\fR then the contents of \fIFN\fR is treated as
 binary.
 .TP
+\fB\-Q\fR, \fB\-\-sinq_inraw\fR=\fIRFN\fR
+where \fIRFN\fR is a filename containing binary standard INQUIRY response
+data that matches either \fIDEVICE\fR or \fIFN\fR. Linux places this
+standard INQUIRY response in its sysfs pseudo filesystem. A typical
+location is at /sys/class/scsi_disk/<hctl>/device/inquiry where <hctl> is
+a four part numeric tuple separated by colons. This tuple distinguishes
+the device from any others on the system. Linux also places some VPD page
+responses in binary in the same directory with names like "vpd_pg83" where
+the last two digits form the hexadecimal VPD page number whose binary
+contents are therein.
+.br
+Some VPD pages (e.g. the Extended Inquiry VPD page) depend on knowing
+the settings in the standard INQUIRY response to interpret the fields
+in that VPD page. This option together with the \fI\-\-all\fR,
+\fI\-\-examine\fR or \fI\-\-page=PG\fR allows this utility to process
+both the standard INQUIRY response and VPD pages in the same invocation. 
+.TP
 \fB\-M\fR, \fB\-\-vendor\fR=\fIVP\fR
 where \fIVP\fR is a vendor (e.g. "sea" for Seagate) or vendor/product
 acronym (e.g. "hp3par" for the 3PAR array from HP). Many vendors have

include/sg_lib_data.h

@@ -46,9 +46,9 @@ extern "C" {
 #define SG_VARIABLE_LENGTH_CMD 0x7f
 #define SG_WRITE_BUFFER 0x3b
 #define SG_ZONING_OUT 0x94
-#define SG_ZBC_OUT SG_ZONING_OUT	/* as SPC calls them */
+#define SG_ZBC_OUT SG_ZONING_OUT        /* as SPC calls them */
 #define SG_ZONING_IN 0x95
-#define SG_ZBC_IN SG_ZONING_IN		/* as SPC calls them */
+#define SG_ZBC_IN SG_ZONING_IN          /* as SPC calls them */
 
 
 
@@ -128,6 +128,7 @@ extern struct sg_lib_value_name_t sg_lib_scsi_feature_sets[];
 extern const char * sg_lib_sense_key_desc[];
 extern const char * sg_lib_pdt_strs[];
 extern const char * sg_lib_transport_proto_strs[];
+extern const char * sg_lib_tapealert_strs[];
 extern int sg_lib_pdt_decay_arr[];
 
 extern struct sg_lib_simple_value_name_t sg_lib_nvme_admin_cmd_arr[];

include/sg_pr2serr.h

@@ -320,7 +320,10 @@ void sgj_js_nv_ihexstr_nex(sgj_state * jsp, sgj_opaque_p jop,
                            const char * str_name, const char * val_s,
                            const char * nex_s);
 
-/* Add hex byte strings irrespective of jsp->pr_hex setting. */
+/* Add named field whose value is a (large) JSON string made up of num_bytes
+ * ASCII hexadecimal bytes (each two hex digits seperated by a space) starting
+ * at byte_arr. The heap is used for intermediate storage so num_bytes can
+ * be arbitrarily large. */
 void sgj_js_nv_hex_bytes(sgj_state * jsp, sgj_opaque_p jop, const char * name,
                          const uint8_t * byte_arr, int num_bytes);
 

lib/Makefile.am

@@ -91,6 +91,7 @@ AM_CFLAGS = -Wall -W $(DBG_CFLAGS)
 # AM_CFLAGS = -Wall -W -Wextra -Wmisleading-indentation -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wnull-dereference -Wshadow -Wjump-misses-init
 # AM_CFLAGS = -Wall -W -pedantic -std=c11
 # AM_CFLAGS = -Wall -W -pedantic -std=c11 --analyze
+# AM_CFLAGS = -Wall -W -pedantic -std=c++11
 # AM_CFLAGS = -Wall -W -pedantic -std=c++14
 # AM_CFLAGS = -Wall -W -pedantic -std=c++1z
 # AM_CFLAGS = -Wall -W -pedantic -std=c++20

lib/Makefile.in

@@ -404,6 +404,7 @@ AM_CFLAGS = -Wall -W $(DBG_CFLAGS)
 # AM_CFLAGS = -Wall -W -Wextra -Wmisleading-indentation -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wnull-dereference -Wshadow -Wjump-misses-init
 # AM_CFLAGS = -Wall -W -pedantic -std=c11
 # AM_CFLAGS = -Wall -W -pedantic -std=c11 --analyze
+# AM_CFLAGS = -Wall -W -pedantic -std=c++11
 # AM_CFLAGS = -Wall -W -pedantic -std=c++14
 # AM_CFLAGS = -Wall -W -pedantic -std=c++1z
 # AM_CFLAGS = -Wall -W -pedantic -std=c++20

lib/sg_lib.c

@@ -3580,7 +3580,7 @@ sg_f2hex_arr(const char * fname, bool as_binary, bool no_space,
     int ret = 0;
     unsigned int h;
     const char * lcp;
-    FILE * fp;
+    FILE * fp = NULL;
     struct stat a_stat;
     char line[512];
     char carry_over[4];
@@ -3791,11 +3791,11 @@ sg_f2hex_arr(const char * fname, bool as_binary, bool no_space,
         return SG_LIB_LBA_OUT_OF_RANGE;
     }
     *mp_arr_len = off;
-    if (stdin != fp)
+    if (fp && (! has_stdin))
         fclose(fp);
     return 0;
 fini:
-    if (fp && (stdin != fp))
+    if (fp && (! has_stdin))
         fclose(fp);
     return ret;
 }