Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8252523: Add ASN.1 Formatter to work with test utility HexPrinter #268

Closed
wants to merge 5 commits into from

Conversation

@RogerRiggs
Copy link
Contributor

@RogerRiggs RogerRiggs commented Sep 20, 2020

JDK-8252523: Add ASN.1 Formatter to work with test utility HexPrinter

Debugging functions that utilize ASN.1, DER, and BER encoded streams is
difficult without test utilities to show the contents.
The ASN.1 formatter reads a stream and produces annotated output of the
tags, values, and structures.
When used with the test library jdk.test.lib.hexdump.HexPrinter the annotations are synchronized
with the hex formatted output.

Small changes to HexPrinter are included to improve the output readability.

Example decoding of a .pem certificate:

SEQUENCE [910]
  SEQUENCE [630]
    CONTEXT cons 0 [3]
      BYTE 2, 
    BYTE 3, 
    SEQUENCE [13]
      OBJECT ID  [9] 1.2.840.113549.1.1.11 (SHA256withRSA) 
      NULL 
    SEQUENCE [76]
      SET [11]
        SEQUENCE [9]
          OBJECT ID  [3] 2.5.4.6 (CountryName) 
          'IN' 
      ...
      SET [16]
        SEQUENCE [14]
          OBJECT ID  [3] 2.5.4.3 (CommonName) 
          Client1 
    SEQUENCE [30]
      UTCTIME  [13] '150526221718Z' 
      UTCTIME  [13] '250523221718Z' 
    ...
    SEQUENCE [290]
      SEQUENCE [13]
        OBJECT ID  [9] 1.2.840.113549.1.1.1 (RSA) 
        NULL 
      BIT STRING  [271]
      CONTEXT cons 3 [123]
        SEQUENCE [121]
          SEQUENCE [9]
            OBJECT ID  [3] 2.5.29.19 (BasicConstraints) 
            OCTET STRING  [2] <Unprintable> 
          SEQUENCE [44]
            OBJECT ID  [9] 2.16.840.1.113730.1.13 
            OCTET STRING  [31] '..OpenSSL Generated Certificate' 
          SEQUENCE [29]
            OBJECT ID  [3] 2.5.29.14 (SubjectKeyID) 
            OCTET STRING  [22] <Unprintable> 
          SEQUENCE [31]
            OBJECT ID  [3] 2.5.29.35 (AuthorityKeyID) 
            OCTET STRING  [24] <Unprintable> 
      SEQUENCE [13]
        OBJECT ID  [9] 1.2.840.113549.1.1.11 (SHA256withRSA) 
        NULL 
      BIT STRING  [257]

When used with the HexPrinter test utility, the formatting of the
hexadecimal values is selected with the parameters to HexPrinter.

    0000: 30 82 03 8e                                     ; SEQUENCE [910]
    0004:             30 82 02 76                         ;   SEQUENCE [630]
    0008:                         a0 03                   ;     CONTEXT cons 0 [3]
    000a:                               02 01 02          ;       BYTE 2, 
    000d:                                        02 01 03 ;     BYTE 3, 
    0010: 30 0d                                           ;     SEQUENCE [13]
    0012:       06 09 2a 86 48 86 f7 0d 01 01 0b          ;       OBJECT ID  [9] 1.2.840.113549.1.1.11 (SHA256withRSA) 
    001d:                                        05 00    ;       NULL 
    001f:                                              30 ;     SEQUENCE [76]
    0020: 4c                                              ; 
    0021:    31 0b                                        ;       SET [11]
    0023:          30 09                                  ;         SEQUENCE [9]
    0025:                06 03 55 04 06                   ;           OBJECT ID  [3] 2.5.4.6 (CountryName) 
    002a:                               13 02 49 4e       ;           'IN' 
    
    ...                                                           ...
    
    005b:                                  31 10          ;       SET [16]
    005d:                                        30 0e    ;         SEQUENCE [14]
    005f:                                              06 ;           OBJECT ID  [3] 2.5.4.3 (CommonName) 
    0060: 03 55 04 03                                     ; 
    0064:             0c 07 43 6c 69 65 6e 74 31          ;           Client1 
    006d:                                        30 1e    ;     SEQUENCE [30]
    006f:                                              17 ;       UTCTIME  [13] '150526221718Z' 
    0070: 0d 31 35 30 35 32 36 32 32 31 37 31 38 5a       ; 
    007e:                                           17 0d ;       UTCTIME  [13] '250523221718Z' 
    0080: 32 35 30 35 32 33 32 32 31 37 31 38 5a          ; 
    
    ...                                                         ...
    
    00db:                                  30 82 01 22    ;     SEQUENCE [290]
    00df:                                              30 ;       SEQUENCE [13]
    00e0: 0d                                              ; 
    00e1:    06 09 2a 86 48 86 f7 0d 01 01 01             ;         OBJECT ID  [9] 1.2.840.113549.1.1.1 (RSA) 
    00ec:                                     05 00       ;         NULL 
    00ee:                                           03 82 ;       BIT STRING  [271]
    00f0: 01 0f 00 30 82 01 0a 02 82 01 01 00 d8 70 03 54 ; 
    
    ...
    
    01f0: 0a 2d f5 de 59 3e d9 5e 74 93 d2 45 02 03 01 00 ; 
    0200: 01                                              ; 
    0201:    a3 7b                                        ;       CONTEXT cons 3 [123]
    0203:          30 79                                  ;         SEQUENCE [121]
    0205:                30 09                            ;           SEQUENCE [9]
    0207:                      06 03 55 1d 13             ;             OBJECT ID  [3] 2.5.29.19 (BasicConstraints) 
    020c:                                     04 02 30 00 ;             OCTET STRING  [2] <Unprintable> 
    0210: 30 2c                                           ;           SEQUENCE [44]
    0212:       06 09 60 86 48 01 86 f8 42 01 0d          ;             OBJECT ID  [9] 2.16.840.1.113730.1.13 
    021d:                                        04 1f 16 ;             OCTET STRING  [31] '..OpenSSL Generated Certificate' 
    0220: 1d 4f 70 65 6e 53 53 4c 20 47 65 6e 65 72 61 74 ; 
    0230: 65 64 20 43 65 72 74 69 66 69 63 61 74 65       ; 
    023e:                                           30 1d ;           SEQUENCE [29]
    0240: 06 03 55 1d 0e                                  ;             OBJECT ID  [3] 2.5.29.14 (SubjectKeyID) 
    0245:                04 16 04 14 87 13 66 bc 7a 4d 8e ;             OCTET STRING  [22] <Unprintable> 
    0250: 98 e7 97 fb cc 56 41 27 c8 5e 4c b2 4d          ; 
    025d:                                        30 1f    ;           SEQUENCE [31]
    025f:                                              06 ;             OBJECT ID  [3] 2.5.29.35 (AuthorityKeyID) 
    0260: 03 55 1d 23                                     ; 
    0264:             04 18 30 16 80 14 1f 21 4f db 10 31 ;             OCTET STRING  [24] <Unprintable> 
    0270: d0 67 83 09 03 d3 cd fc 46 ec cf 1d 8b b4       ; 
    027e:                                           30 0d ;       SEQUENCE [13]
    0280: 06 09 2a 86 48 86 f7 0d 01 01 0b                ;         OBJECT ID  [9] 1.2.840.113549.1.1.11 (SHA256withRSA) 
    028b:                                  05 00          ;         NULL 
    028d:                                        03 82 01 ;       BIT STRING  [257]
    0290: 01 00 3e 2b 5f 32 aa f0 f7 52 2b ba f3 bb 07 ee ; 
    
    ...
    
    0390: 6d 94                                           ; 

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8252523: Add ASN.1 Formatter to work with test utility HexPrinter

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/268/head:pull/268
$ git checkout pull/268

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Sep 20, 2020

👋 Welcome back rriggs! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Sep 20, 2020

@RogerRiggs The following labels will be automatically applied to this pull request: core-libs security.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label (add|remove) "label" command.

@RogerRiggs RogerRiggs marked this pull request as ready for review Sep 20, 2020
@openjdk openjdk bot added the rfr label Sep 20, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 20, 2020

Copy link
Contributor

@wangweij wangweij left a comment

Just some comments based on the output example.

test/lib/jdk/test/lib/hexdump/ASN1Formatter.java Outdated Show resolved Hide resolved
test/lib/jdk/test/lib/hexdump/ASN1Formatter.java Outdated Show resolved Hide resolved
@RogerRiggs
Copy link
Contributor Author

@RogerRiggs RogerRiggs commented Sep 29, 2020

Are there any other comments or suggestions?

@wangweij
Copy link
Contributor

@wangweij wangweij commented Sep 29, 2020

Can you post an example output of that cert? What about indefinite length? (Ex: 0x24, (byte) 0x80, 4, 2, 'a', 'b', 4, 2, 'c', 'd', 0, 0)

@RogerRiggs
Copy link
Contributor Author

@RogerRiggs RogerRiggs commented Sep 29, 2020

For your example the output is:

0000: 24 80                                           ; UNIVERSAL CONSTRUCTED OCTET STRING [INDEFINITE]
0002:       04 02 61 62                               ;   OCTET STRING [2] 'ab' 
0006:                   04 02 63 64                   ; OCTET STRING [2] 'cd' 
000a:                               00 00             ; END-OF-CONTENT ```

It seems to have lost an indent, I'll take a look at that.
@jnimeh
Copy link
Member

@jnimeh jnimeh commented Sep 29, 2020

Also in that last example, it seems to suggest that the second octet string is nested within the first one since it sits at a second indent layer. They are both primitives completely covered by their two byte values so shouldn't they sit at the same indentation level? Or is the indentation not there to suggest nested substructures and is more for separation between elements? Or is this what you mean by "lost an indent"? Also, should the end of content be at the same indentation level as the initial indefinite length encoding?

…h tag-values, added test for indefinite length
@wangweij

This comment has been minimized.

Typo. Indefinite.

@RogerRiggs
Copy link
Contributor Author

@RogerRiggs RogerRiggs commented Sep 29, 2020

Also in that last example, it seems to suggest that the second octet string is nested within the first one since it sits at a second indent layer. They are both primitives completely covered by their two byte values so shouldn't they sit at the same indentation level? Or is the indentation not there to suggest nested substructures and is more for separation between elements? Or is this what you mean by "lost an indent"? Also, should the end of content be at the same indentation level as the initial indefinite length encoding?

Yes, all of the enclosed items should be at the same indent level. (A bug as it turns out).
I chose to indent the END-OF-CONTENT line at the same level to terminate the list of tag-values at that level
All of the items enclosed are at the same level.

The updated output is:

0000: 24 80                                           ; UNIVERSAL CONSTRUCTED OCTET STRING [INDEFINITE]
0002:       04 02 61 62                               ;   OCTET STRING [2] 'ab' 
0006:                   04 02 63 64                   ;   OCTET STRING [2] 'cd' 
000a:                               00 00             ;   END-OF-CONTENT 
@jnimeh
Copy link
Member

@jnimeh jnimeh commented Sep 29, 2020

Regarding the end-of-content identifier, that looks good. Thanks for fixing the indentation for the right-side ASN.1 interpretation of the bytes. My only remaining question is whether the corresponding hex dumps on the left should match the indentation levels as well. I don't have a strong opinion either way on that one but if you're indenting for each element at the same nest level it seems like that could potentially chew up a lot of horizontal space. Was the extra indentation for the second octet string done for readability?

@RogerRiggs
Copy link
Contributor Author

@RogerRiggs RogerRiggs commented Sep 29, 2020

Regarding the end-of-content identifier, that looks good. Thanks for fixing the indentation for the right-side ASN.1 interpretation of the bytes. My only remaining question is whether the corresponding hex dumps on the left should match the indentation levels as well. I don't have a strong opinion either way on that one but if you're indenting for each element at the same nest level it seems like that could potentially chew up a lot of horizontal space. Was the extra indentation for the second octet string done for readability?

Max had requested the current offset of the byte values, so it was easy to see where each new value started and to keep the offsets on a modulo boundary. The formatter on the right is largely decoupled from the hex value tabular form on the left while keeping the correspondence between the formatted items and the bytes.

@jnimeh
Copy link
Member

@jnimeh jnimeh commented Sep 29, 2020

Regarding the end-of-content identifier, that looks good. Thanks for fixing the indentation for the right-side ASN.1 interpretation of the bytes. My only remaining question is whether the corresponding hex dumps on the left should match the indentation levels as well. I don't have a strong opinion either way on that one but if you're indenting for each element at the same nest level it seems like that could potentially chew up a lot of horizontal space. Was the extra indentation for the second octet string done for readability?

Max had requested the current offset of the byte values, so it was easy to see where each new value started and to keep the offsets on a modulo boundary. The formatter on the right is largely decoupled from the hex value tabular form on the left while keeping the correspondence between the formatted items and the bytes.

If you and Max find that better from a visual perspective then that works for me. Thanks for clarifying that.

Copy link
Contributor

@wangweij wangweij left a comment

LGTM.

@RogerRiggs RogerRiggs changed the title 8252523: Add ASN1 Formatter to work with HexPrinter 8252523: Add ASN.1 Formatter to work with HexPrinter Sep 30, 2020
@RogerRiggs RogerRiggs changed the title 8252523: Add ASN.1 Formatter to work with HexPrinter 8252523: Add ASN.1 Formatter to work with test utility HexPrinter Sep 30, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 30, 2020

@RogerRiggs This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for more details.

After integration, the commit message for the final commit will be:

8252523: Add ASN.1 Formatter to work with test utility HexPrinter

Reviewed-by: weijun

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 142 new commits pushed to the master branch:

  • 709cfe5: 8253815: Remove unused HeapRegionManager::_num_committed from SA
  • f80a606: 8253375: OSX build fails with Xcode 12.0 (12A7209)
  • 04775f1: 8253768: Deleting unused pipe_class definitions in adl-file (x86_64.ad).
  • dc3a0f5: 8253183: Fragile memory barrier selection for some weak memory model platforms
  • 8331e63: 8253778: ShenandoahSafepoint::is_at_shenandoah_safepoint should not access VMThread state from other threads
  • 4c65365: 8253792: javax.swing.text.GapContent documentation typos
  • 3078b5f: 6441211: Small Error in API at javax.swing.plaf.synth.Region
  • 4622a18: 8253791: Issue with useAppleColor check in CSystemColors.m
  • ac02afe: 8253721: Flag -XX:AVX3Threshold does not accept Zero value
  • 4d9f207: 7151826: [TEST_BUG] [macosx] The test javax/swing/JPopupMenu/4966112/bug4966112.java not for mac
  • ... and 132 more: https://git.openjdk.java.net/jdk/compare/224a30f12ce27257af94e2607c53e5f59ac2bd0f...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Sep 30, 2020
@RogerRiggs
Copy link
Contributor Author

@RogerRiggs RogerRiggs commented Sep 30, 2020

/integrate

@openjdk openjdk bot closed this Sep 30, 2020
@openjdk openjdk bot added the integrated label Sep 30, 2020
@openjdk openjdk bot removed ready rfr labels Sep 30, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 30, 2020

@RogerRiggs Since your change was applied there have been 152 commits pushed to the master branch:

  • 06d8cf6: 8253812: Cleanup AbstractMemberWriter
  • 424d7d6: 8252881: [JVMCI] ResolvedJavaType.resolveMethod fails in fastdebug when invoked with a constructor
  • 2a406f3: 8138732: Rename @HotSpotIntrinsicCandidate to @IntrinsicCandidate and move it to the jdk.internal.vm.annotation package
  • 4b16f8a: 8253872: ArgumentHandler must use the same delimiters as in jvmti_tools.cpp
  • 4fb8c77: 8253733: Cleanup internal taglet API
  • e5ba020: 8253829: Wrong length compared in SSPI bridge
  • 9e453d9: 8239090: Improve CPU feature support in VM_Version
  • 8f7c9a7: 8252001: remove usage of PropertyResolvingWrapper in vmTestbase/nsk/jdi
  • 8cf8e46: 8253700: spurious "extends Throwable" at end of Optional.orElseThrow method declaration
  • 8b3d676: 8238737: remove DeoptimizeAllClassesRate from CTW library
  • ... and 142 more: https://git.openjdk.java.net/jdk/compare/224a30f12ce27257af94e2607c53e5f59ac2bd0f...master

Your commit was automatically rebased without conflicts.

Pushed as commit 092c227.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@RogerRiggs RogerRiggs deleted the RogerRiggs:8252523-asn1-formatter branch Oct 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.