8250968: Symlinks attributes not preserved when using jarsigner on zip files #56

coffeys · 2020-09-07T13:48:57Z

Continuation of RFR thread from http://mail.openjdk.java.net/pipermail/security-dev/2020-August/022373.html

CSR has been approved.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8250968: Symlinks attributes not preserved when using jarsigner on zip files

Reviewers

Lance Andersen (@LanceAndersen - Reviewer)
Weijun Wang (@wangweij - Reviewer) ⚠️ Review applies to 2f656ce
hchao - Author ⚠️ Added manually

Download

$ git fetch https://git.openjdk.java.net/jdk pull/56/head:pull/56
$ git checkout pull/56

bridgekeeper · 2020-09-07T13:50:31Z

👋 Welcome back coffeys! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-09-07T13:51:16Z

@coffeys The following labels will be automatically applied to this pull request: jdk security.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label (add|remove) "label" command.

coffeys · 2020-09-07T13:54:26Z

/label core-libs

openjdk · 2020-09-07T13:55:02Z

@coffeys
The core-libs label was successfully added.

mlbridge · 2020-09-07T13:58:08Z

Webrevs

01: Full - Incremental (27cb91f)
00: Full (2f656ce)

coffeys · 2020-09-07T14:02:23Z

/label remove jdk

openjdk · 2020-09-07T14:03:13Z

@coffeys
The jdk label was successfully removed.

coffeys · 2020-09-07T14:15:46Z

/csr needed

openjdk · 2020-09-07T14:16:07Z

@coffeys the issue for this pull request, JDK-8250968, already has an approved CSR request: JDK-8252517

wangweij · 2020-09-07T16:33:06Z

/review

openjdk · 2020-09-07T16:33:53Z

@wangweij Unknown command review - for a list of valid commands use /help.

LanceAndersen

I think this looks good over all Sean. In your SymLinkTest, I probably would have the test delete the file if it exists prior to writing to it.

openjdk · 2020-09-07T16:42:21Z

@coffeys This change now passes all automated pre-integration checks. When the change also fulfills all project specific requirements, type /integrate in a new comment to proceed. After integration, the commit message will be:

8250968: Symlinks attributes not preserved when using jarsigner on zip files

Reviewed-by: lancea, weijun, hchao

If you would like to add a summary, use the /summary command.
To credit additional contributors, use the /contributor command.
To add additional solved issues, use the /issue command.

Since the source branch of this PR was last updated there have been 4 commits pushed to the master branch:

8d6d43c: 8251193: bin/idea.sh is generating wrong folder definitions for JVMCI modules
70d5cac: 8251152: ARM32: jtreg c2 Test8202414 test crash
e0d5b5f: 8252627: Make it safe for JFR thread to read threadObj
e29c3f6: 8252661: Change SafepointMechanism terminology to talk less about "blocking"

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge master into your branch, and then specify the current head hash when integrating, like this: /integrate 8d6d43c33b0d4b757addafae12739e60e785e53f.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

coffeys · 2020-09-07T18:58:08Z

/reviewer add hchao

openjdk · 2020-09-07T18:58:25Z

@coffeys
Reviewer hchao successfully added.

coffeys · 2020-09-07T19:00:10Z

/integrate

openjdk · 2020-09-07T19:01:16Z

@coffeys Since your change was applied there have been 4 commits pushed to the master branch:

8d6d43c: 8251193: bin/idea.sh is generating wrong folder definitions for JVMCI modules
70d5cac: 8251152: ARM32: jtreg c2 Test8202414 test crash
e0d5b5f: 8252627: Make it safe for JFR thread to read threadObj
e29c3f6: 8252661: Change SafepointMechanism terminology to talk less about "blocking"

Your commit was automatically rebased without conflicts.

Pushed as commit 7686e87.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

This patch optimizes the backend implementation of VectorMaskToLong for AArch64, given a more efficient approach to mov value bits from predicate register to general purpose register as x86 PMOVMSK[1] does, by using BEXT[2] which is available in SVE2. With this patch, the final code (input mask is byte type with SPECIESE_512, generated on an SVE vector reg size of 512-bit QEMU emulator) changes as below: Before: mov z16.b, p0/z, #1 fmov x0, d16 orr x0, x0, x0, lsr openjdk#7 orr x0, x0, x0, lsr openjdk#14 orr x0, x0, x0, lsr openjdk#28 and x0, x0, #0xff fmov x8, v16.d[1] orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#8 orr x8, xzr, #0x2 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#16 orr x8, xzr, #0x3 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#24 orr x8, xzr, #0x4 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#32 mov x8, #0x5 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#40 orr x8, xzr, #0x6 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#48 orr x8, xzr, #0x7 whilele p1.d, xzr, x8 lastb x8, p1, z16.d orr x8, x8, x8, lsr openjdk#7 orr x8, x8, x8, lsr openjdk#14 orr x8, x8, x8, lsr openjdk#28 and x8, x8, #0xff orr x0, x0, x8, lsl openjdk#56 After: mov z16.b, p0/z, #1 mov z17.b, #1 bext z16.d, z16.d, z17.d mov z17.d, #0 uzp1 z16.s, z16.s, z17.s uzp1 z16.h, z16.h, z17.h uzp1 z16.b, z16.b, z17.b mov x0, v16.d[0] [1] https://www.felixcloutier.com/x86/pmovmskb [2] https://developer.arm.com/documentation/ddi0602/2020-12/SVE-Instructions/BEXT--Gather-lower-bits-from-positions-selected-by-bitmask- Change-Id: Ia983a20c89f76403e557ac21328f2f2e05dd08e0

…dk#56)

…ng into ldp/stp on AArch64 Macro-assembler on aarch64 can merge adjacent loads or stores into ldp/stp[1]. For example, it can merge: ``` str w20, [sp, openjdk#16] str w10, [sp, openjdk#20] ``` into ``` stp w20, w10, [sp, openjdk#16] ``` But C2 may generate a sequence like: ``` str x21, [sp, openjdk#8] str w20, [sp, openjdk#16] str x19, [sp, openjdk#24] <--- str w10, [sp, openjdk#20] <--- Before sorting str x11, [sp, openjdk#40] str w13, [sp, openjdk#48] str x16, [sp, openjdk#56] ``` We can't do any merging for non-adjacent loads or stores. The patch is to sort the spilling or unspilling sequence in the order of offset during instruction scheduling and bundling phase. After that, we can get a new sequence: ``` str x21, [sp, openjdk#8] str w20, [sp, openjdk#16] str w10, [sp, openjdk#20] <--- str x19, [sp, openjdk#24] <--- After sorting str x11, [sp, openjdk#40] str w13, [sp, openjdk#48] str x16, [sp, openjdk#56] ``` Then macro-assembler can do ld/st merging: ``` str x21, [sp, openjdk#8] stp w20, w10, [sp, openjdk#16] <--- Merged str x19, [sp, openjdk#24] str x11, [sp, openjdk#40] str w13, [sp, openjdk#48] str x16, [sp, openjdk#56] ``` To justify the patch, we run `HelloWorld.java` ``` public class HelloWorld { public static void main(String [] args) { System.out.println("Hello World!"); } } ``` with `java -Xcomp -XX:-TieredCompilation HelloWorld`. Before the patch, macro-assembler can do ld/st merging for 3688 times. After the patch, the number of ld/st merging increases to 3871 times, by ~5 %. Tested tier1~3 on x86 and AArch64. [1] https://github.com/openjdk/jdk/blob/a95062b39a431b4937ab6e9e73de4d2b8ea1ac49/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp#L2079

* Update LICENSE file * The experimental version is aimed to refactor the tool * verificationType mistakenly added to source code is renamed to type * Adapted mvn bindings to at8 * Added tests to represent bad stream handling in jdec and correctone in jdis * Made jdec to write output to proper toolOutput instead of bad logOutput * 7903208: [Jasm] Add support for generics (the Signature attribute) * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * Replaced hardcoded String[] args, by varargs String... where reasonable Once the "tool" mandatory array memebr was removed from each tool's main method, and considering calls from libraries, and form tests where the argument is very often just one file, or more readable "a1", a2"... without new String[]{} declaration, changed those String[] enforcing headers to more benevolent String... * Intentionally removed ACC_SUPER class modifier was causing hotswap to fail When jasm's disasm, modify, asm cycle output binary was used for class hotswap, remote JDK was not accepting it with java.lang.UnsupportedOperationException: class redefinition failed: attempted to change the class modifiers beacuse of ommited supoer keyword (although it have already no real reason) This patch is returning the kwyword without conditions, when it was included in original bytecode * extended test to verify that super is not used always if super is not in source code, then it is not in disambled code * 7903248: jasm: FieldData.ConstantValue holds undefined reference to CP while writing fields to a class * Added support for stdin in jdis * stdin is now read also by jasm,jcod,jdec more tests needed * Removed jdis specific missleading provide method * Tool output moved where it belongs to commons * Removed duplicated declaration of ArrayList<ToolInput> fileList * Changed requireNonNull message in getDataInputStream * Added one more abstraction layer providing highly reusable byte[] based input * Properly exiting after verson is printed * Stream based inputs are now drained once needed, not during construction * reworked stdin read to be initiated by - As a side effect, files and stdin can be read together remove dduplicated code in tests by getting class in compile time added tests and adapted older to new behavior * added tests if asmtools assmble itself into valid bytecode Added two set of tests jdec->jcoder->load jdis->jasm->load Both in two variants, with -g, and without Both jdec->jcoder->load works fine Both jdis->jasm->load now fails on three files from 278: /org/openjdk/asmtools/jasm/JasmEnvironment$InputFile.class, /org/openjdk/asmtools/jcoder/Jcoder.class, /org/openjdk/asmtools/jasm/Parser.class Fail looks valid. Unluckily, the issue where -g disassembled and back assembled com.google.gson.Gson produce invalid bytecode was not hit The class BruteForceHelper is reusable for any set of classes * Naive fix for enforced dot.suffix * repalce stdout by configurable object This checkout is not buildable, but server as showcase for issue with dual logging for compilers * Replaced dualstream logger by wrapper so it can be later made fully customisable * Enabled shared i18n properties via reused asmtools/i18n.props * All four tools now use neww ToolOutput output As a consequence, all four tools by by default prints to stdout, and honours -d properly * Fixed typo which casued behavior regression found by unittests * Refatoed tests so they can harbour resources in maven way * Moved all logging to stderr. use -dls to return original behavior The -dls switch is oging to be removed once stderr is properl adapted * Implemented library-like input and output This commit is adding byte[]/String input/output clasases for direct library usage. Added tests, advertising how simple is usage of those inputs/outputs There are two hunks, which fixes issues ovberlooked in previous refactorings: - traceln now correctly calls getOutputs - jdec now uses proper logger insted of accidental stderr * Enabled -g for jcoder to be set from external code * Temporary workaround around tests being order-sensitive As -g is now making some disassmebld code not asemble-able back, and order of tests is not deterministic, and Options are static, thus if -g is now set in some test, it is not unset in other tests. Real fix is to move Options out of static context to context of environment as it is done for jcoder. * Added github actions Just for record for "act" for local testing. To run with podman based distros, several steps are ncessary. See nektos/act#303 ; especially * nektos/act#303 (comment) * and nektos/act#303 (comment) * the bind and socket eg: systemctl enable --now --user podman.socket systemctl start --user podman.socket export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock ../act/bin/act --bind --container-daemon-socket $XDG_RUNTIME_DIR/podman/podman.sock To rerun the build on clean env, you have to stop and start the podman socket again Without explicit 'mvn test' the tests are not running. Not sure why * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * Fixed issue when jasm produced only one file from source with multiple ones Added tests for this issue Added test veryfying that the jcoder is not affected * 7903401: jtreg fails if set of jdk tests process jasm,jdis files with defects * 7903402: jdis: tool writes incorrect StackMapTable if the first same_frame has type 0 (openjdk#51) * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * 7903401: jtreg fails if set of jdk tests process jasm,jdis files with defects * 7903402: jdis: tool writes incorrect StackMapTable if the first same_frame has type 0 * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * Refactored new ToolInput/Outputs so they reside in own packages and not in original interfaces (openjdk#53) * Refactored new ToolInput/Outputs so they reside in own packages and not in original interfaces * Removed unused imports * Added licence headers * Added ajvadoc description to three main interfaces. * Removed unnecessary guard condition before changging \ to / for fqn * Replaced "\n" by System.lineSeparator() * Used better names for highlighted abbrevations * 7903405: compiler does not warn about instruction arguments that exceed allowed limits (openjdk#54) * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * 7903401: jtreg fails if set of jdk tests process jasm,jdis files with defects * 7903402: jdis: tool writes incorrect StackMapTable if the first same_frame has type 0 * 7902888: Excess entries in BootstrapMethods with the same bsm, bsmKind, bsmArgs * 7903405: compiler does not warn about instruction arguments that exceed allowed limits * 7903405: compiler does not warn about instruction arguments that exceed allowed limits * Delete ToolOutput.java * Fixed junit test for CODETOOLS-7903405 (openjdk#56) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 (openjdk#57) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * Fix tabs * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 (Part II) (openjdk#58) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * Fix tabs * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * CODETOOLS-7903506: Asmtools: jdis prints BootstrapMethod attribute if detailed output is off (openjdk#59) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * Fix tabs * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903506: Asmtools: jdis prints BootstrapMethod attribute if detailed output is off * 7903509: jcoder, jasm: add option to override class file version in source file(s) (openjdk#60) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * Fix tabs * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903509: jcoder, jasm: add option to override class file version in source file(s) * CODETOOLS-7903531: jdis: Suppress printing comments by adding an option (openjdk#61) * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * Fix tabs * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903458: Umbrella: Preparations for switching to Asmtools 8.0 * 7903531: jdis: Suppress printing comments by adding an option --------- Co-authored-by: Jiri Vanek <jvanek@redhat.com>

JDK-8250968

2f656ce

openjdk bot added jdk jdk-dev@openjdk.org security security-dev@openjdk.org labels Sep 7, 2020

coffeys changed the title ~~RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files~~ 8250968: Symlinks attributes not preserved when using jarsigner on zip files Sep 7, 2020

openjdk bot added the rfr Pull request is ready for review label Sep 7, 2020

openjdk bot added the core-libs core-libs-dev@openjdk.org label Sep 7, 2020

openjdk bot removed the jdk jdk-dev@openjdk.org label Sep 7, 2020

LanceAndersen approved these changes Sep 7, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 7, 2020

wangweij approved these changes Sep 7, 2020

View reviewed changes

Copyright and test clean up

27cb91f

LanceAndersen approved these changes Sep 7, 2020

View reviewed changes

openjdk bot closed this Sep 7, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated labels Sep 7, 2020

openjdk bot removed the rfr Pull request is ready for review label Sep 7, 2020

coffeys deleted the JDK-8250968 branch September 8, 2020 08:33

yftsai mentioned this pull request Feb 17, 2022

8005885: enhance PrintCodeCache to print more data #7389

Closed

3 tasks

caojoshua added a commit to caojoshua/jdk that referenced this pull request Jul 14, 2023

[PEA] paranoid test for re-allocating an object at merge point (openj…

00aa8e5

…dk#56)

8250968: Symlinks attributes not preserved when using jarsigner on zip files #56

8250968: Symlinks attributes not preserved when using jarsigner on zip files #56

Uh oh!

Conversation

coffeys commented Sep 7, 2020 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Download

Uh oh!

bridgekeeper bot commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

coffeys commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

mlbridge bot commented Sep 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

coffeys commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

coffeys commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

wangweij commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

LanceAndersen left a comment

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented Sep 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coffeys commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

coffeys commented Sep 7, 2020

Uh oh!

openjdk bot commented Sep 7, 2020

Uh oh!

Uh oh!

coffeys commented Sep 7, 2020 •

edited by openjdk bot

Loading

mlbridge bot commented Sep 7, 2020 •

edited

Loading

openjdk bot commented Sep 7, 2020 •

edited

Loading