8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format #776

iignatev · 2020-10-20T23:14:40Z

Hi all,

could you please review this small patch?

according to RFC3659, time values in MLSx response have the following format:

time-val = 14DIGIT [ "." 1*DIGIT ]

The leading, mandatory, fourteen digits are to be interpreted as, in
order from the leftmost, four digits giving the year, with a range of
1000--9999, two digits giving the month of the year, with a range of
01--12, two digits giving the day of the month, with a range of
01--31, two digits giving the hour of the day, with a range of
00--23, two digits giving minutes past the hour, with a range of
00--59, and finally, two digits giving seconds past the minute, with
a range of 00--60 (with 60 being used only at a leap second). Years
in the tenth century, and earlier, cannot be expressed. This is not
considered a serious defect of the protocol.

The optional digits, which are preceded by a period, give decimal
fractions of a second. These may be given to whatever precision is
appropriate to the circumstance, however implementations MUST NOT add
precision to time-vals where that precision does not exist in the
underlying value being transmitted.

Symbolically, a time-val may be viewed as

YYYYMMDDHHMMSS.sss

The "." and subsequent digits ("sss") are optional. However the "."
MUST NOT appear unless at least one following digit also appears.

MLSxParser, however, uses SimpleDateFormat("yyyyMMddhhmmss") (which is wrong b/c it uses hh instead of HH and doesn't account for optional .sss) to parse modify/create facts and ignore any parse exceptions.

FtpClient actually already has and uses SimpleDateFormat w/ right formats in getLastModified where it parses MDTM response, the patch refactors the code to reuse the same SimpleDateFormat in MLSxParser.

testing:

tier1
test/jdk/sun/net on {linux,windows,macosx}-x64

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Testing

	Linux x32	Linux x64	Windows x64	macOS x64
Build	✔️ (1/1 passed)	✔️ (5/5 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)
Test (tier1)		✔️ (9/9 passed)	❌ (1/9 failed)	✔️ (9/9 passed)

Failed test task

Windows x64 (hs/tier1 compiler)

Issue

JDK-8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format

Reviewers

Daniel Fuchs (@dfuch - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/776/head:pull/776
$ git checkout pull/776

bridgekeeper · 2020-10-20T23:15:42Z

👋 Welcome back iignatyev! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-10-20T23:18:04Z

@iignatev The following label will be automatically applied to this pull request:

net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2020-10-20T23:50:17Z

Webrevs

dfuch · 2020-10-22T10:33:06Z

Hi Igor, is this testable - and if so shouldn't there be a test?
best regards,
-- daniel

iignatev · 2020-10-22T16:04:56Z

Hi Igor, is this testable - and if so shouldn't there be a test?
best regards,
-- daniel

Hi Daniel,

it's testable and originally I planned to add a new test, however upon checking the existing mock ftp-servers, I realized that none of them support MLSx commands, and adding that support doesn't seem to be justified for such a trivial fix. With that being said, I can create a test for this if you believe it's necessary.

-- Igor

iignatev · 2020-10-23T17:34:34Z

I took another stab on the test and it turned out to be easier than I originally anticipated. I, however, decided to take a safer approach and don't use FtpServerfromtest/jdk/sun/net/www/ftptesttest-library, and instead introduced a test-specific mock server in the test code. I'm running the test multiple times on{windows,linux,macosx}-x64` to check how stable it's in our environment, so far there are no failures.

-- Igor

dfuch

Thanks for adding a test Igor!
If that passes on all platform then this looks good to me.

openjdk · 2020-10-23T18:05:22Z

@iignatev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format

Reviewed-by: dfuchs

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 88 new commits pushed to the master branch:

3f6abd2: 8255330: gtest/MetaspaceGtests.java fail on 32-bit platforms
cf821b0: 8232024: Don't pollute log output with multiple errors of the same type
185c8bc: 8255338: CodeSections are never frozen
cc86113: 8255271: Avoid generating duplicate interpreter entries for subword types
5ec1b80: 8255287: aarch64: fix SVE patterns for vector shift count
12daf2b: 8255340: [s390] build failure after JDK-8255208
9007bc2: 8255129: [PPC64, s390] Check vector_size_supported and add VectorReinterpret node
df79257: 8255274: [PPC64, s390] wrong StringLatin1.indexOf version matched
55a0cad: 8254982: (tz) Upgrade time-zone data to tzdata2020c
ac1748e: 8254670: SVE test uses linux-specific api
... and 78 more: https://git.openjdk.java.net/jdk/compare/acacae5e0c58dc711de16edcef0022fe162c1ad9...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

iignatev · 2020-10-23T22:30:22Z

Thanks, Daniel.

/integrate

openjdk · 2020-10-23T22:33:47Z

@iignatev Since your change was applied there have been 88 commits pushed to the master branch:

3f6abd2: 8255330: gtest/MetaspaceGtests.java fail on 32-bit platforms
cf821b0: 8232024: Don't pollute log output with multiple errors of the same type
185c8bc: 8255338: CodeSections are never frozen
cc86113: 8255271: Avoid generating duplicate interpreter entries for subword types
5ec1b80: 8255287: aarch64: fix SVE patterns for vector shift count
12daf2b: 8255340: [s390] build failure after JDK-8255208
9007bc2: 8255129: [PPC64, s390] Check vector_size_supported and add VectorReinterpret node
df79257: 8255274: [PPC64, s390] wrong StringLatin1.indexOf version matched
55a0cad: 8254982: (tz) Upgrade time-zone data to tzdata2020c
ac1748e: 8254670: SVE test uses linux-specific api
... and 78 more: https://git.openjdk.java.net/jdk/compare/acacae5e0c58dc711de16edcef0022fe162c1ad9...master

Your commit was automatically rebased without conflicts.

Pushed as commit 6545e19.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Marcono1234 · 2020-10-25T00:35:20Z

src/java.base/share/classes/sun/net/ftp/impl/FtpClient.java

        }
        return null;
    }

+    private static Date parseRfc3659TimeValue(String s) {


Shouldn't this method be synchronized because SimpleDateFormat is not thread-safe, but instances are stored in the static field dateFormats?
(Note though that this issue existed before as well)

thanks @Marcono1234, it indeed needs to be synchronized, I'll take care of fixing that.

@iignatev if you decide to log a followup a better alternative might be to use DateTimeFormatter

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format

a286753

openjdk bot added the net net-dev@openjdk.org label Oct 20, 2020

iignatev marked this pull request as ready for review October 20, 2020 23:46

openjdk bot added the rfr Pull request is ready for review label Oct 20, 2020

added test

7a1c050

use only filename in assert message

9b69cf9

dfuch approved these changes Oct 23, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 23, 2020

openjdk bot closed this Oct 23, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 23, 2020

Marcono1234 reviewed Oct 25, 2020

View reviewed changes

iignatev deleted the 8255078 branch October 26, 2020 14:27

iignatev mentioned this pull request Oct 26, 2020

8255405: sun/net/ftp/imp/FtpClient uses SimpleDateFormat in not thread-safe manner #867

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format #776

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format #776

iignatev commented Oct 20, 2020 •

edited by openjdk bot

Loading

bridgekeeper bot commented Oct 20, 2020

openjdk bot commented Oct 20, 2020

mlbridge bot commented Oct 20, 2020 •

edited

Loading

dfuch commented Oct 22, 2020

iignatev commented Oct 22, 2020

iignatev commented Oct 23, 2020

dfuch left a comment

openjdk bot commented Oct 23, 2020 •

edited

Loading

iignatev commented Oct 23, 2020

openjdk bot commented Oct 23, 2020

Marcono1234 Oct 25, 2020

iignatev Oct 26, 2020

dfuch Oct 27, 2020 •

edited

Loading

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format #776

8255078: sun/net/ftp/imp/FtpClient$MLSxParser uses wrong datetime format #776

Conversation

iignatev commented Oct 20, 2020 • edited by openjdk bot Loading

Progress

Testing

Issue

Reviewers

Download

bridgekeeper bot commented Oct 20, 2020

openjdk bot commented Oct 20, 2020

mlbridge bot commented Oct 20, 2020 • edited Loading

Webrevs

dfuch commented Oct 22, 2020

iignatev commented Oct 22, 2020

iignatev commented Oct 23, 2020

dfuch left a comment

Choose a reason for hiding this comment

openjdk bot commented Oct 23, 2020 • edited Loading

iignatev commented Oct 23, 2020

openjdk bot commented Oct 23, 2020

Marcono1234 Oct 25, 2020

Choose a reason for hiding this comment

iignatev Oct 26, 2020

Choose a reason for hiding this comment

dfuch Oct 27, 2020 • edited Loading

Choose a reason for hiding this comment

iignatev commented Oct 20, 2020 •

edited by openjdk bot

Loading

mlbridge bot commented Oct 20, 2020 •

edited

Loading

openjdk bot commented Oct 23, 2020 •

edited

Loading

dfuch Oct 27, 2020 •

edited

Loading