Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8231640: (prop) Canonical property storage #5372

Closed
wants to merge 28 commits into from
Closed
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
85748cf
8231640: (prop) Canonical property storage
jaikiran Aug 29, 2021
641864e
use @implNote to explain the use of the environment variable
jaikiran Sep 7, 2021
64e2065
implement review suggestions:
jaikiran Sep 8, 2021
1ded17f
adjust testcases to verify the new semantics
jaikiran Sep 8, 2021
867ec99
update javadoc @implNote to match latest changes
jaikiran Sep 8, 2021
848ded8
Implement Stuart's suggestion on date format:
jaikiran Sep 9, 2021
5326b7c
update the new tests to match the new date format expectations and al…
jaikiran Sep 9, 2021
7736a8f
Implement Stuart's suggestion to use Collections instead of Arrays fo…
jaikiran Sep 9, 2021
c9d3cb8
update javadoc to match latest changes
jaikiran Sep 9, 2021
a29d0f0
dummy commit to trigger GitHub actions job to try and reproduce an un…
jaikiran Sep 9, 2021
a9b71d2
Implement Roger's suggestions:
jaikiran Sep 10, 2021
06ff3bd
jcheck fix - trailing whitespace in test
jaikiran Sep 10, 2021
1d24a3a
allow free-form text for java.util.Properties.storeDate system property
jaikiran Sep 10, 2021
c1dfb18
Update javadoc based on Stuart's inputs
jaikiran Sep 12, 2021
ff34ad5
Implement Roger's and the agreed upon decision to allow free-form tex…
jaikiran Sep 14, 2021
6447f9b
Merge latest from master branch
jaikiran Sep 14, 2021
9237466
unused imports
jaikiran Sep 14, 2021
315f3c8
update the javadoc to clarify how line terminator characters are hand…
jaikiran Sep 14, 2021
14711a9
Add a @implNote to specify the order in which the properties are writ…
jaikiran Sep 14, 2021
6f5f1be
Introduce a test to make sure backslash character in the system prope…
jaikiran Sep 14, 2021
7098a2c
Address review suggestions:
jaikiran Sep 15, 2021
79d1052
- Clarify how overriden Properties#entrySet() method impacts the orde…
jaikiran Sep 15, 2021
e2effb9
(Only doc/comment changes):
jaikiran Sep 16, 2021
eb31d28
- Implement Mark's suggestion in CSR, to rename java.util.Properties.…
jaikiran Sep 17, 2021
458c1fd
Implement Mark's suggestion in CSR to include the java.properties.dat…
jaikiran Sep 17, 2021
e350721
Roger's suggestion to reword the implSpec text
jaikiran Sep 18, 2021
ed5c221
reduce the line length of code comment
jaikiran Sep 22, 2021
944cbf1
Revert "Implement Mark's suggestion in CSR to include the java.proper…
jaikiran Sep 22, 2021
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
@@ -46,6 +46,7 @@
import java.util.function.BiFunction;
import java.util.function.Function;

import jdk.internal.util.StaticProperty;
import sun.nio.cs.ISO_8859_1;
import sun.nio.cs.UTF_8;

@@ -807,17 +808,25 @@ public void save(OutputStream out, String comments) {
* If the comments argument is not null, then an ASCII {@code #}
* character, the comments string, and a line separator are first written
* to the output stream. Thus, the {@code comments} can serve as an
* identifying comment. Any one of a line feed ('\n'), a carriage
* return ('\r'), or a carriage return followed immediately by a line feed
* in comments is replaced by a line separator generated by the {@code Writer}
* and if the next character in comments is not character {@code #} or
* character {@code !} then an ASCII {@code #} is written out
* after that line separator.
* identifying comment. Any one of a line feed ({@code \n}), a carriage
* return ({@code \r}), or a carriage return followed immediately by a line feed
* ({@code \r\n}) in comments is replaced by a
* {@link System#lineSeparator() line separator} and if the next
* character in comments is not character {@code #} or character {@code !} then
* an ASCII {@code #} is written out after that line separator.

This comment has been minimized.

@dfuch

dfuch Sep 14, 2021
Member

Should something be done for comments ending with \ (backslash) ? It might otherwise suppress the first property assignment that follows.

This comment has been minimized.

@jaikiran

jaikiran Sep 14, 2021
Author Member

There has been no change in how we deal with this aspect. The existing specification (stated in the load method) says:

* Properties are processed in terms of lines. There are two
* kinds of line, <i>natural lines</i> and <i>logical lines</i>.
* A natural line is defined as a line of
* characters that is terminated either by a set of line terminator
* characters ({@code \n} or {@code \r} or {@code \r\n})
* or by the end of the stream. A natural line may be either a blank line,
* a comment line, or hold all or some of a key-element pair. A logical
* line holds all the data of a key-element pair, which may be spread
* out across several adjacent natural lines by escaping
* the line terminator sequence with a backslash character
* {@code \}.  **Note that a comment line cannot be extended
* in this manner;**

(emphasis on that last sentence).
I'll anyway go ahead and add new tests around this to be sure that this works as advertised.

This comment has been minimized.

@dfuch

dfuch Sep 14, 2021
Member

Oh - great - thanks - verifying by a test that it also applies to the comment specified by java.util.Properties.storeDate would be good.

This comment has been minimized.

@jaikiran

jaikiran Sep 14, 2021
Author Member

Done. A new test StoreReproducibilityTest#testBackSlashInStoreDateValue has been added in the latest updated version of this PR. This test passes. Plus I checked the written out properties file, from these tests, for such values in java.util.Properties.storeDate and the content matches what the spec says.
Just for quick reference - a run of that test case with the "newline-plus-backslash...." system property value (cannot paste that exact string value from that test case because GitHub editor is messing up the special characters) generates output like below:

#some user specified comment
#newline-plus-backslash\
#c=d
a=b
* <p>
* Next, a comment line is always written, consisting of an ASCII
* {@code #} character, the current date and time (as if produced
* by the {@code toString} method of {@code Date} for the
* current time), and a line separator as generated by the {@code Writer}.
* If the {@systemProperty java.properties.date} is set on the command line
* and is non-empty (as determined by {@link String#isEmpty() String.isEmpty}),
* a comment line is written as follows.
* First, a {@code #} character is written, followed by the contents
* of the property, followed by a line separator. Any line terminator characters
* in the value of the system property are treated the same way as noted above
* for the comments argument.
* If the system property is not set or is empty, a comment line is written
* as follows.
* First, a {@code #} character is written, followed by the current date and time
* formatted as if by the {@link Date#toString() Date.toString} method,
* followed by a line separator.
* <p>
* Then every entry in this {@code Properties} table is
* written out, one per line. For each entry the key string is
@@ -833,6 +842,10 @@ public void save(OutputStream out, String comments) {
* After the entries have been written, the output stream is flushed.
* The output stream remains open after this method returns.
*
* @implSpec The keys and elements are written in the natural sort order
* of the keys in the {@code entrySet()} unless {@code entrySet()} is
* overridden by a subclass to return a different value than {@code super.entrySet()}.
*

This comment has been minimized.

@RogerRiggs

RogerRiggs Sep 9, 2021
Contributor

Update to refer to the property and add the {@systemProperty ...} javadoc tag so the property is listed in the system property reference.

This comment has been minimized.

@RogerRiggs

RogerRiggs Sep 17, 2021
Contributor

"different set implementation" as part of the spec may challenge the compatibility test developers to prove or disprove that statement.
The type of an instance is frequently understood to be the "implementation".
The visible type returned from Properties.entrySet() is SynchronizedSet.
Anyone can create one of those.
That statement might mislead a subclass into thinking they can not/must not return a SynchronizedSet if they want the built-in sorting.
I'm thinking that wording it in other term of the subclass might be better:
"...sort order of the keys in the entrySet() unless entrySet() is overridden by a subclass to return a different value than 'super.entrySet'(). "
The existing implementation is fine.

This comment has been minimized.

@dfuch

dfuch Sep 17, 2021
Member

I agree that this is a better formulation than what I suggested :-)

This comment has been minimized.

@jaikiran

jaikiran Sep 18, 2021
Author Member

Done. I've updated the PR to use Roger's suggested text and yes it's much more precise than what we had so far.

* @param writer an output character stream writer.
* @param comments a description of the property list.
* @throws IOException if writing this property list to the specified
@@ -903,12 +916,25 @@ private void store0(BufferedWriter bw, String comments, boolean escUnicode)
if (comments != null) {
writeComments(bw, comments);
}
bw.write("#" + new Date().toString());
bw.newLine();
writeDateComment(bw);

synchronized (this) {
for (Map.Entry<Object, Object> e : entrySet()) {
String key = (String)e.getKey();
String val = (String)e.getValue();
@SuppressWarnings("unchecked")
Collection<Map.Entry<String, String>> entries = (Set<Map.Entry<String, String>>) (Set) entrySet();
// entrySet() can be overridden by subclasses. Here we check to see if
// the returned instance type is the one returned by the Properties.entrySet()
// implementation. If yes, then we sort those entries in the natural order
// of their key. Else, we consider that the subclassed implementation may
// potentially have returned a differently ordered entries and so we just
// use the iteration order of the returned instance.
if (entries instanceof Collections.SynchronizedSet<?> ss

This comment has been minimized.

@AlanBateman

AlanBateman Sep 22, 2021
Contributor

Would you mind re-formatting the comment to keep the line length a bit more consistent with the rest of the code, is's just a bit annoying to have it wrapping.

This comment has been minimized.

@jaikiran

jaikiran Sep 22, 2021
Author Member

Done. I've updated the PR to reduce the line length of that code comment.

&& ss.c instanceof EntrySet) {
entries = new ArrayList<>(entries);

This comment has been minimized.

@RogerRiggs

RogerRiggs Sep 15, 2021
Contributor

From the description referring to a 'different instance', I expected to see == or != in this test.
Since Properties.entrySet() always returns a new synchronized set of a new EntrySet,
specifying that the underlying map is the same instance is more difficult.
Perhaps either the SynchronizedSet or the EntrySet instance could be cached and compared.

(typo: missing space after period... ".If yes")

This comment has been minimized.

@dfuch

dfuch Sep 15, 2021
Member

A custom subclass couldn't create an instance of EntrySet directly. Therefore if it wants to reorder the entries, it would have to return a new TreeSet or LinkedHashSet - or some other custom set implementation. If the result of calling entrySet() is an EntrySet wrapped in an UnmodifiableSet it's therefore legitimate to assume that the set hasn't been reordered, and that we can reorder it. Or am I missing something?

This comment has been minimized.

@jaikiran

jaikiran Sep 16, 2021
Author Member

I've updated the PR to clarify what this part of the code is doing. I took Daniel's input to reword the javadoc as well as tried to improve the code comment where this check is done. Essentially, we check for the "type" of the returned intstance and if we see that it's a SynchronizedSet whose underlying collection is a private class EntrySet then we know that it's the one created by Properties.entrySet(). Like Daniel says, subclasses won't be able to create or use this internal private class and if they did override the entrySet() to change the order of the entries they would have to do it by returning us a different "type" from that method.

((List<Map.Entry<String, String>>) entries).sort(Map.Entry.comparingByKey());
}
for (Map.Entry<String, String> e : entries) {
String key = e.getKey();
String val = e.getValue();
key = saveConvert(key, true, escUnicode);
/* No need to escape embedded and trailing spaces for value, hence
* pass false to flag.
@@ -921,6 +947,19 @@ private void store0(BufferedWriter bw, String comments, boolean escUnicode)
bw.flush();
}

private static void writeDateComment(BufferedWriter bw) throws IOException {
// value of java.properties.date system property isn't sensitive
// and so doesn't need any security manager checks to make the value accessible
// to the callers
String sysPropVal = StaticProperty.javaPropertiesDate();
if (sysPropVal != null && !sysPropVal.isEmpty()) {
writeComments(bw, sysPropVal);
} else {
bw.write("#" + new Date());
bw.newLine();
}

This comment has been minimized.

@RogerRiggs

RogerRiggs Sep 14, 2021
Contributor

Since there is no longer a need to format an arbitrary date, I'd suggest going back to the original Date.toString() code. It removes the need to replicate the format using DateTimeBuilder and is known to be the same as before.

If you keep the DateTimeFormat version, you would want "uuuu" instead of "yyyy". In java.time.
DateTimeFormatterBuilder uses slightly different pattern letter conventions as compared to SimpleDateFormat.

This comment has been minimized.

@jaikiran

jaikiran Sep 15, 2021
Author Member

Since there is no longer a need to format an arbitrary date, I'd suggest going back to the original Date.toString() code. It removes the need to replicate the format using DateTimeBuilder and is known to be the same as before.

Done. I pushed an update to the PR which switches back to using Date.toString() for the date comment. It also does a minor adjustment to the javadoc to clarify this behaviour.

}

/**
* Loads all of the properties represented by the XML document on the
* specified input stream into this properties table.
@@ -51,6 +51,7 @@
private static final String JAVA_IO_TMPDIR;
private static final String NATIVE_ENCODING;
private static final String FILE_ENCODING;
private static final String JAVA_PROPERTIES_DATE;

private StaticProperty() {}

@@ -67,6 +68,7 @@ private StaticProperty() {}
JDK_SERIAL_FILTER_FACTORY = getProperty(props, "jdk.serialFilterFactory", null);
NATIVE_ENCODING = getProperty(props, "native.encoding");
FILE_ENCODING = getProperty(props, "file.encoding");
JAVA_PROPERTIES_DATE = getProperty(props, "java.properties.date", null);
}

private static String getProperty(Properties props, String key) {
@@ -227,4 +229,16 @@ public static String nativeEncoding() {
public static String fileEncoding() {
return FILE_ENCODING;
}

/**
* Return the {@code java.properties.date} system property.
*
* <strong>{@link SecurityManager#checkPropertyAccess} is NOT checked
* in this method.</strong>
*
* @return the {@code java.properties.date} system property
*/
public static String javaPropertiesDate() {
return JAVA_PROPERTIES_DATE;
}
}