ByteString introduced as AsciiString super class #3579

Scottmitch · 2015-04-03T21:34:01Z

Motivation:
The usage and code within AsciiString has exceeded the original design scope for this class. Its usage as a binary string is confusing and on the verge of violating interface assumptions in some spots.

Modifications:

ByteString will be created as a base class to AsciiString. All of the generic byte handling processing will live in ByteString and all the special character encoding will live in AsciiString.

Results:
The AsciiString interface will be clarified. Users of AsciiString can now be clear of the limitations the class imposes while users of the ByteString class don't have to live with those limitations.

Scottmitch · 2015-04-03T21:34:13Z

@trustin @nmittler @buchgr - FYI.

Scottmitch · 2015-04-03T21:34:40Z

codec-http2/src/main/java/io/netty/handler/codec/http2/CompressorHttp2ConnectionEncoder.java

-    protected EmbeddedChannel newContentCompressor(AsciiString contentEncoding) throws Http2Exception {
-        if (GZIP.equalsIgnoreCase(contentEncoding) || X_GZIP.equalsIgnoreCase(contentEncoding)) {
+    protected EmbeddedChannel newContentCompressor(ByteString contentEncoding) throws Http2Exception {
+        if (GZIP.equals(contentEncoding) || X_GZIP.equals(contentEncoding)) {


Note this change. equalsIgnoreCase doesn't fit well when dealing with binary strings. You have to think about encodings and translations...blah. HTTP/2 requires that header names are lower case. So a byte wise comparison should be sufficient, unless the user switches off the "force to lower case default"...in which case they are explicitly taking this risk.

Scottmitch · 2015-04-05T20:02:12Z

@netkins build

normanmaurer · 2015-04-06T16:35:01Z

codec-http/src/main/java/io/netty/handler/codec/http/DefaultHttpHeaders.java

+        private static ByteStringProcessor VALIDATE_NAME_PROCESSOR = new ByteStringProcessor() {
+            @Override
+            public boolean process(byte value) throws Exception {
+                // Check to see if the character is not an ASCII character


missing . on EOL

nmittler · 2015-04-07T20:04:37Z

codec/src/main/java/io/netty/handler/codec/ByteString.java

+ * {@link #array()}. Care must be taken when directly accessing the memory as that may invalidate assumptions that
+ * this object is immutable.
+ */
+public class ByteString {


Just curious, did you write this from scratch or is it a copy from another project?

As far as I know it is based on the old AsciiString that was written by @trustin . He based it on code from Apache Harmony

General comment, since you allow ByteString to be extended, would it make sense to mark many of the frequently called methods as final to help with optimization?

@normanmaurer +1. This is mostly the result of pulling all the encoding neutral stuff from AsciiString.

@nmittler - Yip. I made a bunch of them final.

nmittler · 2015-04-07T20:25:09Z

@Scottmitch looks great overall! Just a few questions/comments.

Scottmitch · 2015-04-07T22:26:26Z

@nmittler - Thanks for review! I think I got all your comments...feel free to take a look.

trustin · 2015-04-08T01:11:27Z

Thanks @Scottmitch! Will review today.

Scottmitch · 2015-04-08T04:03:00Z

@trustin - Sounds good. Ping me when you are finished.

trustin · 2015-04-10T03:12:30Z

codec/src/main/java/io/netty/handler/codec/AsciiString.java

@@ -706,83 +533,12 @@ public boolean equalsIgnoreCase(CharSequence string) {
        final byte[] value = this.value;
        final char[] buffer = new char[length];
        for (int i = 0, j = start; i < length; i++, j++) {
-            buffer[i] = (char) (value[j] & 0xFF);
+            buffer[i] = (char) (value[j]);


Same problem. We should not drop & 0xFF.

trustin · 2015-04-10T03:30:23Z

Moving ByteString, AsciiString and ByteVisitor to netty-common (since it's not limited to writing a codec anymore)
@trustin - We may be creating a circular dependency here. ByteString exposes methods that take ByteBuf types...and so we would have to add netty-buffer as a dependency of netty-common but the netty-buffer package already depends upon netty-common. We may have to introduce an abstraction layer...or move some other stuff....any suggestions?

Those methods are just shortcut methods. We could move them to ByteBufUtil?

trustin · 2015-04-10T03:33:28Z

codec/src/main/java/io/netty/handler/codec/ByteString.java

+     * Create a copy of the underlying storage from {@link value} into a byte array.
+     * The copy will start at {@link ByteBuffer#position()} and copy {@link ByteBuffer#remaining()} bytes.
+     */
+    public static byte[] getBytes(ByteBuffer value) {


Is there any reason these getBytes[] methods should be public? They look unrelated to ByteString and useful only for internal purposes.

I guess they don't have to be. The thought was incase you don't want to have a ByteString (which Ideally should be unmodifiable) object you could directly to byte[].

trustin · 2015-04-10T03:34:05Z

Review done. Please ping me for another round. Thank you for your patience!

Scottmitch · 2015-04-10T17:59:50Z

Moving ByteString, AsciiString and ByteVisitor to netty-common (since it's not limited to writing a codec anymore)

@trustin - We may be creating a circular dependency here. ByteString exposes methods that take ByteBuf types...and so we would have to add netty-buffer as a dependency of netty-common but the netty-buffer package already depends upon netty-common. We may have to introduce an abstraction layer...or move some other stuff....any suggestions?

Those methods are just shortcut methods. We could move them to ByteBufUtil?

Good idea. I had to manually do some conversions which I was leaning on ByteBuf to do for me. Please review. I'm wondering if I should worry about manually releasing the ByteBuffer object I allocate to do the conversion...

Scottmitch · 2015-04-10T18:21:01Z

common/src/main/java/io/netty/util/ByteString.java

+     * Create a copy of the underlying storage from {@link value} into a byte array.
+     * The copy will start at {@link ByteBuffer#position()} and copy {@link ByteBuffer#remaining()} bytes.
+     */
+    public static byte[] getBytes(ByteBuffer value) {


@trustin - Your comment was collapsed I think but the rational for having these public is if a user wanted to directly get back a modifiable byte[]. The "desired" use case for ByteString is to be "unmodifiable". I can make private though...

Let's make them private and consider moving them somewhere else and making them public in a later discussion.

+1

Am 14.04.2015 um 16:43 schrieb Scott Mitchell notifications@github.com:

In common/src/main/java/io/netty/util/ByteString.java:

public ByteString(CharSequence value, Charset charset) {

this.value = getBytes(value, charset);

}

/**

\* @see {@link #getBytes(CharSequence, Charset, int, int)}

*/

public ByteString(CharSequence value, Charset charset, int start, int length) {

this.value = getBytes(value, charset, start, length);

}

/**

\* Create a copy of the underlying storage from {@link value} into a byte array.

\* The copy will start at {@link ByteBuffer#position()} and copy {@link ByteBuffer#remaining()} bytes.

*/

public static byte[] getBytes(ByteBuffer value) {
SGTM

—
Reply to this email directly or view it on GitHub.

Scottmitch · 2015-04-10T18:21:58Z

@trustin - Ready for another round.

trustin · 2015-04-14T03:42:49Z

@Scottmitch Looks great! Please cherry-pick once you make the getBytes() methods private. If you are inclined to making them public, we could move it somewhere more appropriate.

Motivation: The usage and code within AsciiString has exceeded the original design scope for this class. Its usage as a binary string is confusing and on the verge of violating interface assumptions in some spots. Modifications: - ByteString will be created as a base class to AsciiString. All of the generic byte handling processing will live in ByteString and all the special character encoding will live in AsciiString. Results: The AsciiString interface will be clarified. Users of AsciiString can now be clear of the limitations the class imposes while users of the ByteString class don't have to live with those limitations.

Scottmitch · 2015-04-14T15:21:08Z

@trustin - Methods made private for now. As you said we can always increase visibility later. I'll pull this in after #3626.

Scottmitch · 2015-04-14T15:36:25Z

@buchgr - FYI. I'm going to pull this in after your #3626 PR. This may cause a bit of heart-ache for your headers optimization work, but hopefully not much if you are focused mostly on DefaultHeaders.

buchgr · 2015-04-14T15:46:50Z

@Scottmitch thanks! Will update once I get to work!

Scottmitch · 2015-04-15T00:08:23Z

PR #3631 will serve to review the forward-port into master.

Motivation: While forward porting #3579 there were a few areas that had not been previously back ported. Modifications: Backport the missed areas to ensure consistency. Result: More consistent 4.1 and master branches.

Scottmitch · 2015-04-15T00:11:50Z

Cherry-picked into 4.1 (9a7a85d and e36c143)

Scottmitch added the feature label Apr 3, 2015

Scottmitch self-assigned this Apr 3, 2015

Scottmitch added this to the 4.1.0.Beta5 milestone Apr 3, 2015

Scottmitch reviewed Apr 3, 2015
View reviewed changes

Scottmitch mentioned this pull request Apr 5, 2015

[Socket|Epoll]SslEchoTest failures (unexpected cipher and heap space error) #3590

Closed

normanmaurer reviewed Apr 6, 2015
View reviewed changes

Scottmitch mentioned this pull request Apr 6, 2015

Release 4.1.0.Beta5 #3588

Closed

Scottmitch force-pushed the ascii_string_refactor branch from 6373547 to afb4708 Compare April 6, 2015 22:49

nmittler reviewed Apr 7, 2015
View reviewed changes

trustin reviewed Apr 10, 2015
View reviewed changes

Scottmitch force-pushed the ascii_string_refactor branch 2 times, most recently from e9ec00b to 920189b Compare April 10, 2015 18:18

Scottmitch reviewed Apr 10, 2015
View reviewed changes

Scottmitch force-pushed the ascii_string_refactor branch from 920189b to 920c332 Compare April 14, 2015 15:20

This was referenced Apr 14, 2015

Improve performance of AsciiString.equals(Object). #3626

Closed

Case sensitivity in HTTP/2 Headers. #3630

Closed

ByteString introduced as AsciiString super class #3631

Closed

Scottmitch closed this Apr 15, 2015

Scottmitch deleted the ascii_string_refactor branch April 15, 2015 00:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ByteString introduced as AsciiString super class #3579

ByteString introduced as AsciiString super class #3579

Scottmitch commented Apr 3, 2015

Scottmitch commented Apr 3, 2015

Scottmitch Apr 3, 2015

Scottmitch commented Apr 5, 2015

normanmaurer Apr 6, 2015

nmittler Apr 7, 2015

normanmaurer Apr 7, 2015

nmittler Apr 7, 2015

Scottmitch Apr 7, 2015

Scottmitch Apr 7, 2015

nmittler commented Apr 7, 2015

Scottmitch commented Apr 7, 2015

trustin commented Apr 8, 2015

Scottmitch commented Apr 8, 2015

trustin Apr 10, 2015

trustin commented Apr 10, 2015

trustin Apr 10, 2015

Scottmitch Apr 10, 2015

trustin commented Apr 10, 2015

Scottmitch commented Apr 10, 2015

Scottmitch Apr 10, 2015

trustin Apr 14, 2015

Scottmitch Apr 14, 2015

normanmaurer Apr 14, 2015

Scottmitch commented Apr 10, 2015

trustin commented Apr 14, 2015

Scottmitch commented Apr 14, 2015

Scottmitch commented Apr 14, 2015

buchgr commented Apr 14, 2015

Scottmitch commented Apr 15, 2015

Scottmitch commented Apr 15, 2015

ByteString introduced as AsciiString super class #3579

ByteString introduced as AsciiString super class #3579

Conversation

Scottmitch commented Apr 3, 2015

Scottmitch commented Apr 3, 2015

Choose a reason for hiding this comment

Scottmitch commented Apr 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmittler commented Apr 7, 2015

Scottmitch commented Apr 7, 2015

trustin commented Apr 8, 2015

Scottmitch commented Apr 8, 2015

Choose a reason for hiding this comment

trustin commented Apr 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trustin commented Apr 10, 2015

Scottmitch commented Apr 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Scottmitch commented Apr 10, 2015

trustin commented Apr 14, 2015

Scottmitch commented Apr 14, 2015

Scottmitch commented Apr 14, 2015

buchgr commented Apr 14, 2015

Scottmitch commented Apr 15, 2015

Scottmitch commented Apr 15, 2015