Motivation: An IPv6 string can have a zone index which is followed by the '%' sign. When a user passes an IPv6 string with a zone index, NetUtil.createByteArrayFromIpAddressString() returns an incorrect value. Modification: - Strip the zone index before conversion Result: An IPv6 string with a zone index is decoded correctly.
Motivation: HTTP/2 codec does not properly test exception passed to exceptionCaught() for instanceof Http2Exception (since the exception will always be wrapped in a PipelineException), so it will never properly handle Http2Exceptions in the pipeline. Also if any streams are present, the connection close logic will execute twice when a pipeline exception. This is because the exception logic calls ctx.close() which then triggers the handleInActive() logic to execute. This clears all of the remaining streams and then attempts to run the closeListener logic (which has already been run). Modifications: Changed exceptionCaught logic to properly extract Http2Exception from the PipelineException. Also added logic to the closeListener so that is only run once. Changed Http2CodecUtil.toHttp2Exception() to avoid NPE when creating an exception with cause.getMessage(). Refactored Http2ConnectionHandler to more cleanly separate inbound and outbound flows (Http2ConnectionDecoder/Http2ConnectionEncoder). Added a test for verifying that a pipeline exception closes the connection. Result: Exception handling logic is tidied up.
Motivation: The java implementations for Inet6Address.getHostName() do not follow the RFC 5952 (http://tools.ietf.org/html/rfc5952#section-4) for recommended string representation. This introduces inconsistencies when integrating with other technologies that do follow the RFC. Modifications: -NetUtil.java to have another public static method to convert InetAddress to string. Inet4Address will use the java InetAddress.getHostAddress() implementation and there will be new code to implement the RFC 5952 IPV6 string conversion. -New unit tests to test the new method Result: Netty provides a RFC 5952 compliant string conversion method for IPV6 addresses
Motivation: Currently the Executor created by (Nio|Epoll)EventLoopGroup is not correctly shutdown. This might lead to resource shortages, due to resources not being freed asap. Modifications: If (Nio|Epoll)EventLoopGroup create their internal Executor via a constructor provided `ExecutorServiceFactory` object or via MultithreadEventLoopGroup.newDefaultExecutorService(...) the ExecutorService.shutdown() method will be called after (Nio|Epoll)EventLoopGroup is shutdown. ExecutorService.shutdown() will not be called if the Executor object was passed to the (Nio|Epoll)EventLoopGroup (that is, it was instantiated outside of Netty). Result: Correctly release resources on (Nio|Epoll)EventLoopGroup shutdown.
Motivation: Outbound flow control does not properly remove the head of queue after it's written. This will cause streams with multiple frames to get stuck and not send all of the data. Modifications: Modified the DefaultHttp2OutboundFlowController to properly remove the head of the pending write queue once a queued frame has been written. Added an integration test that sends a large message to verify that all DATA frames are properly collected at the other end. Result: Outbound flow control properly handles several queued messages.
Motivation: This fixes bug #2848 which caused Recycler to become unbounded and cache infinite number of objects with maxCapacity that's not a power of two. This can result in general sluggishness of the application and OutOfMemoryError. Modifications: The test for maxCapacity has been moved out of test to check if the buffer has filled. The buffer is now also capped at maxCapacity and cannot grow over it as it jumps from one power of two to the other. Additionally, a unit test was added to verify maxCapacity is honored even when it's not a power of two. Result: With these changes the user is able to use a custom maxCapacity number and not have it ignored. The unit test assures this bug will not repeat itself.
Motivation: To make it more easy to shutdown an EventExecutorGroup / EventLoopGroup we should let both of them extend AutoCloseable. Modifications: Let EventExecutorGroup extend AutoCloseable and impement it. Result: Easier shutdown of EventExecutorGroup and EventLoopGroup
Motivation: The recent changes f8bee2e to use a FJP introduced a regression on Android that cause a NPE. Modifications: - Use AtomicReferenceFieldUpdater so it works also on Android - Fix inspection warnings Result: Netty works again on Android too.
Motivation: Recently we changed the default value of SOMAXCONN that is used when we can not determine it by reading /proc/sys/net/core/somaxconn. While doing this we missed to update the javadocs to reflect the new default value that is used. Modifications: List correct default value in the javadocs of SOMAXCONN. Result: Correct javadocs.
Related issue: #2407 Motivation: The current fallback SOMAXCONN value is 3072. It is way too large comparing to the default SOMAXCONN value of popular OSes. Modifications: Decrease the fallback SOMAXCONN value to 128 or 200 depending on the current OS Result: Saner fallback value
Motivation: After a channel is created it's usually assigned to an EventLoop. During the lifetime of a Channel the EventLoop is then responsible for processing all I/O and compute tasks of the Channel. For various reasons (e.g. load balancing) a user might require the ability for a Channel to be assigned to another EventLoop during its lifetime. Modifications: Introduce under the hood changes that ensure that Netty's thread model is obeyed during and after the deregistration of a channel. Ensure that tasks (one time and periodic) are executed by the right EventLoop at all times. Result: A Channel can be deregistered from one and re-registered with another EventLoop.
Related issue: #2250 Motivation: Prior to this commit, Netty's non blocking EventLoops were each assigned a fixed thread by which all of the EventLoop's I/O and handler logic would be performed. While this is a fine approach for most users of Netty, some advanced users require more flexibility in scheduling the EventLoops. Modifications: Remove all direct usages of threads in MultithreadEventExecutorGroup, SingleThreadEventExecutor et al., and introduce an Executor abstraction instead. The way to think about this change is, that each iteration of an eventloop is now a task that gets scheduled in a ForkJoinPool. While the ForkJoinPool is the default, one also has the ability to plug in his/her own Executor (aka thread pool) into a EventLoop(Group). Result: Netty hands off thread management to a ForkJoinPool by default. Users can also provide their own thread pool implementation and get some control over scheduling Netty's EventLoops
Motivation: The calculation of the max wait time for HashedWheelTimerTest.testExecutionOnTime() was wrong and so the test sometimes failed. Modifications: Fix the max wait time. Result: No more test-failures
Motivation: We forgot to do a null check on the cause parameter of ChannelFuture.setFailure(cause) Modifications: Add a null check Result: Fixed issue: #2728
Motivation: We did various changes related to the ChannelOutboundBuffer in 4.0 branch. This commit port all of them over and so make sure our branches are synced in terms of these changes. Related to [#2734], [#2709], [#2729], [#2710] and [#2693] . Modification: Port all changes that was done on the ChannelOutboundBuffer. This includes the port of the following commits: - 73dfd7c - 997d8c3 - e282e50 - 5e5d1a5 - 8ee3575 - d6f0d12 - 16e5076 - 3f3e66c Result: - Less memory usage by ChannelOutboundBuffer - Same code as in 4.0 branch - Make it possible to use ChannelOutboundBuffer with Channel implementation that not extends AbstractChannel
…rse it Motivation: As /proc/sys/net/core/somaxconn does not exists on non-linux platforms you see a noisy stacktrace when debug level is enabled while the static method of NetUtil is executed. Modifications: Check if the file exists before try to parse it. Result: Less noisy logging on non-linux platforms.
…constants Related issue: #2354 Motivation: AbstractConstant.compareTo() can return 0 even if the specified constant object is not the same instance with 'this'. Modifications: - Compare the identityHashCode of constant first. If that fails, allocate a small direct buffer and use its memory address as a unique value. If the platform does not provide a way to get the memory address of a direct buffer, use a thread-local random value. - Signal cannot extend AbstractConstant. Use delegation. Result: It is practically impossible for AbstractConstant.compareTo() to return 0 for different constant objects.
Motivation: While benchmarking the native transport with gathering writes I noticed that it is quite slow. This is due the fact that we need to do a lot of array copies to get the buffers into the iov array. Modification: Introduce a new class calles IovArray which allows to fill buffers directly in a iov array that can be passed over to JNI without any array copies. This gives a nice optimization in terms of speed when doing gathering writes. Result: Big performance improvement when doing gathering writes. See the included benchmark... Before: [nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256 http://xxx:8080/plaintext Running 2m test @ http://xxx:8080/plaintext 16 threads and 256 connections Thread Stats Avg Stdev Max +/- Stdev Latency 23.44ms 16.37ms 259.57ms 91.77% Req/Sec 181.99k 31.69k 304.60k 78.12% 346544071 requests in 2.00m, 46.48GB read Requests/sec: 2887885.09 Transfer/sec: 396.59MB With this change: [nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256 http://xxx:8080/plaintext Running 2m test @ http://xxx:8080/plaintext 16 threads and 256 connections Thread Stats Avg Stdev Max +/- Stdev Latency 21.93ms 16.33ms 305.73ms 92.34% Req/Sec 194.56k 33.75k 309.33k 77.04% 369617503 requests in 2.00m, 49.57GB read Requests/sec: 3080169.65 Transfer/sec: 423.00MB
Motivation: Due some race-condition while handling canellation of TimerTasks it was possibleto corrupt the linked-list structure that is represent by HashedWheelBucket and so produce a NPE. Modification: Fix the problem by adding another MpscLinkedQueue which holds the cancellation tasks and process them on each tick. This allows to use no synchronization / locking at all while introduce a latency of max 1 tick before the TimerTask can be GC'ed. Result: No more NPE
- Rewrite with linear probing, no state array, compaction at cleanup - Optimize keys() and values() to not use reflection - Optimize hashCode() and equals() for efficient iteration - Fixed equals() to not return true for equals(null) - Optimize iterator to not allocate new Entry at each next() - Added toString() - Added some new unit tests
Motivation: Now Netty has a few problems with null values. Modifications: - Check HAProxyProxiedProtocol in HAProxyMessage constructor and throw NPE if it is null. If HAProxyProxiedProtocol is null we will set AddressFamily as null. So we will get NPE inside checkAddress(String, AddressFamily) and it won't be easy to understand why addrFamily is null. - Check File in DiskFileUpload.toString(). If File is null we will get NPE when calling toString() method. - Check Result<String> in MqttDecoder.decodeConnectionPayload(...). If !mqttConnectVariableHeader.isWillFlag() || !mqttConnectVariableHeader.hasUserName() || !mqttConnectVariableHeader.hasPassword() we will get NPE when we will try to create new instance of MqttConnectPayload. - Check Unsafe before calling unsafe.getClass() in PlatformDependent0 static block. - Removed unnecessary null check in WebSocket08FrameEncoder.encode(...). Because msg.content() can not return null. - Removed unnecessary null check in DefaultStompFrame(StompCommand) constructor. Because we have this check in the super class. - Removed unnecessary null checks in ConcurrentHashMapV8.removeTreeNode(TreeNode<K,V>). - Removed unnecessary null check in OioDatagramChannel.doReadMessages(List<Object>). Because tmpPacket.getSocketAddress() always returns new SocketAddress instance. - Removed unnecessary null check in OioServerSocketChannel.doReadMessages(List<Object>). Because socket.accept() always returns new Socket instance. - Pass Unpooled.buffer(0) instead of null inside CloseWebSocketFrame(boolean, int) constructor. If we will pass null we will get NPE in super class constructor. - Added throw new IllegalStateException in GlobalEventExecutor.awaitInactivity(long, TimeUnit) if it will be called before GlobalEventExecutor.execute(Runnable). Because now we will get NPE. IllegalStateException will be better in this case. - Fixed null check in OpenSslServerContext.setTicketKeys(byte). Now we throw new NPE if byte is not null. Result: Added new null checks when it is necessary, removed unnecessary null checks and fixed some NPE problems.
Motivation: Fix some typos in Netty. Modifications: - Fix potentially dangerous use of non-short-circuit logic in Recycler.transfer(Stack<?>). - Removed double 'the the' in javadoc of EmbeddedChannel. - Write to log an exception message if we can not get SOMAXCONN in the NetUtil's static block.
Modifications: - Added a static modifier for CompositeByteBuf.Component. This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary. - Removed unnecessary boxing/unboxing operations in HttpResponseDecoder, RtspResponseDecoder, PerMessageDeflateClientExtensionHandshaker and PerMessageDeflateServerExtensionHandshaker A boxed primitive is created from a String, just to extract the unboxed primitive value. - Removed unnecessary 3 times calculations in DiskAttribute.addContent(...). - Removed unnecessary checks if file exists before call mkdirs() in NativeLibraryLoader and PlatformDependent. Because the method mkdirs() has this check inside. - Removed unnecessary `instanceof AsciiString` check in StompSubframeAggregator.contentLength(StompHeadersSubframe) and StompSubframeDecoder.getContentLength(StompHeaders, long). Because StompHeaders.get(CharSequence) always returns java.lang.String.
Motivations: In our new version of HWT we used some kind of lazy cancelation of timeouts by put them back in the queue and let them pick up on the next tick. This multiple problems: - we may corrupt the MpscLinkedQueue if the task is used as tombstone - this sometimes lead to an uncessary delay especially when someone did executed some "heavy" logic in the TimeTask Modifications: Use a Lock per HashedWheelBucket for save and fast removal. Modifications: Cancellation of tasks can be done fast and so stuff can be GC'ed and no more infinite-loop possible
Motivation: When system is in short of entrophy, the initialization of ThreadLocalRandom can take at most 3 seconds. The initialization occurs when ThreadLocalRandom.current() is invoked first time, which might be much later than the moment when the application has started. If we start the initialization of ThreadLocalRandom as early as possible, we can reduce the perceived time taken for the retrieval. Modification: Begin the initialization of ThreadLocalRandom in InternalLoggerFactory, potentially one of the firstly initialized class in a Netty application. Make DefaultChannelId retrieve the current process ID before retrieving the current machine ID, because retrieval of a machine ID is more likely to use ThreadLocalRandom.current(). Use a dummy channel ID for EmbeddedChannel, which prevents many unit tests from creating a ThreadLocalRandom instance. Result: We gain extra 100ms at minimum for initialSeedUniquifier generation. If an application has its own initialization that takes long enough time and generates good amount of entrophy, it is very likely that we will gain a lot more.
- Sometimes useful to know it how long it takes from the log, to make sure it's not something else that is blocking.