DRAFT - SIP on Netty #19310

wtlucy · 2021-11-10T14:05:08Z

For #15779

High Level Changes:

sipcontainer classes relying on the channel framework have been reimplemented on top of Netty
- WsByteBuffer replaced for[by] ByteBuf
- Channel Framework service and APIs replaced by Liberty Netty framework io.openliberty.netty.NettyFramework and io.netty.* APIs
- classes previously relying on channel framework functionality have been moved into new packages *.chfw.*
  - e.g. com/ibm/ws/sip/channel/resolver/dns/impl/Dns.java -> com/ibm/ws/sip/channel/resolver/dns/impl/chfw/Dns.java
  - new Netty implementations are in *.netty.* packages, e.g. com/ibm/ws/sip/channel/resolver/dns/impl/netty/Dns.java
- a new server.xml config element <sipContainer useNettyTransport="true/false"> has been added, which toggles between the chfw and netty implementations
- A few improvements needed to be made to the Liberty Netty framework:
  - support for outbound connection creation via NettyFramework.startOutbound()
  - TLS support via a new io.openliberty.netty.internal.tls.impl
    - this bundle provides a new service NettyTlsProvider, which provides APIs to build a io.netty.handler.ssl.SslContext from the active Liberty TLS configs, via com.ibm.websphere.ssl.JSSEHelper.
    - The SslContext can then be used to create Netty SslHandler

Detailed SIP component changes

DNS resolver (domainResolver)

Netty changes mostly contianed to com.ibm.ws.sip.channel.resolver.dns.impl.netty and com.ibm.ws.sip.channel.resolver.impl.netty
- com.ibm.ws.sip.channel.resolver.dns.impl.netty largely contains WsByteBuffer -> ByteBuf changes
- com.ibm.ws.sip.channel.resolver.impl.netty makes outbound TCP and UDP requests via SipResolverTcpTransport and SipResolverUdpTransport
  - those impls use NettyFramework to create their outbound connection bootstraps, and then NettyFramework.startOutbound() to start and register the connections
  - each impl provides a ChannelInitializerWrapper which sets up their pipelines - in both cases including a SimpleChannelInboundHandler which is used to pass new data to the correct methods
selection between chfw and Netty implementations occurs in com.ibm.ws.sip.container.resolver.DomainResolverImpl.getSipURILookup(SipURILookupCallbackImpl, SIPUri) and com.ibm.ws.sip.container.resolver.DomainResolverImpl.init()

SIP Endpoint (sipEndpoint)

Netty changes split between com.ibm.ws.sip.stack.transport.netty and com.ibm.ws.sip.stack.transport.sip.netty
com.ibm.ws.sip.stack.transport refactored rather than split completely (this might want to be revisited)
com.ibm.ws.sip.stack.transport.GenericEndpointImpl is the @Component which registers and manages endpoints
- Netty and chfw impls are determined in activate()
- previous chfw chain start logic is largely reused by Netty impl, including naming and config props
- endpoints started via GenericTCPChain and GenericUDPChain update()
  - bootstraps created by passing endpoint config to NettyFramework.create*Bootstrap()
  - registered and started via NettyFramework.start()
  - TLS endpoints covered by GenericTCPChain, and implemented by adding a io.netty.handler.ssl.SslHandler.SslHandler to the head of the ChannelInitializer pipeline

SIP Endpoint (sipEndpoint)

Netty changes split between com.ibm.ws.sip.stack.transport.netty and com.ibm.ws.sip.stack.transport.sip.netty
com.ibm.ws.sip.stack.transport refactored rather than split completely (this might want to be revisited)
com.ibm.ws.sip.stack.transport.GenericEndpointImpl is the @Component which registers and manages endpoints
- Netty and chfw impls are determined in activate()
- previous chfw chain start logic is largely reused by Netty impl, including naming and config props
- endpoints started via GenericTCPChain and GenericUDPChain update()
  - bootstraps created by passing endpoint config to NettyFramework.create*Bootstrap()
  - registered and started via NettyFramework.start()
  - TLS endpoints covered by GenericTCPChain, and implemented by adding a io.netty.handler.ssl.SslHandler.SslHandler to the head of the ChannelInitializer pipeline
SipOutboundConnLink used for outbound connections
- chfw and Netty impls swapped via com.ibm.ws.sip.stack.transaction.transport.SIPConnectionsModel.initSupportedTransports()

Remaining work

the packaging of these changes likely needs to be re-thought. Currently this PR results in new bundles being added to the runtime by a GA feature, which is the desired end result but is probably not OK for this type of code.
com.ibm.ws.sip.stack.transport.sip.netty.SipOutboundConnLink should be updated to use NettyFramework APIs to create a bootstrap and connect.

LibbyBot · 2021-11-10T14:05:16Z

Code analysis and actions

DO NOT DELETE THIS COMMENT.

76 product code files were changed.
Please describe in a separate comment how you tested your changes.

wtlucy · 2021-11-29T22:57:19Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-11-29T22:57:24Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-11-29T22:57:28Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_WAivsFFfEeysneUx1bJGow

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-11-30T16:30:23Z

The build wtlucy-19310-20211129-1504
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_WAivsFFfEeysneUx1bJGow
completed successfully!

wtlucy · 2021-11-30T21:06:25Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-11-30T21:06:34Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-11-30T21:06:40Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_B5efQFIZEeysneUx1bJGow

Target locations of links might be accessible only to IBM employees.

wtlucy · 2021-12-01T22:30:40Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-01T22:30:50Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-01T22:30:52Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_9TTVkFLtEeysneUx1bJGow

Target locations of links might be accessible only to IBM employees.

wtlucy · 2021-12-02T23:11:20Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-02T23:11:29Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-02T23:11:33Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_zmROgFO8EeysneUx1bJGow

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-03T18:43:02Z

The build wtlucy-19310-20211202-1912
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_zmROgFO8EeysneUx1bJGow
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_zmROgFO8EeysneUx1bJGow

wtlucy · 2021-12-06T21:25:56Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-06T21:26:02Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-06T21:26:06Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_vSvU0FbSEeyWaZwSyDrk3A

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-07T13:18:23Z

The build wtlucy-19310-20211206-1334
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_vSvU0FbSEeyWaZwSyDrk3A
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_vSvU0FbSEeyWaZwSyDrk3A

wtlucy · 2021-12-08T22:42:01Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-08T22:42:07Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-08T22:42:11Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_swoXwFhvEeyWaZwSyDrk3A

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-09T14:18:21Z

The build wtlucy-19310-20211208-1448
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_swoXwFhvEeyWaZwSyDrk3A
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_swoXwFhvEeyWaZwSyDrk3A

LibbyBot · 2021-12-15T00:12:39Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_VN_IcF0zEeySV8PQupfSfg

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-15T17:21:23Z

The build wtlucy-19310-20211214-1619
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_VN_IcF0zEeySV8PQupfSfg
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_VN_IcF0zEeySV8PQupfSfg

wtlucy · 2021-12-15T23:17:52Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-15T23:18:03Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-15T23:18:05Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_36_aUF30EeySV8PQupfSfg

Target locations of links might be accessible only to IBM employees.

wtlucy · 2021-12-17T20:32:11Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-17T20:32:17Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-17T20:32:21Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_DTnyQF9wEeySV8PQupfSfg

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-18T15:20:59Z

The build wtlucy-19310-20211217-1541
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_DTnyQF9wEeySV8PQupfSfg
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_DTnyQF9wEeySV8PQupfSfg

wtlucy · 2021-12-20T21:27:12Z

#build #spawn.fullfat.buckets=com.ibm.ws.sipcontainer_fat

LibbyBot · 2021-12-20T21:27:22Z

Please code review feature-related files, @OpenLiberty/delivery-approvers

LibbyBot · 2021-12-20T21:27:26Z

Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_PpqDMGHTEeywTM1lnhUXVw

Target locations of links might be accessible only to IBM employees.

LibbyBot · 2021-12-21T14:31:45Z

The build wtlucy-19310-20211220-1334
https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_PpqDMGHTEeywTM1lnhUXVw
completed and has errors or failures.

For help analyzing your personal build, go to https://cognitive.hursley.ibm.com/buildAnalysis.html?uuid=_PpqDMGHTEeywTM1lnhUXVw

wtlucy · 2021-12-21T14:36:59Z

The latest build has finally completely passed the sipcontainer_fat bucket!

hutchig · 2022-01-05T14:42:07Z

....ws.sipcontainer/src/com/ibm/ws/sip/channel/resolver/impl/netty/SipResolverTcpTransport.java

+
+            _bufferArray[0] = _lengthBuffer;
+            _bufferArray[1] = requestBuffer;
+            ByteBuf newBuf = Unpooled.copiedBuffer(_bufferArray);


Bill, was there a glitch with the composite buffer below that was solved by using an unpooled copied buffer above?

hutchig · 2022-02-11T15:22:51Z

dev/com.ibm.ws.sipcontainer/src/com/ibm/ws/sip/container/internal/SipContainerComponent.java

@@ -166,6 +172,9 @@ protected void modified(Map<String, Object> properties) {
 		if (c_logger.isTraceDebugEnabled())
 			c_logger.traceDebug("SipContainerComponent modified", properties);
 		PropertiesStore.getInstance().getProperties().updateProperties(properties);
+        useNetty = MetatypeUtils.parseBoolean(


hutchig · 2022-03-02T14:58:17Z

dev/com.ibm.ws.sipcontainer/src/com/ibm/ws/sip/container/resolver/DomainResolverImpl.java

-			SipResolverService.initialize(dsProps,m_chfw);
+
+			_useNetty = SipContainerComponent.useNetty();
+		     if (_useNetty) {


What does '_' prefix mean in this context?

isaacrivriv · 2023-06-13T19:55:11Z

Closing in favor of #24890

LibbyBot added the CLA Signed label Nov 10, 2021

wtlucy self-assigned this Nov 10, 2021

wtlucy force-pushed the netty_sip branch 3 times, most recently from 2169774 to 0ee0c0c Compare November 29, 2021 20:07

wtlucy force-pushed the netty_sip branch from 8b1a966 to ddcf4c4 Compare December 1, 2021 22:29

wtlucy force-pushed the netty_sip branch from ddcf4c4 to 6b69336 Compare December 2, 2021 22:58

wtlucy force-pushed the netty_sip branch from d39caad to 633ecf1 Compare December 15, 2021 23:17

wtlucy force-pushed the netty_sip branch from ee5d54f to 5625019 Compare December 17, 2021 20:31

wtlucy force-pushed the netty_sip branch from 5c7487d to 8cb1739 Compare December 20, 2021 15:01

TLS support for netty endpoints

19b33e0

wtlucy force-pushed the netty_sip branch from 3c1af5f to b0c0136 Compare December 20, 2021 20:38

add Netty support for sipcontainer

820985f

wtlucy force-pushed the netty_sip branch from b0c0136 to 820985f Compare December 20, 2021 21:26

hutchig reviewed Jan 5, 2022

View reviewed changes

pnicolucci unassigned wtlucy Jan 5, 2022

hutchig reviewed Feb 11, 2022

View reviewed changes

hutchig reviewed Mar 2, 2022

View reviewed changes

isaacrivriv mentioned this pull request Jun 13, 2023

Update Outbound SIP to use Netty Bundle #25478

Closed

isaacrivriv closed this Jun 13, 2023

LibbyBot added CLA Needed and removed CLA Signed labels Jun 13, 2023

DRAFT - SIP on Netty #19310

DRAFT - SIP on Netty #19310

Conversation

wtlucy commented Nov 10, 2021 • edited Loading

High Level Changes:

Detailed SIP component changes

DNS resolver (domainResolver)

SIP Endpoint (sipEndpoint)

SIP Endpoint (sipEndpoint)

Remaining work

LibbyBot commented Nov 10, 2021

Code analysis and actions

wtlucy commented Nov 29, 2021

LibbyBot commented Nov 29, 2021

LibbyBot commented Nov 29, 2021

LibbyBot commented Nov 30, 2021

wtlucy commented Nov 30, 2021

LibbyBot commented Nov 30, 2021

LibbyBot commented Nov 30, 2021

wtlucy commented Dec 1, 2021

LibbyBot commented Dec 1, 2021

LibbyBot commented Dec 1, 2021

wtlucy commented Dec 2, 2021

LibbyBot commented Dec 2, 2021

LibbyBot commented Dec 2, 2021

LibbyBot commented Dec 3, 2021

wtlucy commented Dec 6, 2021

LibbyBot commented Dec 6, 2021

LibbyBot commented Dec 6, 2021

LibbyBot commented Dec 7, 2021

wtlucy commented Dec 8, 2021

LibbyBot commented Dec 8, 2021

LibbyBot commented Dec 8, 2021

LibbyBot commented Dec 9, 2021

LibbyBot commented Dec 15, 2021

LibbyBot commented Dec 15, 2021

wtlucy commented Dec 15, 2021

LibbyBot commented Dec 15, 2021

LibbyBot commented Dec 15, 2021

wtlucy commented Dec 17, 2021

LibbyBot commented Dec 17, 2021

LibbyBot commented Dec 17, 2021

LibbyBot commented Dec 18, 2021

wtlucy commented Dec 20, 2021

LibbyBot commented Dec 20, 2021

LibbyBot commented Dec 20, 2021

LibbyBot commented Dec 21, 2021

wtlucy commented Dec 21, 2021

hutchig Jan 5, 2022

Choose a reason for hiding this comment

hutchig Feb 11, 2022

Choose a reason for hiding this comment

hutchig Mar 2, 2022

Choose a reason for hiding this comment

isaacrivriv commented Jun 13, 2023

wtlucy commented Nov 10, 2021 •

edited

Loading