Issue when creating segments / networks on NSX

[grehenl]

Hello everyone,

I have a lab with a vCenter / NSX and i am trying to add Cloudstack ontop. I have followed the docs to configure vCenter with NSX and the vCenter cluster has been added to Cloudstack successfully.

Cloudstack version: 4.21.0.0
NSX Version: 4.2.0.1.0.24210154
vCenter Version: 8.0.3

But when i try to create an isolated network the call fails with error:

unable to create NSX network ad00c442-6a64-4772-ab94-43141cc98337 due to: Failed API call to NSX controller

The strange thing is that in NSX the T1 Gateway is being created

Here are the logs from Cloudstack:

2025-10-27 07:10:55,010 DEBUG [o.a.c.s.NsxApiClient] (DirectAgent-22:[ctx-c6ddeea7]) (logid:ea3ad574) NSX Tier-1 gateway with name: D1-A2-Z1-N224 not found
2025-10-27 07:10:55,195 ERROR [o.a.c.s.NsxApiClient] (DirectAgent-161:[ctx-385a40c3]) (logid:ea3ad574) Error creating segment D1-A2-Z1-S224: General error has occurred.
2025-10-27 07:10:55,195 ERROR [o.a.c.r.NsxResource] (DirectAgent-161:[ctx-385a40c3]) (logid:ea3ad574) Failed to create network: adf
2025-10-27 07:10:55,195 ERROR [o.a.c.u.NsxControllerUtils] (qtp1245065720-24:[ctx-d382e352, ctx-25d4fd50]) (logid:ea3ad574) NSX API Command failed
2025-10-27 07:10:55,219 ERROR [c.c.a.ApiServer] (qtp1245065720-24:[ctx-d382e352, ctx-25d4fd50]) (logid:ea3ad574) unhandled exception executing api command: [Ljava.lang.String;@2edc4927 com.cloud.utils.exception.CloudRuntimeException: unable to create NSX network 2b833a44-68a5-4d63-b449-8129c4074289due to: Failed API call to NSX controller
at org.apache.cloudstack.service.NsxGuestNetworkGuru.setup(NsxGuestNetworkGuru.java:153)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.setupNetwork(NetworkOrchestrator.java:838)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:3098)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:2995)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2995)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2734)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2332)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2270)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.network.NetworkServiceImpl.commitNetwork(NetworkServiceImpl.java:2270)
at com.cloud.network.NetworkServiceImpl.createGuestNetwork(NetworkServiceImpl.java:1830)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at jdk.proxy3/jdk.proxy3.$Proxy200.createGuestNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.CreateNetworkCmd.execute(CreateNetworkCmd.java:431)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:854)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:668)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:416)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:193)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:190)
at com.cloud.api.ApiServlet.doPost(ApiServlet.java:149)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:840)

I have also tried to create a VPC which works ok and i can ping the public IP address it allocates but when i try to create a network inside the VPC it fails again with the same error.

So my guess is that there is something going on when Cloudstack tries to create a segment on NSX.

Is this a bug or have i forgotten to configure something?

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[nvazquez]

Hi @grehenl - can you please upload the entire logs for the createNetwork operation?

[grehenl]

Hi @nvazquez. Here are all the logs after the server received the create network operation:

command=createNetwork
response=json
zoneId=9579a160-77be-41ef-bbeb-da27bf164b81
name=test-net
networkOfferingId=1ce43b6c-067e-4024-b53e-71a59edf08e5
account=admin
domainid=c1defa80-adca-11f0-9e4a-005056abaea2
sessionkey=3mZdG6NAsfW9c-4UGnOstx5yfCU

2025-10-27 15:02:53,749 DEBUG [c.c.a.ApiServer] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}]' is allowed to perform API calls: 0.0.0.0/0,::/0
2025-10-27 15:02:53,753 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Account for user id 170559ee-adcb-11f0-9e4a-005056abaea2 is Root Admin or Domain Admin, all APIs are allowed.
2025-10-27 15:02:53,753 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2025-10-27 15:02:53,753 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) API rate limiting is disabled. We will not use ApiRateLimitService.
2025-10-27 15:02:53,784 DEBUG [c.c.u.AccountManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Access granted to Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}] to Domain {"id":1,"name":"ROOT","path":"/","uuid":"c1defa80-adca-11f0-9e4a-005056abaea2"} by AffinityGroupAccessChecker
2025-10-27 15:02:53,800 DEBUG [c.c.u.AccountManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Access granted to Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}] to NetworkOffering {"id":20,"name":"DefaultNATNSXNetworkOffering","trafficType":"Guest","uuid":"1ce43b6c-067e-4024-b53e-71a59edf08e5"} by AffinityGroupAccessChecker
2025-10-27 15:02:53,856 DEBUG [c.c.n.g.BigSwitchBcfGuestNetworkGuru] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Refusing to design this network, the physical isolation type is not BCF_SEGMENT
2025-10-27 15:02:53,860 DEBUG [o.a.c.n.c.m.ContrailGuru] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Refusing to design this network
2025-10-27 15:02:53,864 DEBUG [o.a.c.s.NetrisGuestNetworkGuru] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Refusing to design this network
2025-10-27 15:02:53,925 DEBUG [o.a.c.s.NsxGuestNetworkGuru] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Creating a Tier 1 Gateway for the network Network {"id": 225, "name": "test-net", "uuid": "c81c54d7-0b13-458a-8935-e1924b1c23a8", "networkofferingid": 20} before creating the NSX segment
2025-10-27 15:02:53,932 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Wait time setting on org.apache.cloudstack.agent.api.CreateNsxTier1GatewayCommand is 1800 seconds
2025-10-27 15:02:53,936 DEBUG [c.c.a.m.ClusteredDirectAgentAttache] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 9-7464716382366597122: Routed from 345051446858
2025-10-27 15:02:53,936 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597122: Sending { Cmd , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.agent.api.CreateNsxTier1GatewayCommand":{"networkResourceId":"225","networkResourceName":"test-net","isResourceVpc":"false","sourceNatEnabled":"true","zoneId":"1","accountId":"2","domainId":"1","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:53,936 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597122: Executing: { Cmd , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.agent.api.CreateNsxTier1GatewayCommand":{"networkResourceId":"225","networkResourceName":"test-net","isResourceVpc":"false","sourceNatEnabled":"true","zoneId":"1","accountId":"2","domainId":"1","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:53,937 DEBUG [c.c.a.m.D.Task] (DirectAgent-40:[ctx-ab9ae9ae]) (logid:6a6ff78d) Seq 9-7464716382366597122: Executing request
2025-10-27 15:02:54,055 DEBUG [o.a.c.s.NsxApiClient] (DirectAgent-40:[ctx-ab9ae9ae]) (logid:1742fc38) NSX Tier-1 gateway with name: D1-A2-Z1-N225 not found
2025-10-27 15:02:54,751 DEBUG [c.c.a.ApiServer] (qtp1245065720-23:[ctx-6bea88a7, ctx-337f861b]) (logid:93860308) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}]' is allowed to perform API calls: 0.0.0.0/0,::/0
2025-10-27 15:02:54,755 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1245065720-23:[ctx-6bea88a7, ctx-337f861b]) (logid:93860308) Account for user id 170559ee-adcb-11f0-9e4a-005056abaea2 is Root Admin or Domain Admin, all APIs are allowed.
2025-10-27 15:02:54,755 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1245065720-23:[ctx-6bea88a7, ctx-337f861b]) (logid:93860308) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2025-10-27 15:02:54,755 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1245065720-23:[ctx-6bea88a7, ctx-337f861b]) (logid:93860308) API rate limiting is disabled. We will not use ApiRateLimitService.
2025-10-27 15:02:54,756 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-8ea268ab]) (logid:666c44ea) No inactive management server node found
2025-10-27 15:02:54,756 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-8ea268ab]) (logid:666c44ea) Peer scan is finished. profiler: Done. Duration: 7ms , profilerQueryActiveList: Done. Duration: 1ms, , profilerSyncClusterInfo: Done. Duration: 2ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 4ms
2025-10-27 15:02:55,205 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-1:[ctx-7b389695]) (logid:34a1eec6) Scan hung worker VM to recycle
2025-10-27 15:02:55,280 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-1:[ctx-7b389695]) (logid:34a1eec6) Ping from [id: 2, uuid: c9f24c1e-5e91-4522-921e-b6be6cca796f, name: esxi-06.lab.local]
2025-10-27 15:02:55,287 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-1:[ctx-7b389695]) (logid:34a1eec6) Process host VM state report from ping process. host: Host {"id":2,"name":"esxi-06.lab.local","type":"Routing","uuid":"c9f24c1e-5e91-4522-921e-b6be6cca796f"}
2025-10-27 15:02:55,295 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-1:[ctx-7b389695]) (logid:34a1eec6) Process VM state report. Host {"id":2,"name":"esxi-06.lab.local","type":"Routing","uuid":"c9f24c1e-5e91-4522-921e-b6be6cca796f"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:55,297 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-1:[ctx-7b389695]) (logid:34a1eec6) Done with process of VM state report. host: Host {"id":2,"name":"esxi-06.lab.local","type":"Routing","uuid":"c9f24c1e-5e91-4522-921e-b6be6cca796f"}
2025-10-27 15:02:55,367 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:[ctx-aa2d4555]) (logid:f8b364ee) Scan hung worker VM to recycle
2025-10-27 15:02:55,429 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-2:[ctx-aa2d4555]) (logid:f8b364ee) Ping from [id: 3, uuid: e282b75e-22c7-4f83-bcb6-9115d462a6d3, name: esxi-05.lab.local]
2025-10-27 15:02:55,432 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-2:[ctx-aa2d4555]) (logid:f8b364ee) Process host VM state report from ping process. host: Host {"id":3,"name":"esxi-05.lab.local","type":"Routing","uuid":"e282b75e-22c7-4f83-bcb6-9115d462a6d3"}
2025-10-27 15:02:55,434 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-2:[ctx-aa2d4555]) (logid:f8b364ee) Process VM state report. Host {"id":3,"name":"esxi-05.lab.local","type":"Routing","uuid":"e282b75e-22c7-4f83-bcb6-9115d462a6d3"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:55,435 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-2:[ctx-aa2d4555]) (logid:f8b364ee) Done with process of VM state report. host: Host {"id":3,"name":"esxi-05.lab.local","type":"Routing","uuid":"e282b75e-22c7-4f83-bcb6-9115d462a6d3"}
2025-10-27 15:02:55,464 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) Scan hung worker VM to recycle
2025-10-27 15:02:55,523 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) Ping from [id: 4, uuid: c4ae8fe7-1ce2-4367-929b-66d59bb40e8d, name: esxi-04.lab.local]
2025-10-27 15:02:55,530 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) Process host VM state report from ping process. host: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}
2025-10-27 15:02:55,536 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) Process VM state report. Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}, number of records in report: 2. VMs: [1126:PowerOn, 1127:PowerOn]]
2025-10-27 15:02:55,551 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) VM state report is updated. Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}, VM instance {"id":1126,"instanceName":"s-1126-VM","state":"Running","type":"SecondaryStorageVm","uuid":"3ac290fd-c7a4-448d-aa63-4a32a3702f3d"}, power state: PowerOn
2025-10-27 15:02:55,559 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-5:[ctx-438ec010]) (logid:f9d7fc68) HA health check task is running...
2025-10-27 15:02:55,565 DEBUG [c.c.c.CapacityManagerImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) VM instance {"id":1126,"instanceName":"s-1126-VM","state":"Running","type":"SecondaryStorageVm","uuid":"3ac290fd-c7a4-448d-aa63-4a32a3702f3d"} state transited from [Running] to [Running] with event [FollowAgentPowerOnReport]. VM's original host: Host {"id":8,"name":"esxi-03.lab.local","type":"Routing","uuid":"534829dc-7770-4899-a920-db3ff90bbeb7"}, new host: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}, host before state transition: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}
2025-10-27 15:02:55,567 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) VM state report is updated. Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}, VM instance {"id":1127,"instanceName":"r-1127-VM","state":"Running","type":"DomainRouter","uuid":"3dc7bf90-4db3-43ad-a5e3-aee8d933a0c5"}, power state: PowerOn
2025-10-27 15:02:55,577 DEBUG [c.c.c.CapacityManagerImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) VM instance {"id":1127,"instanceName":"r-1127-VM","state":"Running","type":"DomainRouter","uuid":"3dc7bf90-4db3-43ad-a5e3-aee8d933a0c5"} state transited from [Running] to [Running] with event [FollowAgentPowerOnReport]. VM's original host: Host {"id":6,"name":"esxi-02.lab.local","type":"Routing","uuid":"cc60c607-ceb2-4149-89a6-f9171638e52e"}, new host: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}, host before state transition: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}
2025-10-27 15:02:55,579 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-3:[ctx-f877ddc6]) (logid:5684b607) Done with process of VM state report. host: Host {"id":4,"name":"esxi-04.lab.local","type":"Routing","uuid":"c4ae8fe7-1ce2-4367-929b-66d59bb40e8d"}
2025-10-27 15:02:55,601 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) Scan hung worker VM to recycle
2025-10-27 15:02:55,630 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-648d5223]) (logid:d2b620bd) No inactive management server node found
2025-10-27 15:02:55,630 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-648d5223]) (logid:d2b620bd) Peer scan is finished. profiler: Done. Duration: 3ms , profilerQueryActiveList: Done. Duration: 1ms, , profilerSyncClusterInfo: Done. Duration: 1ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 0ms
2025-10-27 15:02:55,662 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) Ping from [id: 5, uuid: 98ef0113-d994-438a-9e25-3e178a9924c9, name: esxi-07.lab.local]
2025-10-27 15:02:55,664 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) Process host VM state report from ping process. host: Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}
2025-10-27 15:02:55,666 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) Process VM state report. Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}, number of records in report: 1. VMs: [1125:PowerOn]]
2025-10-27 15:02:55,675 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) VM state report is updated. Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}, VM instance {"id":1125,"instanceName":"v-1125-VM","state":"Running","type":"ConsoleProxy","uuid":"8403860d-cf0a-4345-8fca-95ad384d223f"}, power state: PowerOn
2025-10-27 15:02:55,688 DEBUG [c.c.c.CapacityManagerImpl] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) VM instance {"id":1125,"instanceName":"v-1125-VM","state":"Running","type":"ConsoleProxy","uuid":"8403860d-cf0a-4345-8fca-95ad384d223f"} state transited from [Running] to [Running] with event [FollowAgentPowerOnReport]. VM's original host: Host {"id":6,"name":"esxi-02.lab.local","type":"Routing","uuid":"cc60c607-ceb2-4149-89a6-f9171638e52e"}, new host: Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}, host before state transition: Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}
2025-10-27 15:02:55,690 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-4:[ctx-c330c7b6]) (logid:d89a660c) Done with process of VM state report. host: Host {"id":5,"name":"esxi-07.lab.local","type":"Routing","uuid":"98ef0113-d994-438a-9e25-3e178a9924c9"}
2025-10-27 15:02:55,768 WARN [c.c.a.m.D.PingTask] (DirectAgentCronJob-5:[ctx-9ff1244f]) (logid:ad06f821) Unable to get current status on [id: 12, uuid: 31ac5615-badc-40f7-ac85-78db365bdde8, name: NSX]
2025-10-27 15:02:55,769 WARN [c.c.a.m.D.PingTask] (DirectAgentCronJob-6:[ctx-879622cd]) (logid:78b4f78a) Unable to get current status on [id: 1, uuid: 77cd1464-3cb0-4791-a75d-8c0abb239e0b, name: NSX]
2025-10-27 15:02:55,963 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-7:[ctx-d2e4a925]) (logid:65e68863) Scan hung worker VM to recycle
2025-10-27 15:02:56,000 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-8:[ctx-907cbca1]) (logid:0f1ed4b9) Scan hung worker VM to recycle
2025-10-27 15:02:56,027 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-7:[ctx-d2e4a925]) (logid:65e68863) Ping from [id: 7, uuid: 50d0c767-c54a-49c7-884b-a537f4b4d62f, name: esxi-01.lab.local]
2025-10-27 15:02:56,032 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:[ctx-d2e4a925]) (logid:65e68863) Process host VM state report from ping process. host: Host {"id":7,"name":"esxi-01.lab.local","type":"Routing","uuid":"50d0c767-c54a-49c7-884b-a537f4b4d62f"}
2025-10-27 15:02:56,037 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:[ctx-d2e4a925]) (logid:65e68863) Process VM state report. Host {"id":7,"name":"esxi-01.lab.local","type":"Routing","uuid":"50d0c767-c54a-49c7-884b-a537f4b4d62f"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:56,039 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:[ctx-d2e4a925]) (logid:65e68863) Done with process of VM state report. host: Host {"id":7,"name":"esxi-01.lab.local","type":"Routing","uuid":"50d0c767-c54a-49c7-884b-a537f4b4d62f"}
2025-10-27 15:02:56,069 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-8:[ctx-907cbca1]) (logid:0f1ed4b9) Ping from [id: 6, uuid: cc60c607-ceb2-4149-89a6-f9171638e52e, name: esxi-02.lab.local]
2025-10-27 15:02:56,072 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-8:[ctx-907cbca1]) (logid:0f1ed4b9) Process host VM state report from ping process. host: Host {"id":6,"name":"esxi-02.lab.local","type":"Routing","uuid":"cc60c607-ceb2-4149-89a6-f9171638e52e"}
2025-10-27 15:02:56,073 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-9:[ctx-f015d0cb]) (logid:f737acb6) Scan hung worker VM to recycle
2025-10-27 15:02:56,074 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-8:[ctx-907cbca1]) (logid:0f1ed4b9) Process VM state report. Host {"id":6,"name":"esxi-02.lab.local","type":"Routing","uuid":"cc60c607-ceb2-4149-89a6-f9171638e52e"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:56,076 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-8:[ctx-907cbca1]) (logid:0f1ed4b9) Done with process of VM state report. host: Host {"id":6,"name":"esxi-02.lab.local","type":"Routing","uuid":"cc60c607-ceb2-4149-89a6-f9171638e52e"}
2025-10-27 15:02:56,124 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-9:[ctx-f015d0cb]) (logid:f737acb6) Ping from [id: 8, uuid: 534829dc-7770-4899-a920-db3ff90bbeb7, name: esxi-03.lab.local]
2025-10-27 15:02:56,126 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-9:[ctx-f015d0cb]) (logid:f737acb6) Process host VM state report from ping process. host: Host {"id":8,"name":"esxi-03.lab.local","type":"Routing","uuid":"534829dc-7770-4899-a920-db3ff90bbeb7"}
2025-10-27 15:02:56,129 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-9:[ctx-f015d0cb]) (logid:f737acb6) Process VM state report. Host {"id":8,"name":"esxi-03.lab.local","type":"Routing","uuid":"534829dc-7770-4899-a920-db3ff90bbeb7"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:56,130 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-9:[ctx-f015d0cb]) (logid:f737acb6) Done with process of VM state report. host: Host {"id":8,"name":"esxi-03.lab.local","type":"Routing","uuid":"534829dc-7770-4899-a920-db3ff90bbeb7"}
2025-10-27 15:02:56,205 INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-10:[ctx-09b1ba96]) (logid:1cfe5863) Scan hung worker VM to recycle
2025-10-27 15:02:56,259 DEBUG [c.c.a.m.D.PingTask] (DirectAgentCronJob-10:[ctx-09b1ba96]) (logid:1cfe5863) Ping from [id: 9, uuid: 53be1dc7-a2ea-4c1a-a973-584b17cb7ce0, name: esxi-08.lab.local]
2025-10-27 15:02:56,262 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-10:[ctx-09b1ba96]) (logid:1cfe5863) Process host VM state report from ping process. host: Host {"id":9,"name":"esxi-08.lab.local","type":"Routing","uuid":"53be1dc7-a2ea-4c1a-a973-584b17cb7ce0"}
2025-10-27 15:02:56,264 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-10:[ctx-09b1ba96]) (logid:1cfe5863) Process VM state report. Host {"id":9,"name":"esxi-08.lab.local","type":"Routing","uuid":"53be1dc7-a2ea-4c1a-a973-584b17cb7ce0"}, number of records in report: 0. VMs: []]
2025-10-27 15:02:56,265 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-10:[ctx-09b1ba96]) (logid:1cfe5863) Done with process of VM state report. host: Host {"id":9,"name":"esxi-08.lab.local","type":"Routing","uuid":"53be1dc7-a2ea-4c1a-a973-584b17cb7ce0"}
2025-10-27 15:02:56,614 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-10:[]) (logid:) SeqA 10-63262: Seq 10-63262: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"1125","_loadInfo":"{
"connections": [],
"removedSessions": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:56,620 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-10:[]) (logid:) SeqA 10-: Sending 63262
2025-10-27 15:02:56,965 DEBUG [c.c.a.m.D.Task] (DirectAgent-40:[ctx-ab9ae9ae]) (logid:1742fc38) Seq 9-7464716382366597122: Response Received:
2025-10-27 15:02:56,965 DEBUG [c.c.a.t.Request] (DirectAgent-40:[ctx-ab9ae9ae]) (logid:1742fc38) Seq 12-7464716382366597122: Processing: { Ans: , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 10, [{"org.apache.cloudstack.NsxAnswer":{"objectExists":"false","result":"true","details":"","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:56,965 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597122: Received: { Ans: , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 10, { NsxAnswer } }
2025-10-27 15:02:56,966 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Details from executing class org.apache.cloudstack.agent.api.CreateNsxTier1GatewayCommand:
2025-10-27 15:02:56,969 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Wait time setting on org.apache.cloudstack.agent.api.CreateNsxSegmentCommand is 1800 seconds
2025-10-27 15:02:56,971 DEBUG [c.c.a.m.ClusteredDirectAgentAttache] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 9-7464716382366597123: Routed from 345051446858
2025-10-27 15:02:56,972 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597123: Sending { Cmd , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.agent.api.CreateNsxSegmentCommand":{"networkId":"225","networkName":"test-net","networkGateway":"10.1.1.1","networkCidr":"10.1.1.0/24","zoneId":"1","accountId":"2","domainId":"1","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:56,972 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597123: Executing: { Cmd , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.agent.api.CreateNsxSegmentCommand":{"networkId":"225","networkName":"test-net","networkGateway":"10.1.1.1","networkCidr":"10.1.1.0/24","zoneId":"1","accountId":"2","domainId":"1","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:56,972 DEBUG [c.c.a.m.D.Task] (DirectAgent-22:[ctx-d8c9a601]) (logid:98393316) Seq 9-7464716382366597123: Executing request
2025-10-27 15:02:57,130 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-4633ccf2]) (logid:234a89c3) No inactive management server node found
2025-10-27 15:02:57,130 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-4633ccf2]) (logid:234a89c3) Peer scan is finished. profiler: Done. Duration: 3ms , profilerQueryActiveList: Done. Duration: 1ms, , profilerSyncClusterInfo: Done. Duration: 1ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 0ms
2025-10-27 15:02:58,312 ERROR [o.a.c.s.NsxApiClient] (DirectAgent-22:[ctx-d8c9a601]) (logid:1742fc38) Error creating segment D1-A2-Z1-S225: General error has occurred.
2025-10-27 15:02:58,312 ERROR [o.a.c.r.NsxResource] (DirectAgent-22:[ctx-d8c9a601]) (logid:1742fc38) Failed to create network: test-net
2025-10-27 15:02:58,313 DEBUG [c.c.a.m.D.Task] (DirectAgent-22:[ctx-d8c9a601]) (logid:1742fc38) Seq 9-7464716382366597123: Response Received:
2025-10-27 15:02:58,313 DEBUG [c.c.a.t.Request] (DirectAgent-22:[ctx-d8c9a601]) (logid:1742fc38) Seq 12-7464716382366597123: Processing: { Ans: , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 10, [{"org.apache.cloudstack.NsxAnswer":{"objectExists":"false","result":"false","details":"Exception: com.cloud.utils.exception.CloudRuntimeException
Message: Error creating segment D1-A2-Z1-S225: General error has occurred.
Stack: com.cloud.utils.exception.CloudRuntimeException: Error creating segment D1-A2-Z1-S225: General error has occurred.
at org.apache.cloudstack.resource.NsxResource.executeRequest(NsxResource.java:358)
at org.apache.cloudstack.resource.NsxResource.executeRequest(NsxResource.java:114)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:306)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
","wait":"0","bypassHostMaintenance":"false"}}] }
2025-10-27 15:02:58,313 DEBUG [c.c.a.t.Request] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Seq 12-7464716382366597123: Received: { Ans: , MgmtId: 345051446858, via: 12(NSX), Ver: v1, Flags: 10, { NsxAnswer } }
2025-10-27 15:02:58,313 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Details from executing class org.apache.cloudstack.agent.api.CreateNsxSegmentCommand: Exception: com.cloud.utils.exception.CloudRuntimeException
Message: Error creating segment D1-A2-Z1-S225: General error has occurred.
Stack: com.cloud.utils.exception.CloudRuntimeException: Error creating segment D1-A2-Z1-S225: General error has occurred.
at org.apache.cloudstack.resource.NsxResource.executeRequest(NsxResource.java:358)
at org.apache.cloudstack.resource.NsxResource.executeRequest(NsxResource.java:114)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:306)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)

2025-10-27 15:02:58,313 ERROR [o.a.c.u.NsxControllerUtils] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) NSX API Command failed
2025-10-27 15:02:58,314 DEBUG [o.a.c.e.o.NetworkOrchestrator] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Releasing lock for Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}]
2025-10-27 15:02:58,332 DEBUG [c.c.u.d.T.Transaction] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) Rolling back the transaction: Time = 4501 Name = qtp1245065720-426; called by -TransactionLegacy.rollback:905-TransactionLegacy.removeUpTo:848-TransactionLegacy.close:672-Transaction.execute:36-Transaction.execute:45-NetworkOrchestrator.createGuestNetwork:2995-NetworkOrchestrator.createGuestNetwork:2734-NetworkServiceImpl$4.doInTransaction:2332-NetworkServiceImpl$4.doInTransaction:2270-Transaction.execute:38-NetworkServiceImpl.commitNetwork:2270-NetworkServiceImpl.createGuestNetwork:1830
2025-10-27 15:02:58,347 ERROR [c.c.a.ApiServer] (qtp1245065720-426:[ctx-f4f68471, ctx-a6fa8ccf]) (logid:1742fc38) unhandled exception executing api command: [Ljava.lang.String;@74485e76 com.cloud.utils.exception.CloudRuntimeException: unable to create NSX network 28aba938-439d-4232-8c51-16134e59211fdue to: Failed API call to NSX controller
at org.apache.cloudstack.service.NsxGuestNetworkGuru.setup(NsxGuestNetworkGuru.java:153)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.setupNetwork(NetworkOrchestrator.java:838)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:3098)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:2995)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2995)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2734)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2332)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2270)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.network.NetworkServiceImpl.commitNetwork(NetworkServiceImpl.java:2270)
at com.cloud.network.NetworkServiceImpl.createGuestNetwork(NetworkServiceImpl.java:1830)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at jdk.proxy3/jdk.proxy3.$Proxy200.createGuestNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.CreateNetworkCmd.execute(CreateNetworkCmd.java:431)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:854)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:668)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:416)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:193)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:190)
at com.cloud.api.ApiServlet.doPost(ApiServlet.java:149)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:840)

command=createNetwork
response=json
zoneId=9579a160-77be-41ef-bbeb-da27bf164b81
name=test-net
networkOfferingId=1ce43b6c-067e-4024-b53e-71a59edf08e5
account=admin
domainid=c1defa80-adca-11f0-9e4a-005056abaea2
sessionkey=3mZdG6NAsfW9c-4UGnOstx5yfCU

2025-10-27 15:02:58,628 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-25814cfe]) (logid:94cb7355) No inactive management server node found
2025-10-27 15:02:58,628 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-25814cfe]) (logid:94cb7355) Peer scan is finished. profiler: Done. Duration: 2ms , profilerQueryActiveList: Done. Duration: 0ms, , profilerSyncClusterInfo: Done. Duration: 0ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 0ms
2025-10-27 15:02:59,463 DEBUG [c.c.a.ApiServer] (qtp1245065720-21:[ctx-42c67ffd, ctx-dc541641]) (logid:ece62172) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"1704d489-adcb-11f0-9e4a-005056abaea2"}]' is allowed to perform API calls: 0.0.0.0/0,::/0
2025-10-27 15:02:59,468 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1245065720-21:[ctx-42c67ffd, ctx-dc541641]) (logid:ece62172) Account for user id 170559ee-adcb-11f0-9e4a-005056abaea2 is Root Admin or Domain Admin, all APIs are allowed.
2025-10-27 15:02:59,468 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1245065720-21:[ctx-42c67ffd, ctx-dc541641]) (logid:ece62172) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2025-10-27 15:02:59,468 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1245065720-21:[ctx-42c67ffd, ctx-dc541641]) (logid:ece62172) API rate limiting is disabled. We will not use ApiRateLimitService.
2025-10-27 15:02:59,561 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-5:[ctx-49433249]) (logid:c5a1eb67) HA health check task is running...
2025-10-27 15:03:00,130 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-e33eedb8]) (logid:cf721aa7) No inactive management server node found
2025-10-27 15:03:00,130 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-e33eedb8]) (logid:cf721aa7) Peer scan is finished. profiler: Done. Duration: 2ms , profilerQueryActiveList: Done. Duration: 0ms, , profilerSyncClusterInfo: Done. Duration: 0ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 0ms

[nvazquez]

Thanks @grehenl - I see the error is coming from the NSX side: Message: Error creating segment D1-A2-Z1-S225: General error has occurred. without giving more details about the failure.

Can you please check the NSX side and also the associated vCenter for failing tasks about this segment creation? That should provide more details about this failure

[grehenl]

@nvazquez There are no failing tasks in vCenter but i see the exceptions raised in NSX logs. Keep in mind that there are no issues creating segments manually from the NSX interface and attaching them to the created T1 gateways, so my guess is that there's something wrong with the API calls.

Here are the NSX logs:

2025-10-28T07:17:50.836Z INFO http-nio-127.0.0.1-7440-exec-70 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:50.844Z INFO http-nio-127.0.0.1-7440-exec-78 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:50.844Z INFO http-nio-127.0.0.1-7440-exec-78 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="229350e5-989c-46ef-a1b3-7dd0a8b5bf4f" subcomp="manager" username="admin"] Reading Cluster Id file /config/cluster-manager/clusterId
2025-10-28T07:17:50.844Z INFO http-nio-127.0.0.1-7440-exec-78 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="229350e5-989c-46ef-a1b3-7dd0a8b5bf4f" subcomp="manager" username="admin"] Using cluster id ce8b394d-34ed-4c05-8645-e265d6a334d6 from file /config/cluster-manager/clusterId
2025-10-28T07:17:54.452Z INFO http-nio-127.0.0.1-7440-exec-76 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:54.453Z INFO http-nio-127.0.0.1-7440-exec-74 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:54.454Z INFO http-nio-127.0.0.1-7440-exec-76 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="77f31bbe-a79d-4c9d-b6ee-3db249a53e17" subcomp="manager" username="admin"] Reading Cluster Id file /config/cluster-manager/clusterId
2025-10-28T07:17:54.454Z INFO http-nio-127.0.0.1-7440-exec-74 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="5e0f0e65-c9c8-4f69-8ba5-afb4a519ff50" subcomp="manager" username="admin"] Reading Cluster Id file /config/cluster-manager/clusterId
2025-10-28T07:17:54.454Z INFO http-nio-127.0.0.1-7440-exec-76 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="77f31bbe-a79d-4c9d-b6ee-3db249a53e17" subcomp="manager" username="admin"] Using cluster id ce8b394d-34ed-4c05-8645-e265d6a334d6 from file /config/cluster-manager/clusterId
2025-10-28T07:17:54.454Z INFO http-nio-127.0.0.1-7440-exec-74 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="5e0f0e65-c9c8-4f69-8ba5-afb4a519ff50" subcomp="manager" username="admin"] Using cluster id ce8b394d-34ed-4c05-8645-e265d6a334d6 from file /config/cluster-manager/clusterId
2025-10-28T07:17:59.021Z INFO http-nio-127.0.0.1-7440-exec-75 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.033Z ERROR http-nio-127.0.0.1-7440-exec-75 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" errorCode="PM500012" level="ERROR" reqId="ed620c87-4027-43f5-93be-327048d86ad7" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229 not found
2025-10-28T07:17:59.037Z INFO http-nio-127.0.0.1-7440-exec-77 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.039Z INFO http-nio-127.0.0.1-7440-exec-77 PartialPatchUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Header 'nsx-enable-partial-patch' : null
2025-10-28T07:17:59.039Z INFO http-nio-127.0.0.1-7440-exec-77 PatchOperationAspect 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Partial patch is not requested/enabled.
2025-10-28T07:17:59.052Z INFO http-nio-127.0.0.1-7440-exec-77 ConnectivityUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Setting default HA mode for Tier1: /infra/tier-1s/D1-A2-Z1-N229, isTier1Sr: false
2025-10-28T07:17:59.053Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229 does not exist, creating
2025-10-28T07:17:59.061Z INFO http-nio-127.0.0.1-7440-exec-77 ConnectivityValidationUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] validating Tier0 update, against any NAT rules attached and redirection policy of T0, the current T1 is attached to
2025-10-28T07:17:59.063Z INFO http-nio-127.0.0.1-7440-exec-77 ConnectivityValidationUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] validating Tier0 update, against any DNS forwarders attached and redirection policy of T0, the current T1 is attached to
2025-10-28T07:17:59.066Z INFO http-nio-127.0.0.1-7440-exec-77 NetworkServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Added provider relationships for newNetwork /infra/tier-1s/D1-A2-Z1-N229, provider /infra/tier-0s/T0-GW-EC-01
2025-10-28T07:17:59.066Z INFO http-nio-127.0.0.1-7440-exec-77 NetworkServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Adding DHCP config relationships for network /infra/tier-1s/D1-A2-Z1-N229
2025-10-28T07:17:59.066Z INFO http-nio-127.0.0.1-7440-exec-77 NetworkServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Added relationships for Tier1 /infra/tier-1s/D1-A2-Z1-N229, IPv6 profile /infra/ipv6-ndra-profiles/default
2025-10-28T07:17:59.067Z INFO http-nio-127.0.0.1-7440-exec-77 NetworkServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Added relationships for Tier1 /infra/tier-1s/D1-A2-Z1-N229, IPv6 profile /infra/ipv6-dad-profiles/default
2025-10-28T07:17:59.069Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229/nat/INTERNAL does not exist, creating
2025-10-28T07:17:59.070Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229/nat/USER does not exist, creating
2025-10-28T07:17:59.070Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229/nat/DEFAULT does not exist, creating
2025-10-28T07:17:59.071Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229/nat/NAT64 does not exist, creating
2025-10-28T07:17:59.072Z INFO http-nio-127.0.0.1-7440-exec-77 EdgeFirewallServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Switching to SYSTEM_USER principal identity
2025-10-28T07:17:59.074Z INFO http-nio-127.0.0.1-7440-exec-77 EdgeFirewallServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Switching to SYSTEM_USER principal identity
2025-10-28T07:17:59.074Z INFO http-nio-127.0.0.1-7440-exec-77 CommunicationEntryOrderingUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Number of communication entries 0
2025-10-28T07:17:59.074Z INFO http-nio-127.0.0.1-7440-exec-77 CommunicationEntryOrderingUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Reserved the sequence number 100 for communication entry /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229/rules/default_rule
2025-10-28T07:17:59.074Z INFO http-nio-127.0.0.1-7440-exec-77 AbstractCommunicationEntryServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Successfully modified the sequence number for /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229/rules/default_rule to 100
2025-10-28T07:17:59.074Z INFO http-nio-127.0.0.1-7440-exec-77 BaseCommunicationEntryService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Communication entry validate validateScopeForEdge null
2025-10-28T07:17:59.075Z INFO http-nio-127.0.0.1-7440-exec-77 AbstractCommunicationEntryServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Communication Map /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 target Type is null source excluded false dest excluded false
2025-10-28T07:17:59.075Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Entity /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229/rules/default_rule does not exist, creating
2025-10-28T07:17:59.075Z INFO http-nio-127.0.0.1-7440-exec-77 LightweightIdasImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Allocated 5127/1 ids from string_id: "f145da76-eee0-4271-9215-3a435c4e6001"
successfully in 0 ms.
2025-10-28T07:17:59.075Z INFO http-nio-127.0.0.1-7440-exec-77 AbstractCommunicationEntryServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Setting auto generated rule_id 5127 for /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229/rules/default_rule
2025-10-28T07:17:59.075Z INFO http-nio-127.0.0.1-7440-exec-77 FirewallRevisionHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Creating Dirty marker for:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 for request 82404e29-9e9a-4157-88a9-02e39553a73a. RetryAttempted = false
2025-10-28T07:17:59.076Z INFO http-nio-127.0.0.1-7440-exec-77 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Created dirty object marker:DirtyObjectMarker/947526f4-9830-4ac7-8fad-5f3b5a0678f2;/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 enforcementpoint:null
2025-10-28T07:17:59.077Z INFO http-nio-127.0.0.1-7440-exec-77 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Created dirty object marker per provider:DirtyObjectMarkerPerProvider/947526f4-9830-4ac7-8fad-5f3b5a0678f2;/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229;GatewayRuleSectionProvider for marker:DirtyObjectMarker/947526f4-9830-4ac7-8fad-5f3b5a0678f2;/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 enforcementpoint:null provider: GatewayRuleSectionProvider
2025-10-28T07:17:59.077Z INFO http-nio-127.0.0.1-7440-exec-77 AbstractCommunicationEntryServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] created/updated the communicationEntry default_rule /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229/rules/default_rule
2025-10-28T07:17:59.077Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyObjectHolder 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] got HAPI flag as ==null
2025-10-28T07:17:59.077Z INFO http-nio-127.0.0.1-7440-exec-77 AbstractCommunicationEntryServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] clearing Policy Object Holder for Comm. entry
2025-10-28T07:17:59.077Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyObjectHolder 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Clearing the Policy Object Cache
2025-10-28T07:17:59.080Z INFO http-nio-127.0.0.1-7440-exec-77 PolicyConnectivityFacadeImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Tier1 with name /infra/tier-1s/D1-A2-Z1-N229 created/updated
2025-10-28T07:17:59.080Z INFO http-nio-127.0.0.1-7440-exec-77 FederationGatewayConfigServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="82404e29-9e9a-4157-88a9-02e39553a73a" subcomp="manager" username="admin"] Got federation config with path as /infra/tier-1s/D1-A2-Z1-N229/federation-config for router /infra/tier-1s/D1-A2-Z1-N229
2025-10-28T07:17:59.094Z INFO http-nio-127.0.0.1-7440-exec-69 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.108Z INFO http-nio-127.0.0.1-7440-exec-69 LocaleServiceConverter 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="b3687c55-d647-404c-8e17-1b2bd41c896c" subcomp="manager" username="admin"] found redistribution config from do to dto
2025-10-28T07:17:59.112Z INFO http-nio-127.0.0.1-7440-exec-72 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.115Z INFO http-nio-127.0.0.1-7440-exec-72 PartialPatchUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Header 'nsx-enable-partial-patch' : null
2025-10-28T07:17:59.115Z INFO http-nio-127.0.0.1-7440-exec-72 PatchOperationAspect 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Partial patch is not requested/enabled.
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing DCN for object ID: SecurityFeatureToggle//infra/tier-1s/D1-A2-Z1-N229/security-config. Operation type: UPDATE, enforcementPoint: /infra/sites/default/enforcement-points/default, revision: 0
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing DCN for object ID: Network//infra/tier-1s/D1-A2-Z1-N229. Operation type: UPDATE, enforcementPoint: /infra/sites/default/enforcement-points/default, revision: 0
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Handling explicitly added path from marker DirtyObjectMarker/947526f4-9830-4ac7-8fad-5f3b5a0678f2;/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 for processing
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Will process DirtyObjectMarker DCN:revision 0 path /infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229, operation UPDATE, object type CommunicationMap
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing DCN for object ID: CommunicationMap//infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229. Operation type: UPDATE, enforcementPoint: /infra/sites/default/enforcement-points/default, revision: 0
2025-10-28T07:17:59.115Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing DCN for object ID: CommunicationMap//infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229. Operation type: UPDATE, enforcementPoint: /infra/sites/default/enforcement-points/default, revision: 0
2025-10-28T07:17:59.122Z INFO UfoIndexer-BatchExecutor-search_policy-2 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Pending dirty marker for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 setting state to:IN_PROGRESS
2025-10-28T07:17:59.123Z INFO UfoIndexer-BatchExecutor-search_policy-2 FederationGatewayConfigServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Got federation config with path as /infra/tier-1s/D1-A2-Z1-N229/federation-config for router /infra/tier-1s/D1-A2-Z1-N229
2025-10-28T07:17:59.125Z INFO UfoIndexer-BatchExecutor-search_policy-2 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Pending dirty marker for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 setting state to:IN_PROGRESS
2025-10-28T07:17:59.128Z INFO http-nio-127.0.0.1-7440-exec-72 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] Entity /infra/tier-1s/D1-A2-Z1-N229/locale-services/default does not exist, creating
2025-10-28T07:17:59.128Z INFO http-nio-127.0.0.1-7440-exec-72 LocaleSvcServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] Validating locale-service with path /infra/tier-1s/D1-A2-Z1-N229/locale-services/default: com.vmware.nsx.management.policy.connectivity.model.LocaleService@18eae3e9[redistributionConfig =,edgeClusterPath =/infra/sites/default/enforcement-points/default/edge-clusters/b2ca6972-6f2b-46d0-8a20-4b7d15e2d3bc,edgeClusterLabelPath =,preferredEdgePaths =,haVipConfigs =,isHaVipConfigured =false,bfdProfilePath =,providerRouteRedistributionTypes =]
2025-10-28T07:17:59.135Z INFO http-nio-127.0.0.1-7440-exec-72 LocaleSvcServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] don't create ospf, return
2025-10-28T07:17:59.135Z INFO http-nio-127.0.0.1-7440-exec-72 LocaleSvcServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] Successfully touched network: /infra/tier-1s/D1-A2-Z1-N229
2025-10-28T07:17:59.136Z INFO http-nio-127.0.0.1-7440-exec-72 ConnectivityUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] Setting default HA mode for Tier1: /infra/tier-1s/D1-A2-Z1-N229, isTier1Sr: true
2025-10-28T07:17:59.136Z INFO http-nio-127.0.0.1-7440-exec-72 LocaleSvcServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] Updating Tier1 HaMode to ACTIVE_STANDBY on locale service /infra/tier-1s/D1-A2-Z1-N229/locale-services/default update
2025-10-28T07:17:59.137Z INFO http-nio-127.0.0.1-7440-exec-72 PolicyConnectivityFacadeImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="588ce3d5-a92e-4762-ad11-f795769e4839" subcomp="manager" username="admin"] RouteMap /infra/tier-1s/D1-A2-Z1-N229/locale-services/default created/updated
2025-10-28T07:17:59.155Z INFO http-nio-127.0.0.1-7440-exec-71 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.168Z INFO UfoDelayedIndexer-0 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 to:UNINITIALIZED
2025-10-28T07:17:59.170Z INFO UfoDelayedIndexer-0 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Pending dirty marker for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 setting state to:IN_PROGRESS
2025-10-28T07:17:59.170Z INFO http-nio-127.0.0.1-7440-exec-62 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.171Z INFO UfoDelayedIndexer-0 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-1s/D1-A2-Z1-N229 to:UNINITIALIZED
2025-10-28T07:17:59.187Z INFO http-nio-127.0.0.1-7440-exec-37 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.188Z INFO PolicyEngineDCNHandler-1-1 PolicyEngineDCNHandler 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing DCN for object ID: Network//infra/tier-1s/D1-A2-Z1-N229. Operation type: UPDATE, enforcementPoint: /infra/sites/default/enforcement-points/default, revision: 0
2025-10-28T07:17:59.189Z INFO http-nio-127.0.0.1-7440-exec-37 TransportZoneQueryDao 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f25b94cd-7589-46ab-bfe5-a89c6954ffab" subcomp="manager" username="admin"] TZQueryDao getPersistedQueryParameters
2025-10-28T07:17:59.196Z INFO http-nio-127.0.0.1-7440-exec-67 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:17:59.199Z INFO http-nio-127.0.0.1-7440-exec-67 PartialPatchUtils 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Header 'nsx-enable-partial-patch' : null
2025-10-28T07:17:59.199Z INFO http-nio-127.0.0.1-7440-exec-67 PatchOperationAspect 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Partial patch is not requested/enabled.
2025-10-28T07:17:59.208Z INFO http-nio-127.0.0.1-7440-exec-67 PolicyServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="a3da8b8c-483a-4f67-9bd8-d7503ddabd2b" subcomp="manager" username="admin"] Entity /infra/segments/D1-A2-Z1-S229 does not exist, creating
2025-10-28T07:17:59.209Z WARN http-nio-127.0.0.1-7440-exec-67 PolicyResourceChangeNotificationManager 111131 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" reqId="a3da8b8c-483a-4f67-9bd8-d7503ddabd2b" subcomp="manager" username="admin"] Failure received invoking listener SegmentNotificationListener for change CREATING on resource /infra/segments/D1-A2-Z1-S229
java.lang.NullPointerException: null
at com.vmware.nsx.management.policy.connectivity.service.SegmentNotificationListener.validateNonDiscoveredSegmentUsingSecurityTZ(SegmentNotificationListener.java:254) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentNotificationListener.handleResourceChange(SegmentNotificationListener.java:122) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.dao.PolicyResourceChangeNotificationManager.notify(PolicyResourceChangeNotificationManager.java:161) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.dao.PolicyResourceChangeNotificationManager.notify(PolicyResourceChangeNotificationManager.java:134) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody2(PolicyServiceImpl.java:176) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure3.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody4(PolicyServiceImpl.java:170) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure5.run(PolicyServiceImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:119) ~[?:?]
at io.micrometer.core.aop.TimedAspect.ajc$inlineAccessMethod$io_micrometer_core_aop_TimedAspect$io_micrometer_core_aop_TimedAspect$processWithTimer(TimedAspect.java:1) ~[?:?]
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:97) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create(PolicyServiceImpl.java:170) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody0(PolicyServiceImpl.java:151) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure1.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create(PolicyServiceImpl.java:151) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody10(PolicyServiceImpl.java:528) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure11.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody12(PolicyServiceImpl.java:384) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure13.run(PolicyServiceImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:119) ~[?:?]
at io.micrometer.core.aop.TimedAspect.ajc$inlineAccessMethod$io_micrometer_core_aop_TimedAspect$io_micrometer_core_aop_TimedAspect$processWithTimer(TimedAspect.java:1) ~[?:?]
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:97) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:384) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody8(PolicyServiceImpl.java:366) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure9.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:366) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody6(PolicyServiceImpl.java:360) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure7.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:360) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl.patch_aroundBody6(SegmentServiceImpl.java:669) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl$AjcClosure7.run(SegmentServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl.patch(SegmentServiceImpl.java:648) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.handlePutOrPatchSegment(PolicyConnectivityFacadeImpl.java:1146) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody648(PolicyConnectivityFacadeImpl.java:2195) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure649.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.validators.FacadeInterceptorValidatorAspect.validationAnnotatedMethod(FacadeInterceptorValidatorAspect.java:58) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody650(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure651.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody652(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure653.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:229) ~[?:?]
at com.vmware.nsx.management.policy.reaction.utils.ApiRequestBodyAspect.dispatchApiInvocation(ApiRequestBodyAspect.java:74) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody654(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure655.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.container.application.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:172) ~[?:?]
at com.vmware.nsx.management.container.application.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:430) ~[?:?]
at com.vmware.nsx.management.common.api.FacadeInterceptorAspect.annotatedMethod(FacadeInterceptorAspect.java:80) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody656(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure657.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.utils.PatchOperationAspect.handleConcurrentTransaction(PatchOperationAspect.java:78) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsxapi.policyconnectivity.controller.PolicyConnectivityController.patchInfraSegment(PolicyConnectivityController.java:6655) ~[?:?]
at jdk.internal.reflect.GeneratedMethodAccessor45810.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[?:?]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809) ~[?:?]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:880) ~[?:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) ~[servlet-api.jar:4.0.FR]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-websocket.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) ~[?:?]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) ~[?:?]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) ~[?:?]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[?:?]
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[?:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) ~[?:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) ~[?:?]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) ~[?:?]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) ~[?:?]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) ~[?:?]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) ~[?:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at com.vmware.nsx.management.filter.RequestRedirectFilterLocal.doFilter(RequestRedirectFilterLocal.java:80) ~[?:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[catalina.jar:9.0.88]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) ~[catalina.jar:9.0.88]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[catalina.jar:9.0.88]
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[catalina.jar:9.0.88]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346) ~[catalina.jar:9.0.88]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390) ~[tomcat-coyote.jar:9.0.88]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-coyote.jar:9.0.88]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1786) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-util.jar:9.0.88]
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.88]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-util.jar:9.0.88]
at java.lang.Thread.run(Unknown Source) ~[?:?]
2025-10-28T07:17:59.209Z INFO http-nio-127.0.0.1-7440-exec-67 TraceAspectAjIf 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="a3da8b8c-483a-4f67-9bd8-d7503ddabd2b" subcomp="manager" username="admin"] abnormally finish tracing a workflow with name patch
2025-10-28T07:17:59.209Z INFO http-nio-127.0.0.1-7440-exec-67 TraceAspectAjIf 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="a3da8b8c-483a-4f67-9bd8-d7503ddabd2b" subcomp="manager" username="admin"] abnormally finish tracing a workflow with name handlePutOrPatchSegment
2025-10-28T07:17:59.210Z INFO http-nio-127.0.0.1-7440-exec-67 TraceAspectAjIf 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="a3da8b8c-483a-4f67-9bd8-d7503ddabd2b" subcomp="manager" username="admin"] abnormally finish tracing a workflow with name patchInfraSegment
2025-10-28T07:17:59.210Z ERROR http-nio-127.0.0.1-7440-exec-67 NsxBaseRestController 111131 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP100" level="ERROR" subcomp="manager"]
java.lang.NullPointerException: null
at com.vmware.nsx.management.policy.connectivity.service.SegmentNotificationListener.validateNonDiscoveredSegmentUsingSecurityTZ(SegmentNotificationListener.java:254) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentNotificationListener.handleResourceChange(SegmentNotificationListener.java:122) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.dao.PolicyResourceChangeNotificationManager.notify(PolicyResourceChangeNotificationManager.java:161) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.dao.PolicyResourceChangeNotificationManager.notify(PolicyResourceChangeNotificationManager.java:134) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody2(PolicyServiceImpl.java:176) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure3.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody4(PolicyServiceImpl.java:170) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure5.run(PolicyServiceImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:119) ~[?:?]
at io.micrometer.core.aop.TimedAspect.ajc$inlineAccessMethod$io_micrometer_core_aop_TimedAspect$io_micrometer_core_aop_TimedAspect$processWithTimer(TimedAspect.java:1) ~[?:?]
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:97) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create(PolicyServiceImpl.java:170) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create_aroundBody0(PolicyServiceImpl.java:151) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure1.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.create(PolicyServiceImpl.java:151) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody10(PolicyServiceImpl.java:528) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure11.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody12(PolicyServiceImpl.java:384) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure13.run(PolicyServiceImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:119) ~[?:?]
at io.micrometer.core.aop.TimedAspect.ajc$inlineAccessMethod$io_micrometer_core_aop_TimedAspect$io_micrometer_core_aop_TimedAspect$processWithTimer(TimedAspect.java:1) ~[?:?]
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:97) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:384) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody8(PolicyServiceImpl.java:366) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure9.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:366) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate_aroundBody6(PolicyServiceImpl.java:360) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl$AjcClosure7.run(PolicyServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.service.PolicyServiceImpl.createOrUpdate(PolicyServiceImpl.java:360) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl.patch_aroundBody6(SegmentServiceImpl.java:669) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl$AjcClosure7.run(SegmentServiceImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.service.SegmentServiceImpl.patch(SegmentServiceImpl.java:648) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.handlePutOrPatchSegment(PolicyConnectivityFacadeImpl.java:1146) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody648(PolicyConnectivityFacadeImpl.java:2195) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure649.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.validators.FacadeInterceptorValidatorAspect.validationAnnotatedMethod(FacadeInterceptorValidatorAspect.java:58) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody650(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure651.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96cproceed(AbstractTransactionAspect.aj:67) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect$AbstractTransactionAspect$1.proceedWithInvocation(AbstractTransactionAspect.aj:73) ~[?:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.aspects.IntentTransactionAspect.invokeWithinTransaction(IntentTransactionAspect.java:75) ~[?:?]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:71) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody652(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure653.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:229) ~[?:?]
at com.vmware.nsx.management.policy.reaction.utils.ApiRequestBodyAspect.dispatchApiInvocation(ApiRequestBodyAspect.java:74) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody654(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure655.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.container.application.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:172) ~[?:?]
at com.vmware.nsx.management.container.application.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:430) ~[?:?]
at com.vmware.nsx.management.common.api.FacadeInterceptorAspect.annotatedMethod(FacadeInterceptorAspect.java:80) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment_aroundBody656(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl$AjcClosure657.run(PolicyConnectivityFacadeImpl.java:1) ~[?:?]
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[?:?]
at com.vmware.nsx.management.policy.policyframework.utils.PatchOperationAspect.handleConcurrentTransaction(PatchOperationAspect.java:78) ~[?:?]
at com.vmware.nsx.management.policy.connectivity.facade.PolicyConnectivityFacadeImpl.patchInfraSegment(PolicyConnectivityFacadeImpl.java:2192) ~[?:?]
at com.vmware.nsxapi.policyconnectivity.controller.PolicyConnectivityController.patchInfraSegment(PolicyConnectivityController.java:6655) ~[?:?]
at jdk.internal.reflect.GeneratedMethodAccessor45810.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[?:?]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903) ~[?:?]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809) ~[?:?]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:880) ~[?:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) ~[servlet-api.jar:4.0.FR]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-websocket.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) ~[?:?]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) ~[?:?]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) ~[?:?]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[?:?]
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[?:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) ~[?:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) ~[?:?]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) ~[?:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[?:?]
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) ~[?:?]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) ~[?:?]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) ~[?:?]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) ~[?:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at com.vmware.nsx.management.filter.RequestRedirectFilterLocal.doFilter(RequestRedirectFilterLocal.java:80) ~[?:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:169) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[catalina.jar:9.0.88]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) ~[catalina.jar:9.0.88]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[catalina.jar:9.0.88]
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670) ~[catalina.jar:9.0.88]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[catalina.jar:9.0.88]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346) ~[catalina.jar:9.0.88]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390) ~[tomcat-coyote.jar:9.0.88]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-coyote.jar:9.0.88]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1786) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-coyote.jar:9.0.88]
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-util.jar:9.0.88]
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.88]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-util.jar:9.0.88]
at java.lang.Thread.run(Unknown Source) ~[?:?]
2025-10-28T07:17:59.215Z INFO UfoIndexer-BatchExecutor-search_policy-1 FederationGatewayConfigServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Got federation config with path as /infra/tier-1s/D1-A2-Z1-N229/federation-config for router /infra/tier-1s/D1-A2-Z1-N229
2025-10-28T07:17:59.269Z INFO UfoDelayedIndexer-0 DirtyObjectMarkerService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Pending dirty marker for path:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 setting state to:IN_PROGRESS
2025-10-28T07:18:00.166Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
eac00880-f5ab-43a6-a3db-342f922cb9be
}
with operation UPDATE, adding work-item WorkItem{identifier=AdvertiseRouteList/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}} for processing
2025-10-28T07:18:00.166Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
eac00880-f5ab-43a6-a3db-342f922cb9be
}
with operation UPDATE, adding work-item WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}} for processing
2025-10-28T07:18:00.166Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
eac00880-f5ab-43a6-a3db-342f922cb9be
}
with operation UPDATE, adding work-item WorkItem{identifier=LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}} for processing
2025-10-28T07:18:00.253Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
eac00880-f5ab-43a6-a3db-342f922cb9be
}
with operation UPDATE, adding work-item WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508795}} for processing
2025-10-28T07:18:00.317Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename FirewallSection and Identifier FirewallSection/a5a319a1-f526-465f-8336-e57029e84075.
2025-10-28T07:18:00.350Z INFO workerTaskExecutor-1-45 WorkerInvocationTask 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Invoking worker : LogicalRouterWorker, work-items : [WorkItem{identifier=LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}, WorkItem{identifier=AdvertiseRouteList/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}, WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508795}}]
2025-10-28T07:18:00.350Z INFO workerTaskExecutor-1-45 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Worker triggered with work items [WorkItem{identifier=LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}, WorkItem{identifier=AdvertiseRouteList/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}, WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508795}}]
2025-10-28T07:18:00.350Z INFO workerTaskExecutor-1-45 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing started for work items {AdvertiseRouteList=[WorkItem{identifier=AdvertiseRouteList/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}], LogicalRouter=[WorkItem{identifier=LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}], AdvertisementConfig=[WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508795}}]} of logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.358Z INFO workerTaskExecutor-1-45 LRPublishStateProxyDaoImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created old state for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.373Z INFO workerTaskExecutor-1-45 LRPublishStateProxyDaoImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created old state for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.384Z INFO workerTaskExecutor-1-45 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated new state for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.384Z INFO workerTaskExecutor-1-45 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating Distributed router eac00880-f5ab-43a6-a3db-342f922cb9be for D1-A2-Z1-N229
2025-10-28T07:18:00.384Z INFO workerTaskExecutor-1-45 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populating TIER1 SR for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.400Z INFO workerTaskExecutor-1-45 LightweightIdasImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Allocated 7178/1 ids from string_id: "43ae4fdd883544e8a004f01f351ba51e"
successfully in 1 ms.
2025-10-28T07:18:00.400Z INFO workerTaskExecutor-1-45 IndexAllocationServiceImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Allocating routerId [7178] allocated.
2025-10-28T07:18:00.400Z INFO workerTaskExecutor-1-45 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] TIER1 lr LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be mapped to new SR LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d on transport nodes [65537cac-4c82-4c19-9660-6f75c79b0d2a, f9d634f1-06e2-4247-a110-74df031320ba]
2025-10-28T07:18:00.412Z INFO workerTaskExecutor-1-45 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated overlay transport zone b50e5af4-aa77-4f91-8814-d4ae6fb0ced7 for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be.
2025-10-28T07:18:00.421Z INFO workerTaskExecutor-1-45 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Set SR LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d rank to 0
2025-10-28T07:18:00.421Z INFO workerTaskExecutor-1-45 TransitInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Transport zone of LR changed from [] to [b50e5af4-aa77-4f91-8814-d4ae6fb0ced7] . Recreating transit switches and ports
2025-10-28T07:18:00.421Z INFO workerTaskExecutor-1-45 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] AttachmentState for TIER1 router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be is null. Skip updating router link ports upon TZ change.
2025-10-28T07:18:00.424Z INFO workerTaskExecutor-1-45 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.424Z INFO workerTaskExecutor-1-45 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:53:00 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:5300
2025-10-28T07:18:00.424Z INFO workerTaskExecutor-1-45 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating SR a2bd30b4-4976-4016-b411-8f186784373d rank 0 on edge transport node 65537cac-4c82-4c19-9660-6f75c79b0d2a for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.424Z INFO workerTaskExecutor-1-45 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating SR a2bd30b4-4976-4016-b411-8f186784373d rank 1 on edge transport node f9d634f1-06e2-4247-a110-74df031320ba for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.429Z INFO workerTaskExecutor-1-45 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] auto-plumbed system loopback LogicalRouterPortConfig/aa1508c8-6c34-4bcb-b47a-58c0ca1dc450 marked to create for router id LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d of lr LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.455Z INFO workerTaskExecutor-1-45 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:53:00 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:5300
2025-10-28T07:18:00.458Z INFO workerTaskExecutor-1-45 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found sr transit port generating route default
2025-10-28T07:18:00.458Z INFO workerTaskExecutor-1-45 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 expanding connected routes, ipAddresses 0.0.0.0/0
2025-10-28T07:18:00.459Z INFO workerTaskExecutor-1-45 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 expanding connected routes, ipAddresses ::/0
2025-10-28T07:18:00.459Z INFO workerTaskExecutor-1-45 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.462Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[e945] encountered trimmed addresses [] during sync to 1150508840 on attempt 1 of 2
2025-10-28T07:18:00.466Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
7b0f9674-59ca-5684-afc3-252a396dae49
}
with operation UPDATE, adding work-item WorkItem{identifier=LrPort/7b0f9674-59ca-5684-afc3-252a396dae49, Timestamp{epoch=1299, address=1150508831}} for processing
2025-10-28T07:18:00.480Z INFO workerTaskExecutor-1-45 NatServiceImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created nat/nat64 section 00003200-0000-0000-0000-000000001824 with org null, project null, and vpc null.
2025-10-28T07:18:00.480Z INFO workerTaskExecutor-1-45 NatServiceImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created nat/nat64 section 00003600-0000-0000-0000-000000001824 with org null, project null, and vpc null.
2025-10-28T07:18:00.482Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[4aa] encountered trimmed addresses [] during sync to 1150508840 on attempt 1 of 2
2025-10-28T07:18:00.486Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.495Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[6da1] encountered trimmed addresses [] during sync to 1150508840 on attempt 1 of 2
2025-10-28T07:18:00.516Z INFO workerTaskExecutor-1-45 IPSecVPNSvcServiceImpl 111131 VPN [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created IPSec Section 00003500-0000-0000-0000-000000001824
2025-10-28T07:18:00.521Z INFO workerTaskExecutor-1-45 LRServiceIPLoopbackPortUpdateListener 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully updated service ip on loopback lif LogicalRouterPortConfig/aa1508c8-6c34-4bcb-b47a-58c0ca1dc450
2025-10-28T07:18:00.522Z INFO UfoIndexer-BatchExecutor-search_manager-1 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.525Z INFO workerTaskExecutor-1-45 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/eaae3437-4ffe-44bf-83e7-bcc27f6d2478 of type INTERNAL_TRANSIT for router LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be connected to l2 port null
2025-10-28T07:18:00.525Z INFO workerTaskExecutor-1-45 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 of type INTERNAL_TRANSIT for router LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d connected to l2 port null
2025-10-28T07:18:00.525Z INFO workerTaskExecutor-1-45 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif LogicalRouterPortConfig/aa1508c8-6c34-4bcb-b47a-58c0ca1dc450 of type LOOPBACK for router LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d
2025-10-28T07:18:00.532Z INFO workerTaskExecutor-1-45 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Port container c833ac6e-df2d-41ee-9729-9f18ad83a5ce updated for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.534Z INFO workerTaskExecutor-1-45 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated meta state for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.541Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
c3f682d8-5a39-48c6-8025-ecdfd0c6e205
}
with operation UPDATE, adding work-item WorkItem{identifier=LrPort/c3f682d8-5a39-48c6-8025-ecdfd0c6e205, Timestamp{epoch=1299, address=1150508844}} for processing
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be.
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Initializing entry to Status tracker Cache table LogicalRouterConfig, identifier LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d.
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding entry to Status tracker Entity map table LogicalRouterConfig, identifier LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d, current size 1
2025-10-28T07:18:00.542Z WARN RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Table LogicalRouterPortConfig not supported by CCP for Realization tracking. So skipping it.
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003200-0000-0000-0000-000000001824.
2025-10-28T07:18:00.542Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003600-0000-0000-0000-000000001824.
2025-10-28T07:18:00.563Z INFO UfoIndexer-BatchExecutor-search_manager-3 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:55 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4455
2025-10-28T07:18:00.564Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[dbe1] encountered trimmed addresses [] during sync to 1150508849 on attempt 1 of 2
2025-10-28T07:18:00.570Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.577Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalRouter eac00880-f5ab-43a6-a3db-342f922cb9be has total SR's 1
2025-10-28T07:18:00.586Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 2 transport nodes for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.586Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 2 transport nodes with Unknown status for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.586Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalRouter d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b has total SR's 3
2025-10-28T07:18:00.588Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[16fb] encountered trimmed addresses [] during sync to 1150508849 on attempt 1 of 2
2025-10-28T07:18:00.593Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.604Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 transport nodes for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.604Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 status for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.604Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalRouter d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b has total SR's 3
2025-10-28T07:18:00.606Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 transport nodes for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.606Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 status for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.607Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalRouter d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b has total SR's 3
2025-10-28T07:18:00.609Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 transport nodes for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.609Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 3 status for logical router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.609Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[e51] encountered trimmed addresses [] during sync to 1150508849 on attempt 1 of 2
2025-10-28T07:18:00.613Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(l7_profile_id, from_address.container_uuid, ctx_profile_id, ams_profile_id, applied_to.container_uuid, service_protocol.service_uuid, extended_source.container, to_address.container_uuid, ids_profile_id)
2025-10-28T07:18:00.637Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[a12] encountered trimmed addresses [] during sync to 1150508849 on attempt 1 of 2
2025-10-28T07:18:00.642Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.653Z INFO workerTaskExecutor-1-48 WorkerInvocationTask 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Invoking worker : LogicalRouterWorker, work-items : [WorkItem{identifier=LrPort/c3f682d8-5a39-48c6-8025-ecdfd0c6e205, Timestamp{epoch=1299, address=1150508844}}]
2025-10-28T07:18:00.653Z INFO workerTaskExecutor-1-48 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Worker triggered with work items [WorkItem{identifier=LrPort/c3f682d8-5a39-48c6-8025-ecdfd0c6e205, Timestamp{epoch=1299, address=1150508844}}]
2025-10-28T07:18:00.654Z WARN workerTaskExecutor-1-45 MultiVersionObject 111131 SnapshotProxy[c575] encountered trimmed addresses [] during sync to 1150508849 on attempt 1 of 2
2025-10-28T07:18:00.659Z INFO workerTaskExecutor-1-45 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.668Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalRouter eac00880-f5ab-43a6-a3db-342f922cb9be has total SR's 1
2025-10-28T07:18:00.672Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 2 transport nodes for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.672Z INFO UfoDelayedIndexer-0 AggSvcLogicalRouterService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found 2 transport nodes with Unknown status for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.674Z INFO workerTaskExecutor-1-45 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing completed for work items {AdvertiseRouteList=[WorkItem{identifier=AdvertiseRouteList/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}], LogicalRouter=[WorkItem{identifier=LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508786}}], AdvertisementConfig=[WorkItem{identifier=AdvertisementConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, Timestamp{epoch=1299, address=1150508795}}]} of logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.685Z INFO workerTaskExecutor-1-48 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing started for work items {LrPort=[WorkItem{identifier=LrPort/c3f682d8-5a39-48c6-8025-ecdfd0c6e205, Timestamp{epoch=1299, address=1150508844}}]} of logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.685Z INFO workerTaskExecutor-1-45 WorkerInvocationTask 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Worker processing done , worker-id - LogicalRouterWorker
2025-10-28T07:18:00.696Z INFO workerTaskExecutor-1-48 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated new state for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.697Z INFO workerTaskExecutor-1-48 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populating TIER1 SR for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.702Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:55 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4455
2025-10-28T07:18:00.702Z INFO workerTaskExecutor-1-48 LRPortUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated IPV6 LinkLocal address fe80::50:56ff:fe56:4455 for mac 02:50:56:56:44:55
2025-10-28T07:18:00.702Z INFO workerTaskExecutor-1-48 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating TIER1 router link lif LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 linked logical router id from null to LogicalRouter/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b.
2025-10-28T07:18:00.702Z INFO workerTaskExecutor-1-48 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating TIER1 router link lif LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 linked logical router port id from null to LrPort/7b0f9674-59ca-5684-afc3-252a396dae49.
2025-10-28T07:18:00.703Z INFO workerTaskExecutor-1-48 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Initiating attachment of TIER1 LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be to TIER0 LogicalRouter/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b connected on TIER0 router link port LrPort/7b0f9674-59ca-5684-afc3-252a396dae49
2025-10-28T07:18:00.706Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.706Z INFO workerTaskExecutor-1-48 LRPortUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated IPV6 LinkLocal address fe80::50:56ff:fe56:4452 for mac 02:50:56:56:44:52
2025-10-28T07:18:00.706Z INFO workerTaskExecutor-1-48 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating TIER0 router link lif 7b0f9674-59ca-5684-afc3-252a396dae49 on TIER0 router d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b
2025-10-28T07:18:00.707Z INFO workerTaskExecutor-1-48 UserInterfacesPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Create attachment of TIER1 LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be to TIER0 LogicalRouter/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b connected on TIER0 router link port LrPort/7b0f9674-59ca-5684-afc3-252a396dae49
2025-10-28T07:18:00.713Z INFO workerTaskExecutor-1-48 LrLrpPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Populated overlay transport zone b50e5af4-aa77-4f91-8814-d4ae6fb0ced7 for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be.
2025-10-28T07:18:00.718Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.719Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:53:00 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:5300
2025-10-28T07:18:00.719Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:53:00 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:5300
2025-10-28T07:18:00.720Z INFO workerTaskExecutor-1-48 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating SR a2bd30b4-4976-4016-b411-8f186784373d rank 0 on edge transport node 65537cac-4c82-4c19-9660-6f75c79b0d2a for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.720Z INFO workerTaskExecutor-1-48 SrAndSrClusterPrePublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating SR a2bd30b4-4976-4016-b411-8f186784373d rank 1 on edge transport node f9d634f1-06e2-4247-a110-74df031320ba for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.725Z INFO workerTaskExecutor-1-48 SwitchingValidator 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Skipping transit-rl-eac00880-f5ab-43a6-a3db-342f922cb9be MTEP validation as it is system resource
2025-10-28T07:18:00.728Z WARN workerTaskExecutor-1-48 MultiVersionObject 111131 SnapshotProxy[ee74] encountered trimmed addresses [] during sync to 1150508861 on attempt 1 of 2
2025-10-28T07:18:00.732Z INFO workerTaskExecutor-1-48 ImmutableCorfuTable 111131 ImmutableCorfuTable: creating PersistentCorfuTable with the following indexes: HashSet(managed_resource.forward_relationship)
2025-10-28T07:18:00.741Z INFO workerTaskExecutor-1-48 MacManagementSwitchingProfileServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Switching Profiles for Logical Switch 'LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513' not found. Using Default MacManagement values
2025-10-28T07:18:00.744Z INFO workerTaskExecutor-1-48 Executors 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Thread RangeServiceImpl-3-11 created. Total number of threads created 697
2025-10-28T07:18:00.755Z INFO RangeServiceImpl-3-11 RangeServiceImpl 111131 POOL-MGMT [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Allocated id 70659 from pool 821f3712-d184-44d6-ba90-5c3e982274e1
2025-10-28T07:18:00.755Z INFO workerTaskExecutor-1-48 RangeServiceImpl 111131 POOL-MGMT [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successful Id Allocation Task: Pool id: 821f3712-d184-44d6-ba90-5c3e982274e1 Number of Collisions: 0. Time taken ms 11
2025-10-28T07:18:00.755Z INFO workerTaskExecutor-1-48 LogicalSwitchServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Vni [70659] allocated temporary from vni pool [VniPool/821f3712-d184-44d6-ba90-5c3e982274e1].
2025-10-28T07:18:00.755Z INFO workerTaskExecutor-1-48 RealizationUtil 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] RTF Tracker : null requestId found in NsxExecutionContext. Using random requestId 4efe9041-c81c-4e86-a316-e2841f34e0a4
2025-10-28T07:18:00.760Z INFO workerTaskExecutor-1-48 IdAllocationServiceImpl 111131 POOL-MGMT [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Made allocation 70659 permanent
2025-10-28T07:18:00.760Z INFO workerTaskExecutor-1-48 VNIMServiceImpl 111131 POOL-MGMT [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Marked VNI [70659] as permanent.
2025-10-28T07:18:00.760Z INFO workerTaskExecutor-1-48 LogicalSwitchServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Vni [70659] allocated permanently from vni pool [821f3712-d184-44d6-ba90-5c3e982274e1].
2025-10-28T07:18:00.760Z INFO workerTaskExecutor-1-48 LogicalSwitchServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating relationship between logical switch LogicalSwitch [id=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, name=transit-rl-eac00880-f5ab-43a6-a3db-342f922cb9be, segmentPath=null, switchType=DEFAULT, transportZoneId=TransportZone/b50e5af4-aa77-4f91-8814-d4ae6fb0ced7, vni=70659, vniPoolId=VniPool/821f3712-d184-44d6-ba90-5c3e982274e1, replicationMode=MTEP, adminState=UP, switchingProfileIds=[], switchMode=STANDARD, switchType=DEFAULT, uplinkTeamingPolicyName=null, extraConfigs=[ ], originId=null, originType=null, org=null, project=null, vpc=null] and ip pool []
2025-10-28T07:18:00.761Z INFO workerTaskExecutor-1-48 SwitchingUPMProfilesBindingManager 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating UPM profile bindings for LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513
2025-10-28T07:18:00.761Z INFO workerTaskExecutor-1-48 SwitchingUPMProfilesBindingManager 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding UPM profile bindings for switching entity 5cc38939-6832-4d28-be28-58ffd22d4513
2025-10-28T07:18:00.761Z INFO workerTaskExecutor-1-48 SwitchingUPMProfilesBindingManager 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleting inactive UPM profile bindings from entity LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513
2025-10-28T07:18:00.762Z INFO workerTaskExecutor-1-48 SwitchingProfileUPMServiceAdapterImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] No service config found for switchingEntity class com.vmware.nsx.management.switching.logicalnetwork.model.LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513
2025-10-28T07:18:00.764Z INFO workerTaskExecutor-1-48 LogicalSwitchServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created LogicalSwitch LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513 @ revision 0
2025-10-28T07:18:00.764Z INFO workerTaskExecutor-1-48 L2Configurator 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating external switch Id LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513 connecting TIER1 LR eac00880-f5ab-43a6-a3db-342f922cb9be to TIER0 LR via attachment
2025-10-28T07:18:00.785Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:53:00 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:5300
2025-10-28T07:18:00.785Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Computing routes for port c3f682d8-5a39-48c6-8025-ecdfd0c6e205 old subnet null new subnet [SubnetModel{prefixLength=31, ipConfigs=[100.64.0.7]}, SubnetModel{prefixLength=64, ipConfigs=[fc84:199f:f529:e001::2]}, SubnetModel{prefixLength=64, ipConfigs=[fe80::50:56ff:fe56:4455]}]
2025-10-28T07:18:00.785Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:55 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4455
2025-10-28T07:18:00.786Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] processing sr lif subnet 100.64.0.7 transit ip 169.254.0.2/28, fe80::50:56ff:fe56:5300/64
2025-10-28T07:18:00.789Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 expanding connected routes, ipAddresses 0.0.0.0/0
2025-10-28T07:18:00.790Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] processing sr lif subnet fc84:199f:f529:e001::2 transit ip 169.254.0.2/28, fe80::50:56ff:fe56:5300/64
2025-10-28T07:18:00.790Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 expanding connected routes, ipAddresses ::/0
2025-10-28T07:18:00.791Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] processing sr lif subnet fe80::50:56ff:fe56:4455 transit ip 169.254.0.2/28, fe80::50:56ff:fe56:5300/64
2025-10-28T07:18:00.792Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Found sr transit port generating route default
2025-10-28T07:18:00.792Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 expanding connected routes, ipAddresses 0.0.0.0/0
2025-10-28T07:18:00.792Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For lif LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 expanding connected routes, ipAddresses ::/0
2025-10-28T07:18:00.794Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.795Z INFO workerTaskExecutor-1-48 IpAddressUtils 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] For CIDR:FE80::/64, macAddress:02:50:56:56:44:52 the generated IPv6 LinkLocal Address is fe80::050:56ff:fe56:4452
2025-10-28T07:18:00.798Z INFO workerTaskExecutor-1-48 RouteGraphManager 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Building graph for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.801Z INFO workerTaskExecutor-1-48 RouteGraphManager 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully build graph for LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, affected networks from route graph are []
2025-10-28T07:18:00.801Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Removed subnets due to port config update/delete []
2025-10-28T07:18:00.801Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created SB subnets due to port config update/create []
2025-10-28T07:18:00.801Z INFO workerTaskExecutor-1-48 ConnectedStaticRoutesPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created NB subnets due to port config update/create [fc84:199f:f529:e001::/64, 100.64.0.6/31]
2025-10-28T07:18:00.804Z INFO workerTaskExecutor-1-48 NatServiceImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated nat/nat64 section 00003200-0000-0000-0000-000000001824 with new org null, project null, and vpc null.
2025-10-28T07:18:00.804Z INFO workerTaskExecutor-1-48 NatServiceImpl 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated nat/nat64 section 00003600-0000-0000-0000-000000001824 with new org null, project null, and vpc null.
2025-10-28T07:18:00.814Z INFO workerTaskExecutor-1-48 LRServiceIPLoopbackPortUpdateListener 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully updated service ip on loopback lif LogicalRouterPortConfig/aa1508c8-6c34-4bcb-b47a-58c0ca1dc450
2025-10-28T07:18:00.815Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/eaae3437-4ffe-44bf-83e7-bcc27f6d2478 of type INTERNAL_TRANSIT for router LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be connected to l2 port null
2025-10-28T07:18:00.816Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 of type TIER1_ROUTER_LINK for router LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d connected to l2 port null
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Logical port validations took 4 msec
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Begin temporary allocation of Ip and/or Mac resources for logical port InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c in transaction at 1761635880820
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] End temporary allocation of Ip and/or Mac resources for logical port InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c, total time taken = 0
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Begin create logical port InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c in transaction at 1761635880820
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 PolicyUfoUtils 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Stored LogicalPort InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c in internal store
2025-10-28T07:18:00.820Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalPort created with ID: [InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c], total time in create transaction = 0
2025-10-28T07:18:00.821Z INFO workerTaskExecutor-1-48 FirewallEntityListenerImpl 111131 FIREWALL [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] onEvent: Create resource: LogicalPort [id=68ad6481-6823-4219-9884-5a6a49be911c, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=null, transportZoneType=OVERLAY, attachmentId=c3f682d8-5a39-48c6-8025-ecdfd0c6e205, attachmentType=internal_sr_tier1, internalPortAttachment=attachment_id {
uuid {
left: 14120617547405019334
right: 9234047057017627141
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_SERVICE_ROUTER_TIER1
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false], existingResource: null
2025-10-28T07:18:00.821Z INFO workerTaskExecutor-1-48 GroupingObjectsEntityListener 111131 INVENTORY [nsx@6876 comp="nsx-manager" level="INFO" s2comp="grouping" subcomp="manager"] GroupingObjectsEntityListener : Start Grouping Framework received Event com.vmware.nsx.management.entitylistener.service.EntityEvent@570d2eb8 for resource LogicalPort [id=68ad6481-6823-4219-9884-5a6a49be911c, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=null, transportZoneType=OVERLAY, attachmentId=c3f682d8-5a39-48c6-8025-ecdfd0c6e205, attachmentType=internal_sr_tier1, internalPortAttachment=attachment_id {
uuid {
left: 14120617547405019334
right: 9234047057017627141
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_SERVICE_ROUTER_TIER1
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false] and existing resource null
2025-10-28T07:18:00.821Z INFO workerTaskExecutor-1-48 GroupingObjectsEntityListener 111131 INVENTORY [nsx@6876 comp="nsx-manager" level="INFO" s2comp="grouping" subcomp="manager"] GroupingObjectsEntityListener : End
2025-10-28T07:18:00.822Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created LogicalPort [id=68ad6481-6823-4219-9884-5a6a49be911c, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=TransportZone/b50e5af4-aa77-4f91-8814-d4ae6fb0ced7, transportZoneType=OVERLAY, attachmentId=c3f682d8-5a39-48c6-8025-ecdfd0c6e205, attachmentType=internal_sr_tier1, internalPortAttachment=attachment_id {
uuid {
left: 14120617547405019334
right: 9234047057017627141
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_SERVICE_ROUTER_TIER1
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false], total time in msec = 6
2025-10-28T07:18:00.822Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created LogicalPort: LogicalPort [id=68ad6481-6823-4219-9884-5a6a49be911c, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=TransportZone/b50e5af4-aa77-4f91-8814-d4ae6fb0ced7, transportZoneType=OVERLAY, attachmentId=c3f682d8-5a39-48c6-8025-ecdfd0c6e205, attachmentType=internal_sr_tier1, internalPortAttachment=attachment_id {
uuid {
left: 14120617547405019334
right: 9234047057017627141
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_SERVICE_ROUTER_TIER1
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false]
2025-10-28T07:18:00.822Z INFO workerTaskExecutor-1-48 L2Configurator 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating logical port 68ad6481-6823-4219-9884-5a6a49be911c and linking to logical router port c3f682d8-5a39-48c6-8025-ecdfd0c6e205
2025-10-28T07:18:00.822Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Attaching TIER1 router link port LogicalRouterPortConfig/c3f682d8-5a39-48c6-8025-ecdfd0c6e205 to l2 port 68ad6481-6823-4219-9884-5a6a49be911c
2025-10-28T07:18:00.822Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/0a31477f-46d8-4acd-b08b-73e86eebca43 of type INTERNAL_TRANSIT for router LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d connected to l2 port null
2025-10-28T07:18:00.823Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif: LogicalRouterPortConfig/7b0f9674-59ca-5684-afc3-252a396dae49 of type TIER0_ROUTER_LINK for router LogicalRouterConfig/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b connected to l2 port null
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Logical port validations took 1 msec
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Begin temporary allocation of Ip and/or Mac resources for logical port InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3 in transaction at 1761635880824
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] End temporary allocation of Ip and/or Mac resources for logical port InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, total time taken = 0
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Begin create logical port InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3 in transaction at 1761635880824
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 PolicyUfoUtils 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Stored LogicalPort InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3 in internal store
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] LogicalPort created with ID: [InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3], total time in create transaction = 0
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 FirewallEntityListenerImpl 111131 FIREWALL [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] onEvent: Create resource: LogicalPort [id=5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=null, transportZoneType=OVERLAY, attachmentId=7b0f9674-59ca-5684-afc3-252a396dae49, attachmentType=internal_dr_tier0, internalPortAttachment=attachment_id {
uuid {
left: 8867471617782601348
right: 12665007440378310217
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_DISTRIBUTED_ROUTER_TIER0
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false], existingResource: null
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 GroupingObjectsEntityListener 111131 INVENTORY [nsx@6876 comp="nsx-manager" level="INFO" s2comp="grouping" subcomp="manager"] GroupingObjectsEntityListener : Start Grouping Framework received Event com.vmware.nsx.management.entitylistener.service.EntityEvent@38138315 for resource LogicalPort [id=5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=null, transportZoneType=OVERLAY, attachmentId=7b0f9674-59ca-5684-afc3-252a396dae49, attachmentType=internal_dr_tier0, internalPortAttachment=attachment_id {
uuid {
left: 8867471617782601348
right: 12665007440378310217
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_DISTRIBUTED_ROUTER_TIER0
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false] and existing resource null
2025-10-28T07:18:00.824Z INFO workerTaskExecutor-1-48 GroupingObjectsEntityListener 111131 INVENTORY [nsx@6876 comp="nsx-manager" level="INFO" s2comp="grouping" subcomp="manager"] GroupingObjectsEntityListener : End
2025-10-28T07:18:00.825Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created LogicalPort [id=5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=TransportZone/b50e5af4-aa77-4f91-8814-d4ae6fb0ced7, transportZoneType=OVERLAY, attachmentId=7b0f9674-59ca-5684-afc3-252a396dae49, attachmentType=internal_dr_tier0, internalPortAttachment=attachment_id {
uuid {
left: 8867471617782601348
right: 12665007440378310217
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_DISTRIBUTED_ROUTER_TIER0
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false], total time in msec = 2
2025-10-28T07:18:00.825Z INFO workerTaskExecutor-1-48 LogicalPortServiceImpl 111131 SWITCHING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created LogicalPort: LogicalPort [id=5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, intentPath=, logicalPortState=UP, ephemeral=false, logicalSwitchId=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, transportZoneId=TransportZone/b50e5af4-aa77-4f91-8814-d4ae6fb0ced7, transportZoneType=OVERLAY, attachmentId=7b0f9674-59ca-5684-afc3-252a396dae49, attachmentType=internal_dr_tier0, internalPortAttachment=attachment_id {
uuid {
left: 8867471617782601348
right: 12665007440378310217
}
}
attachment_type: ATTACHMENT_TYPE_LINK
link_attachment {
router_type: CCP_ROUTER_TYPE_DISTRIBUTED_ROUTER_TIER0
}
, switchingProfileIds=[SwitchingProfileTypeIdEntry [type=SwitchSecuritySwitchingProfile, profileId=47ffda0e-035f-4900-83e4-0a2086813ede], SwitchingProfileTypeIdEntry [type=SpoofGuardSwitchingProfile, profileId=fad98876-d7ff-11e4-b9d6-1681e6b88ec1], SwitchingProfileTypeIdEntry [type=IpDiscoverySwitchingProfile, profileId=64814784-7896-3901-9741-badeff705639], SwitchingProfileTypeIdEntry [type=MacManagementSwitchingProfile, profileId=1e7101c8-cfef-415a-9c8c-ce3d8dd078fb], SwitchingProfileTypeIdEntry [type=PortMirroringSwitchingProfile, profileId=null], SwitchingProfileTypeIdEntry [type=QosSwitchingProfile, profileId=f313290b-eba8-4262-bd93-fab5026e9495]], switchMode=STANDARD, extraConfigs=null, systemExtraConfigs=null, internalId=, initState=null, tags=null, pendingConfigFromHostd=false, addressBindings=null, ignoreAddressBindings=null, isIndependentVifPort=false, context=, isEsxVmk=false, isDvport=false]
2025-10-28T07:18:00.825Z INFO workerTaskExecutor-1-48 L2Configurator 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating logical port 5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3 and linking to logical router port 7b0f9674-59ca-5684-afc3-252a396dae49
2025-10-28T07:18:00.825Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Attaching TIER0 router link port LogicalRouterPortConfig/7b0f9674-59ca-5684-afc3-252a396dae49 to l2 port 5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3
2025-10-28T07:18:00.827Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating Lif LogicalRouterPortConfig/aa1508c8-6c34-4bcb-b47a-58c0ca1dc450 of type LOOPBACK for router LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d
2025-10-28T07:18:00.827Z INFO workerTaskExecutor-1-48 LRServiceIPUtils 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Creating Tier1toTier0ArpProxyTrigger for Tier0 LogicalRouter/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b Tier1 LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.828Z INFO workerTaskExecutor-1-48 AdvertisementConfigPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] RA - Removing all advertised routes... for attachment state 'LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be' and tlr 'LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be' / plr 'LogicalRouter/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b'
2025-10-28T07:18:00.830Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Port container c833ac6e-df2d-41ee-9729-9f18ad83a5ce updated for logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.832Z INFO workerTaskExecutor-1-48 LrLrpCCPPublisher 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated meta state for logical router eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.840Z INFO workerTaskExecutor-1-48 LogicalRouterWorker 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing completed for work items {LrPort=[WorkItem{identifier=LrPort/c3f682d8-5a39-48c6-8025-ecdfd0c6e205, Timestamp{epoch=1299, address=1150508844}}]} of logical router LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.840Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
5cc38939-6832-4d28-be28-58ffd22d4513
}
with operation UPDATE, adding work-item WorkItem{identifier=LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513, Timestamp{epoch=1299, address=1150508865}} for processing
2025-10-28T07:18:00.842Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalSwitch and Identifier LogicalSwitch/5cc38939-6832-4d28-be28-58ffd22d4513.
2025-10-28T07:18:00.843Z INFO workerTaskExecutor-1-48 WorkerInvocationTask 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Worker processing done , worker-id - LogicalRouterWorker
2025-10-28T07:18:00.846Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
091385fb-3daa-4f5c-92eb-94bb72b82b4e
}
with operation UPDATE, adding work-item WorkItem{identifier=L3Trigger/091385fb-3daa-4f5c-92eb-94bb72b82b4e, Timestamp{epoch=1299, address=1150508869}} for processing
2025-10-28T07:18:00.846Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3
}
with operation UPDATE, adding work-item WorkItem{identifier=InternalLogicalPort/5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3, Timestamp{epoch=1299, address=1150508869}} for processing
2025-10-28T07:18:00.846Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
68ad6481-6823-4219-9884-5a6a49be911c
}
with operation UPDATE, adding work-item WorkItem{identifier=InternalLogicalPort/68ad6481-6823-4219-9884-5a6a49be911c, Timestamp{epoch=1299, address=1150508869}} for processing
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be.
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding entry to Status tracker Entity map table LogicalRouterConfig, identifier LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be, current size 2
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d.
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding entry to Status tracker Entity map table LogicalRouterConfig, identifier LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d, current size 2
2025-10-28T07:18:00.847Z WARN RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Table LogicalRouterPortConfig not supported by CCP for Realization tracking. So skipping it.
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003200-0000-0000-0000-000000001824.
2025-10-28T07:18:00.847Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003600-0000-0000-0000-000000001824.
2025-10-28T07:18:00.883Z INFO UfoDelayedIndexer-0 AggSvcLogicalPortService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received logical router port status UP for port 68ad6481-6823-4219-9884-5a6a49be911c
2025-10-28T07:18:00.883Z INFO UfoDelayedIndexer-0 AggSvcLogicalPortService 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received logical router port status UP for port 5b3c5adc-bd55-4fa7-b507-1ad8c1dde1a3
2025-10-28T07:18:00.916Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
051f8198-ea03-4a1d-9359-27b8705f7535
}
with operation UPDATE, adding work-item WorkItem{identifier=BgpConfig/051f8198-ea03-4a1d-9359-27b8705f7535, Timestamp{epoch=1299, address=1150508871}} for processing
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/e8df89dc-991b-4037-bc50-ecd1c1415c1d.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node LogicalRouterConfig/e8df89dc-991b-4037-bc50-ecd1c1415c1d
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding entry to Status tracker Entity map table LogicalRouterConfig, identifier LogicalRouterConfig/e8df89dc-991b-4037-bc50-ecd1c1415c1d, current size 2
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/3ed5513c-f1f5-4197-a7a2-5b0262646765.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/d9e788cd-7ec9-4b58-8c6f-37e5f8f8fa2b.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename LogicalRouterConfig and Identifier LogicalRouterConfig/c8612c2d-76f1-4faf-bf1d-edf84884f90b.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003200-0000-0000-0000-000000000c03.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node NatSection/00003200-0000-0000-0000-000000000c03
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Initializing entry to Status tracker Cache table NatSection, identifier NatSection/00003200-0000-0000-0000-000000000c03
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Status Tracker table record from Corfu tablename NatSection and Identifier NatSection/00003600-0000-0000-0000-000000000c03.
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 RealizationStatusTrackerStreamListenerImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Entity belongs to this Node NatSection/00003600-0000-0000-0000-000000000c03
2025-10-28T07:18:00.917Z INFO RealizationStatusTrackerStreamListenerImpl-1-1 StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Adding entry to Status tracker Entity map table NatSection, identifier NatSection/00003600-0000-0000-0000-000000000c03, current size 1
2025-10-28T07:18:00.949Z INFO http-nio-127.0.0.1-7440-exec-70 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:18:00.967Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.968Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1009, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1009
2025-10-28T07:18:00.968Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1009
2025-10-28T07:18:00.969Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.970Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.970Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1015, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1015
2025-10-28T07:18:00.970Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1015
2025-10-28T07:18:00.970Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.971Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.971Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1053, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1053
2025-10-28T07:18:00.971Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1053
2025-10-28T07:18:00.971Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.972Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.972Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:00.972Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:00.972Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.973Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.974Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1037, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1037
2025-10-28T07:18:00.974Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1037
2025-10-28T07:18:00.974Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.974Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.975Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1038, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1038
2025-10-28T07:18:00.975Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1038
2025-10-28T07:18:00.975Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.976Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.976Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1045, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1045
2025-10-28T07:18:00.976Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1045
2025-10-28T07:18:00.976Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.977Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:00.977Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1001, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1001
2025-10-28T07:18:00.977Z INFO http-nio-127.0.0.1-7440-exec-70 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1001
2025-10-28T07:18:00.977Z INFO http-nio-127.0.0.1-7440-exec-70 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="f3727a8e-59ab-4188-8edb-93dc8d37d043" subcomp="manager" username="admin"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:00.979Z INFO http-nio-127.0.0.1-7440-exec-78 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:18:00.980Z INFO http-nio-127.0.0.1-7440-exec-78 LoggingRestTemplate 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" reqId="d26542a0-fdc7-41a5-ad56-c6633aacd04f" subcomp="manager" username="admin"] Request::URI:http://localhost:7443/cm-inventory/api/v1/fabric/discovered-nodes?page_size=1000 method:GET
2025-10-28T07:18:00.990Z INFO http-nio-127.0.0.1-7440-exec-74 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:18:01.000Z INFO http-nio-127.0.0.1-7440-exec-59 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to admin from x-nsx-username header.
2025-10-28T07:18:01.000Z INFO http-nio-127.0.0.1-7440-exec-59 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="14000645-7b19-47d1-97fd-db6532bdb8ea" subcomp="manager" username="admin"] Reading Cluster Id file /config/cluster-manager/clusterId
2025-10-28T07:18:01.000Z INFO http-nio-127.0.0.1-7440-exec-59 ApplianceConfigServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="14000645-7b19-47d1-97fd-db6532bdb8ea" subcomp="manager" username="admin"] Using cluster id ce8b394d-34ed-4c05-8645-e265d6a334d6 from file /config/cluster-manager/clusterId
2025-10-28T07:18:01.003Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-1 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 to:SUCCESS
2025-10-28T07:18:01.004Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-1 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-1s/D1-A2-Z1-N229/security-config to:SUCCESS
2025-10-28T07:18:01.005Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-0 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-1s/D1-A2-Z1-N229 to:SUCCESS
2025-10-28T07:18:01.006Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-1 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-0s/T0-GW-EC-01 to:SUCCESS
2025-10-28T07:18:01.066Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
091385fb-3daa-4f5c-92eb-94bb72b82b4e
}
with operation DELETE, adding work-item WorkItem{identifier=L3Trigger/091385fb-3daa-4f5c-92eb-94bb72b82b4e, Timestamp{epoch=1299, address=1150508883}} for processing
2025-10-28T07:18:01.158Z INFO metrics-sha-reporter-2-thread-1 VmcShaReporter 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" s2comp="vmc-sha-reporter" subcomp="manager"] VmcShaReporter.report begin
2025-10-28T07:18:01.226Z INFO metrics-sha-reporter-2-thread-1 VmcShaReporter 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" s2comp="vmc-sha-reporter" subcomp="manager"] VmcShaReporter.report end
2025-10-28T07:18:02.340Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Removing total 3 Identifiers from Temp map for table name LogicalRouterConfig. Total existing Identifiers [RetryWithMaxRetryCount{retryCount=0, id=LogicalRouterConfig/e8df89dc-991b-4037-bc50-ecd1c1415c1d}, RetryWithMaxRetryCount{retryCount=0, id=LogicalRouterConfig/eac00880-f5ab-43a6-a3db-342f922cb9be}, RetryWithMaxRetryCount{retryCount=0, id=LogicalRouterConfig/a2bd30b4-4976-4016-b411-8f186784373d}]
2025-10-28T07:18:02.340Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Removing total 2 Identifiers from Temp map for table name NatSection. Total existing Identifiers [RetryWithMaxRetryCount{retryCount=0, id=NatSection/00003200-0000-0000-0000-000000000c03}, RetryWithMaxRetryCount{retryCount=0, id=NatSection/00003600-0000-0000-0000-000000000c03}]
2025-10-28T07:18:02.342Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker EdgeRealizationUtil 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Making call to get current entity realization status for entityId LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be and entityType nsx.LogicalRouterConfig
2025-10-28T07:18:02.342Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Preparing current realization status. Request = 'Entity = '[entityId='LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be', entityType='nsx.LogicalRouterConfig']', Related Entities = 'null'.'.
2025-10-28T07:18:02.348Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Using barrier '5407' to query CCP.
2025-10-28T07:18:02.349Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'abcf3244-b980-473b-9785-39cee7a4a003'. Node id = 'cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d', APH Endpoint = 'nsx://7fd6e1a5-956e-4f4b-889f-8c0104b539c4'.
2025-10-28T07:18:02.349Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'abcf3244-b980-473b-9785-39cee7a4a003'. Node id = '2ed72b42-f887-4baf-a0e1-692fce8c56e6', APH Endpoint = 'nsx://100696ac-0899-44e2-b83e-4112266d9318'.
2025-10-28T07:18:02.349Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'abcf3244-b980-473b-9785-39cee7a4a003'. Node id = '9ee7b097-95aa-4ccc-99ae-ea777b9cde3d', APH Endpoint = 'nsx://f31d82bb-d072-4ce5-823d-179df8960838'.
2025-10-28T07:18:02.350Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-8 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'abcf3244-b980-473b-9785-39cee7a4a003' and ResponseMap size '1'.
2025-10-28T07:18:02.350Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-5 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'abcf3244-b980-473b-9785-39cee7a4a003' and ResponseMap size '2'.
2025-10-28T07:18:02.351Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-5 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'abcf3244-b980-473b-9785-39cee7a4a003' and ResponseMap size '3'.
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received responses from '3' controllers '[2ed72b42-f887-4baf-a0e1-692fce8c56e6, cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d, 9ee7b097-95aa-4ccc-99ae-ea777b9cde3d]'.
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker TransportNodeResponseHelper 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Setting aggregate status to CCP status 'SUCCESS' and converged barrier number to '110058' as there are no transport nodes in span.
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning current realization status 'Status = 'SUCCESS', Message = 'There are no transport nodes in the span.', TNs = '[]', Pending Changes = '[81a97a67-95b1-4bb1-a3f1-1a0024d5bcfb, 826c408c-fd4c-4daf-959c-80c5762a9c7b, 09e6832c-373a-4296-9a70-7e01043246a8]'.' for entity 'LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be'.
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker EdgeRealizationUtil 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Completed call to get current entity realization status for entityId LogicalRouter/eac00880-f5ab-43a6-a3db-342f922cb9be, entityType nsx.LogicalRouterConfig.
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker EdgeRealizationUtil 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Making call to get current entity realization status for entityId LogicalSrClusterConfig/00002000-0000-0000-0000-000000001824 and entityType nsx.LogicalSrClusterConfig
2025-10-28T07:18:02.351Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Preparing current realization status. Request = 'Entity = '[entityId='LogicalSrClusterConfig/00002000-0000-0000-0000-000000001824', entityType='nsx.LogicalSrClusterConfig']', Related Entities = 'null'.'.
2025-10-28T07:18:02.352Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Using barrier '5407' to query CCP.
2025-10-28T07:18:02.352Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d'. Node id = 'cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d', APH Endpoint = 'nsx://7fd6e1a5-956e-4f4b-889f-8c0104b539c4'.
2025-10-28T07:18:02.352Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d'. Node id = '2ed72b42-f887-4baf-a0e1-692fce8c56e6', APH Endpoint = 'nsx://100696ac-0899-44e2-b83e-4112266d9318'.
2025-10-28T07:18:02.352Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d'. Node id = '9ee7b097-95aa-4ccc-99ae-ea777b9cde3d', APH Endpoint = 'nsx://f31d82bb-d072-4ce5-823d-179df8960838'.
2025-10-28T07:18:02.354Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-3 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d' and ResponseMap size '1'.
2025-10-28T07:18:02.354Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-4 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d' and ResponseMap size '2'.
2025-10-28T07:18:02.354Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-4 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'c84b3bc1-cb60-4737-8ab0-88fdc75f9a7d' and ResponseMap size '3'.
2025-10-28T07:18:02.354Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received responses from '3' controllers '[2ed72b42-f887-4baf-a0e1-692fce8c56e6, cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d, 9ee7b097-95aa-4ccc-99ae-ea777b9cde3d]'.
2025-10-28T07:18:02.354Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning current realization status 'Status = 'SUCCESS', Message = 'All '2' transport nodes have successfully processed the state.', TNs = '[TN = 'f9d634f1-06e2-4247-a110-74df031320ba', Status = 'SUCCESS', Message = '', Errors = '[]'., TN = '65537cac-4c82-4c19-9660-6f75c79b0d2a', Status = 'SUCCESS', Message = '', Errors = '[]'.]', Pending Changes = '[826c408c-fd4c-4daf-959c-80c5762a9c7b, 09e6832c-373a-4296-9a70-7e01043246a8]'.' for entity 'LogicalSrClusterConfig/00002000-0000-0000-0000-000000001824'.
2025-10-28T07:18:02.354Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker EdgeRealizationUtil 111131 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Completed call to get current entity realization status for entityId LogicalSrClusterConfig/00002000-0000-0000-0000-000000001824, entityType nsx.LogicalSrClusterConfig.
2025-10-28T07:18:02.363Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created publish status for GPRR /infra/realized-state/enforcement-points/default/routing-advertisements/D1-A2-Z1-N229-network-rtadv
2025-10-28T07:18:02.363Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating GPRR /infra/realized-state/enforcement-points/default/routing-advertisements/D1-A2-Z1-N229-network-rtadv with publish status: SUCCESS
2025-10-28T07:18:02.370Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated GPRR with Publish status SUCCESS.
2025-10-28T07:18:02.378Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Created publish status for GPRR /infra/realized-state/enforcement-points/default/tier-1-logical-routers/D1-A2-Z1-N229-t1
2025-10-28T07:18:02.378Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updating GPRR /infra/realized-state/enforcement-points/default/tier-1-logical-routers/D1-A2-Z1-N229-t1 with publish status: SUCCESS
2025-10-28T07:18:02.385Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTrackerHelperService 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated GPRR with Publish status SUCCESS.
2025-10-28T07:18:02.386Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker StatusTracker 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Removing table name entry from Temp Entity Map : LogicalRouterConfig
2025-10-28T07:18:02.386Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Preparing current realization status. Request = 'Entity = '[entityId='NatSection/00003200-0000-0000-0000-000000000c03', entityType='nsx.NatSection']', Related Entities = 'null'.'.
2025-10-28T07:18:02.390Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Using barrier '5407' to query CCP.
2025-10-28T07:18:02.392Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235'. Node id = 'cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d', APH Endpoint = 'nsx://7fd6e1a5-956e-4f4b-889f-8c0104b539c4'.
2025-10-28T07:18:02.392Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235'. Node id = '2ed72b42-f887-4baf-a0e1-692fce8c56e6', APH Endpoint = 'nsx://100696ac-0899-44e2-b83e-4112266d9318'.
2025-10-28T07:18:02.392Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235'. Node id = '9ee7b097-95aa-4ccc-99ae-ea777b9cde3d', APH Endpoint = 'nsx://f31d82bb-d072-4ce5-823d-179df8960838'.
2025-10-28T07:18:02.393Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-5 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235' and ResponseMap size '1'.
2025-10-28T07:18:02.393Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-0 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235' and ResponseMap size '2'.
2025-10-28T07:18:02.394Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-6 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId 'ceaeb6c0-6b2b-464e-a18b-7b74d5744235' and ResponseMap size '3'.
2025-10-28T07:18:02.394Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received responses from '3' controllers '[2ed72b42-f887-4baf-a0e1-692fce8c56e6, cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d, 9ee7b097-95aa-4ccc-99ae-ea777b9cde3d]'.
2025-10-28T07:18:02.394Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker TransportNodeResponseHelper 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Setting aggregate status to CCP status 'SUCCESS' and converged barrier number to '110058' as there are no transport nodes in span.
2025-10-28T07:18:02.394Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning current realization status 'Status = 'SUCCESS', Message = 'There are no transport nodes in the span.', TNs = '[]', Pending Changes = '[]'.' for entity 'NatSection/00003200-0000-0000-0000-000000000c03'.
2025-10-28T07:18:02.395Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Preparing current realization status. Request = 'Entity = '[entityId='NatSection/00003600-0000-0000-0000-000000000c03', entityType='nsx.NatSection']', Related Entities = 'null'.'.
2025-10-28T07:18:02.396Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Using barrier '5407' to query CCP.
2025-10-28T07:18:02.397Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request '1afd67c6-0415-43b7-8476-3f17c891fe1d'. Node id = 'cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d', APH Endpoint = 'nsx://7fd6e1a5-956e-4f4b-889f-8c0104b539c4'.
2025-10-28T07:18:02.397Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request '1afd67c6-0415-43b7-8476-3f17c891fe1d'. Node id = '2ed72b42-f887-4baf-a0e1-692fce8c56e6', APH Endpoint = 'nsx://100696ac-0899-44e2-b83e-4112266d9318'.
2025-10-28T07:18:02.397Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Sending realization status request '1afd67c6-0415-43b7-8476-3f17c891fe1d'. Node id = '9ee7b097-95aa-4ccc-99ae-ea777b9cde3d', APH Endpoint = 'nsx://f31d82bb-d072-4ce5-823d-179df8960838'.
2025-10-28T07:18:02.398Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-4 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId '1afd67c6-0415-43b7-8476-3f17c891fe1d' and ResponseMap size '1'.
2025-10-28T07:18:02.398Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-2 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId '1afd67c6-0415-43b7-8476-3f17c891fe1d' and ResponseMap size '2'.
2025-10-28T07:18:02.399Z INFO nsx-rpc:RPC_PROXY_CONN_PROVIDER:user-executor-8 CallContext 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received Realization RPC callback with CorrelationId '1afd67c6-0415-43b7-8476-3f17c891fe1d' and ResponseMap size '3'.
2025-10-28T07:18:02.399Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationRpcClient 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Received responses from '3' controllers '[2ed72b42-f887-4baf-a0e1-692fce8c56e6, cc0ff611-6fb0-4cd8-a26a-d51b39d0a31d, 9ee7b097-95aa-4ccc-99ae-ea777b9cde3d]'.
2025-10-28T07:18:02.399Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker TransportNodeResponseHelper 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Setting aggregate status to CCP status 'SUCCESS' and converged barrier number to '110058' as there are no transport nodes in span.
2025-10-28T07:18:02.399Z INFO com.vmware.nsx.management.policy.policyframework.realization.StatusTracker RealizationStateServiceImpl 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning current realization status 'Status = 'SUCCESS', Message = 'There are no transport nodes in the span.', TNs = '[]', Pending Changes = '[]'.' for entity 'NatSection/00003600-0000-0000-0000-000000000c03'.
2025-10-28T07:18:03.005Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-1 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/domains/default/gateway-policies/Policy_Default_Infra-tier1-D1-A2-Z1-N229 to:SUCCESS
2025-10-28T07:18:03.005Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-0 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-1s/D1-A2-Z1-N229 to:SUCCESS
2025-10-28T07:18:03.006Z INFO INTENT-PROCESSOR-CONSOLIDATED-SERVICE-1 ConsolidatedRealizedStateServiceImpl 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Updated consolidated state for intentPath:/infra/tier-0s/T0-GW-EC-01 to:SUCCESS
2025-10-28T07:18:03.210Z INFO http-nio-127.0.0.1-7440-exec-75 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to node-mgmt from x-nsx-username header.
2025-10-28T07:18:05.509Z INFO Watcher-of-/config/envoy MultiFileWatcher 111131 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Send update for ip.properties for event ENTRY_MODIFY to observer
2025-10-28T07:18:05.509Z INFO Watcher-of-/config/envoy ConfigFileWatcher 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing change to config file ip.properties
2025-10-28T07:18:05.516Z INFO Watcher-of-/config/envoy NsxEnvoyConfig 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Listener 1/4: https-vip-10.210.5.20 address: "10.210.5.20"
port_value: 443
ipv4_compat: true

2025-10-28T07:18:05.516Z INFO Watcher-of-/config/envoy NsxEnvoyConfig 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Listener 2/4: https-node-v4-10.210.5.21 address: "10.210.5.21"
port_value: 443
ipv4_compat: true

2025-10-28T07:18:05.516Z INFO Watcher-of-/config/envoy NsxEnvoyConfig 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Listener 3/4: https-node-v4-local address: "127.0.0.1"
port_value: 443
ipv4_compat: true

2025-10-28T07:18:05.516Z INFO Watcher-of-/config/envoy NsxEnvoyConfig 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Listener 4/4: internal-preauthenticated address: "127.0.0.1"
port_value: 6567
ipv4_compat: true

2025-10-28T07:18:05.574Z INFO Watcher-of-/config/envoy NsxEnvoyManager 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Wrote nsxEnvoyConfig with version 1731258674232937 to config file
2025-10-28T07:18:05.648Z INFO WorkerCorfuStreamListener-1-1 WorkerCorfuStreamListener 111131 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] DCN:received for uuid {
39577cff-be20-4fb8-ae23-cd1535c69ee5
}
with operation UPDATE, adding work-item WorkItem{identifier=HostModelMsg/39577cff-be20-4fb8-ae23-cd1535c69ee5, Timestamp{epoch=1299, address=1150508984}} for processing
2025-10-28T07:18:05.712Z INFO UfoIndexer-BatchExecutor-search_manager-1 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:05.712Z INFO UfoIndexer-BatchExecutor-search_manager-1 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:05.713Z INFO UfoIndexer-BatchExecutor-search_manager-1 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:05.714Z INFO UfoIndexer-BatchExecutor-search_manager-1 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:05.718Z INFO UfoIndexer-BatchExecutor-search_policy-2 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered nodes for given hardware id , having size 0
2025-10-28T07:18:05.718Z INFO UfoIndexer-BatchExecutor-search_policy-2 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered node by external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:05.718Z INFO UfoIndexer-BatchExecutor-search_policy-2 DiscoveredNodeFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning discovered nodes parent id as 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031 for given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:host-1035
2025-10-28T07:18:05.718Z INFO UfoIndexer-BatchExecutor-search_policy-2 ComputeCollectionFinderImpl 111131 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Returning Compute collection found by given external id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031, has id 669d6730-4016-4a0d-aff7-5fe80e10f386:domain-c1031
2025-10-28T07:18:05.725Z INFO UfoIndexer-BatchExecutor-search_policy-2 TransportNodeStatusServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Management connection status for node 39577cff-be20-4fb8-ae23-cd1535c69ee5 is UP
2025-10-28T07:18:05.726Z INFO UfoIndexer-BatchExecutor-search_manager-1 TransportNodeStatusServiceImpl 111131 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Management connection status for node 39577cff-be20-4fb8-ae23-cd1535c69ee5 is UP
2025-10-28T07:18:05.727Z INFO UfoIndexer-BatchExecutor-search_policy-2 HeatMapServiceImpl 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] isBfdEnable = true, isEvpnEnable = true
2025-10-28T07:18:05.727Z INFO UfoIndexer-BatchExecutor-search_manager-1 HeatMapServiceImpl 111131 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] isBfdEnable = true, isEvpnEnable = true
2025-10-28T07:18:06.052Z INFO http-nio-127.0.0.1-7440-exec-77 PreAuthenticationFilter 111131 PreAuthenticationFilter setting username to node-mgmt from x-nsx-username header.
^C
Tue Oct 28 2025 EET 09:17:55.825
nsx-manager-01>

[nvazquez]

Thanks @grehenl - from the stack trace it seems the error comes from NSX not finding a proper transport zone on the segment creation, on the method: validateNonDiscoveredSegmentUsingSecurityTZ throwing the NullPointerException. Do you have the Transport zone referenced on the nsx_providers table in CloudStack?

[grehenl]

@nvazquez That's what i though too, but the overlay transport zone for the segments is correct in Cloudstack:

(dh) 🐱 > list nsxcontrollers
{
"count": 1,
"nsxController": [
{
"edgecluster": "EdgeCluster-01",
"hostname": "nsx.lab.local",
"name": "NSX",
"nsxprovideruuid": "8ee19900-1b9d-468f-a782-595a7775c3fc",
"port": "443",
"tier0gateway": "T0-GW-EC-01",
"transportzone": "esxi-overlay-tz",
"zoneid": "9579a160-77be-41ef-bbeb-da27bf164b81",
"zonename": "AZ1-VMWARE"
}
]
}

The esxi-overlay-tz is the TZ that all of the esxi and edge nodes are using. Isn't the transportzone in Cloudstack the overlay tz that i use in NSX? Should i use the ID or the URL path (or anything else) instead of just the tz name?