`NetStream::dispose()` (and maybe `NetStream::play()`, `NetStream::close()`) blocking operations

[itlancer]

Problem Description

NetStream::dispose() (and maybe NetStream::play(), NetStream::close()) blocking operations.
Any start/stop video playback cause huge render lag with performance hit and it cannot be used where need smooth media content playback.

Tested with multiple AIR versions, even with latest AIR 51.2.1.2 with multiple different devices, different platforms, OS versions with different applications and different videos.
Issue exists even with small low-resolution videos.
Tested with Video and VideoTexture.
The same issue in all cases.
Most critical for Android and Windows.

Related issues:
#3770
#3750
#3281
#3260
#3115
#2869
#2503
#2402
#2268
#2163
#2162
#1939
#1984
#1316 (reply in thread)
#1291
#646
#307
#224
#155
#151
#87
#15

Steps to Reproduce

Launch application with code below. It playback video in a loop and rotate red rectangle per each frame.

Application example with sources, sample of video and Scout log attached.
netstream_blocking_bug.zip

package {
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.net.NetConnection;
	import flash.net.NetStream;
	import flash.media.Video;
	import flash.events.NetStatusEvent;
	
	public class NetStreamBlockingBug extends Sprite {
		private var nc:NetConnection;
		private var ns:NetStream;
		private var video:Video;
		private var sprite:Sprite = new Sprite();
		
		public function NetStreamBlockingBug() {
			addEventListener(Event.ADDED_TO_STAGE, init);
		}
		
		private function init(e:Event):void {
			removeEventListener(Event.ADDED_TO_STAGE, init);
			
			sprite.graphics.beginFill(0xff0000);
			sprite.graphics.drawRect(0, 0, 100, 100);
			sprite.graphics.endFill();
			sprite.x = 100;
			sprite.y = 500;
			
			addEventListener(Event.ENTER_FRAME, enterFrame);
			
			startVideo();
		}
	
		private function startVideo():void {
			removeChildren();
			
			addChild(sprite);
			
			if (ns != null){
				video.attachNetStream(null);
				ns.removeEventListener(NetStatusEvent.NET_STATUS, nsHandler);
				ns.dispose();
			}
		
			if (video != null){
				video.clear();
			}
			
			video = new Video(640, 480);
			addChild(video);
			
			if (nc != null){
				nc.removeEventListener(NetStatusEvent.NET_STATUS, ncHandler);
				nc.close();
			}
			
			nc = new NetConnection();
			nc.addEventListener(NetStatusEvent.NET_STATUS, ncHandler);
			nc.connect(null);
		}
		
		private function ncHandler(e:NetStatusEvent):void {
			if (e.info.code == "NetConnection.Connect.Success"){
				ns = new NetStream(nc);
				ns.checkPolicyFile = false;
				ns.addEventListener(NetStatusEvent.NET_STATUS, nsHandler);				
				ns.client = {onMetaData:getMeta, onPlayStatus:onPlayStatus};
				
				video.attachNetStream(ns);
				ns.play("video.mp4");
			}
		}

		private function nsHandler(e:NetStatusEvent):void {
			trace(e.info.code);
			if (e.info.code == "NetStream.Play.Stop"){
				startVideo();
			}
		}

		private function getMeta(mdata:Object):void { }
	
		private function onPlayStatus(infoObject:Object):void {
			trace("onPlayStatus", infoObject.code);
		}
	
		public function enterFrame(e:Event):void {
			sprite.rotation += 1;
		}
	}
}

Actual Result:
On video stop/start you can see huge render lag (red rectangle stuck for ~100 milliseconds) (in archive, without conversion to see it clear):
netstream_blocking_bug_video.zip

In Scout you can see huge peaks:

Especially for NetStream::dispose() you can see that it blocks other render/logic at least for enormous 40-70 milliseconds (just one this call!) even with high-performance Intel i9 CPUs.

Expected Result:
Smooth render. Video playback start/stop should not affect render in main thread.

Known Workarounds

none
*write your own native extension to playback video

[ajwfrost]

We've made a couple of changes that improve things a little here - the time for the NetStream::dispose() call varies between 5ms-24ms now. There are some things it's doing that will take some time, and without a complete rearchitecture, we're unlikely to eliminate this. Mostly there are threads that are being shut down and which were each taking a bit of time to do that - but rather than waiting for them to actually exit, we can just orphan them and let them terminate themselves. Plus the closing down of the Windows media transform decoder takes a bit of time (but we can push that to another thread and again just let it happen separately).

Definitely something we can look at further though, if we do end up trying to rearchitect the way the runtime threads are organised, per that 307 issue..

[itlancer]

@ajwfrost
Great. If it could cause some side effects for now (before complete rearchitect) - maybe add new parameter like
NetStream::dispose(inAnotherThread:Boolean=false) for now...
This is most critical for Android and Windows right now.

[itlancer]

@ajwfrost
With AIR 51.2.1.4 with Windows it works better now. At least with high-end devices. We will made more tests soon.
But nothing changes for Android. Absolutely the same lags/consumtion time even with high-end Android devices:

Scout log attached:
netstream_blocking_bug_Android_51.2.1.4.zip

[itlancer]

Well, it much better with latest AIR versions. At least for Windows and Android.
It still not ideal. Especially for complex applications with animations and low-end devices launch/stop video cause visual lags for a whole application.
I hope it will be improved in the future to completely eliminate this affect. Alongside with #307

[itlancer]

@ajwfrost
So, with AIR 52.2.2.7 optimizations here reverted back and we get lags again (at least for Android). So issue actual once again. Could you please share plans/timeline what we should expected in future AIR versions about that?
For us doesn't matter way to playback video - it could be some new APIs, new multimedia pipeline etc. We just need to get robust and performant video playback.