Providing multiple RTSP streams under single output from RtspServer #15
Comments
|
You want to do something like a camera tour where you tour between each stream? It should be very easy to achieve provided the underlying streams are using the same codec, you can essentially just create a RtspStream and a timer and switch which underlying packets are going to the output RtspStream based on your desired logic, e.g. a counter to select from the proper underlying streams packets. The logic would essentially on elapse of the timer attach to the events from the desired stream and enqueue them into the output stream for playback. Give me a try and show me what you have tried and I can better assist you from there! The code would be something like this:
Where someStream is the underlying stream you want to temporarily Take a look @ https://github.com/juliusfriedman/net7mma_core/blob/master/UnitTests/Program.cs#L2125 |
|
Well, may be it is dumb question. I've tried to use I can play original stream in VLC, but stream from server doesn't work in VLC. When I try to connect to Full log: |
|
You only changed values for MP4 Boxes / Atoms, I think the values are actually in BigEndian, I have to double check the spec... I am pretty sure there is something else wrong I will check the tests asap and let you know |
|
I checked your code, everything seems okay but I am able to confirm over UDP there is either something wrong with your server or the parsing logic with the SDP, please do a wireshark capture of the RtspClientTest on your stream and submit it here so I can take a look when I have time. I verified over TCP your stream works correctly with the RtspClient as well as the server through VLC. Please take a look at why the issue occurs over UDP and let me know if you need my help, especially if you dont find anything on your side which causes the UDP session to fail. |
|
It seems something broke in the RtspClient, its missing the Content of the Describe request sometimes... If you add this logic (hack) it seems to work.. It's too late for me to check into this further but at least you know works over TCP. Please try and review your changes in the RtspClient around SendDescribe to see why the RtspClient is failing. I believe it might have something to do with the modified return but I cannot spot it easily. Talk to you soon! |
|
BTW, something is clearly happening with your server and UDP connections, it supplies and SDP the first time but then apparently stalls out and never finished the SDP for the subsequent describes... In RtspClient @ StartPlaying you can also do something like this to verify it... |
|
I can confirm our recent changes in RtspClient / RtpClient result in a
I will continue to look into this when I have time but it probable will not be until the weekend. Please let me know if you find it before me. P.s. Tcp still works well (the problem only happens when using Udp) |
@juliusfriedman Could you please help me with this one more time? I'm doing forwarding from input to output streams like this: But looks like I do not understand your idea correctly and unable to play |
|
It's just a bit more complex than it needs to be in order to work. If you already have your output stream then its just a matter of calling packetize from the appropriate source.. You don't need the OutOfBandData and I personally would work with the RtspStream from the RtspServer rather than at the RtpClient level as it will decrease the complexity greatly. All you need is an output stream: Then from there you will attach to the output stream from the desired input stream like so: This will cause the packets to begin to flow from someStream to outputStream. Then upon a timer elapsing or otherwise you can easily stitch together different streams to create a Remember to disconnect the events from the If you need more help please let me know explicitly what. Thank you again for your other contributions. |
Ok, I've done exactly as needed: Query |
|
Did you verify if you can play your source stream from the server by itself? If so please check out this code which should work with only minor changes: Please keep in mind in this example both streams flow to output at the same time. I didn't add logic here to stop the flow from either stream, just wanted to show how to create the composite stream. P.s. This was already asked for and achieved before: https://github.com/juliusfriedman/net7mma/blob/master/UnitTests/Program.cs#L2498 |
|
P.s. The above code will allow your users to select a stream through VLC or whatever player they are using. If you don't need the stream to be selectable then you can skip having multiple tracks in the SDP and you can just use the code with a timer to switch out which source is attached to the compositeStream. Let me know if you have any further questions! |
|
|
|
My last attempt below. But it still causes artifacts in result video stream, hunging streaming, etc. |
|
|
It seems like you had data flowing from both streams to the output source at the same time... This is what caused the artifacts I assume, if you just leave one stream flowing through the compositeStream then you should not see any artifacts? I think all that your missing is the removal of the events from the stream you don't want to display, It seems like why there are artifacts, because your sending data from both streams at the same time. I dont' see how the I will check out your code but if you could provide me the link to 2 working streams it would be helpful for me to setup an example for you when I get time. |
|
rtsp://8.15.251.101:1935/rtplive/R2_051 both are working. |
|
Even if i do not do composite streaming at all, just Wait a bit and you see artifacts in result stream. May be it is related to TCP proto, but VLC plays original stream without artifacts. |
|
|
I doo see the artifacts, they seem to be reduced with larger buffer size... I am seeing a lot of strange messages about buffer size exceeded, I will have to look into this to see exactly what is causing it. |
|
I made a small push which now includes your streams, I will get to the composite stuff tonight or this weekend and if not sometimes next week. |
|
I pushed up some code which swtiches between the 2 streams, you can setup the stream by pressing It seems to work okay for me, I need to look into why your cameras have so many artifacts in the stream, it seems like a bug on those cameras firmware or sometime of bandwidth exhaustion scenario... I will let you know what I find ASAP. |
|
I pushed up another small commit, it seems I will continue to look into this and let you know what I can find. |
|
I found a small bug where packets were being missed when the buffer was empty, its a small change but seems to remove a lot of artifacts. I will continue to look into why the remaining artifacts are present. (BTW if you use a really large buffer 10*1024) then it seems the artifacts clear up eventually. |
|
I pushed up another small change but overall I think we will have better luck with the |
|
I pushed up that change, it runs in VLC for hours at a time without any issues from what I can see, the artifacts come and go but I am sure we will track that down with time. It should be trivial for your you to add streams to this example. Please let me know if I can be of any other help! |
|
BTW, also to reduce the artifacts when switching you can create a keyframe with sps and pps and send it right when the events are attached, you should get the sps and pps from the mediaDescription of the source stream, here is the example... If you do this then it helps to reduce the artifacts a bit right at the time of switching. |
|
I pushed up some changes, it seems your camera has a lot of extra data on the TCP side, If you run the RtspClient test and attach to the Interleave events with the I key you will see there is always extra data. I have verified this with Wireshark and although I am not sure where this is coming from yet or how VLC seems to deal with it so gracefully (without any debug messages in the log etc) it's something to be aware of for sure... I will continue to look into this when I have time. |
|
Still looking into this, it seems your camera is sending some other type of data (maybe sps and pps) which should be included in teh data stream. I am not sure how live 555/vlc parses it so elegantly especially with all the strange malformed packets but its something I will continue to look into and see where I can get. I will keep you updated. (BTW Thanks for your continued work on the code) |
|
I made a few hacks to the code to get your cameras to work under udp mode and switch gracefully. It was a combination of a bug (using the Connection: close header) and handling when the 2nd describe would not work. Vlc doens't seems to have that issue and I will continue to look into this to make it better where possible. |
|
Made another few small improvements to the sending of data so its not as much of a hack. Looking at the Wire capture there is definitely bandwidth exhaustion on the camera, possibly hardware failure. (I get a lot of malformed packets) Why VLC doesn't seem to have any issues or display any log errors / debug warnings is something I am still curious about and will continue to look into. |
|
Found that you can disable the keep alive requests with your cameras and it seems to reduce artifacts in interleaved mode especially considering all the malformed packets.
I will continue to look into this to see if there is anything else I am missing as well as investigate why VLC is providing so much better quality. FWIW, When using 0 as the bufferSize I get much less artifacts than I do even with a larger buffer (65540 -> 10*1024) |
|
In the latest code (Release) I can run your streams for over an hour and the artifacts are very minimal, please take a look and let me know what you think. |
|
I made some log statements only appear in debug and I still get a lot of artifacts from your stream, I am pretty sure its not performance related (or correctness related) as Wireshark reveals similar errors however both popular unmanaged libraries seems to be able to play from the stream without any artifacts... This leaves a bug in VLC/Live555 / ffmpeg of which I have not had the time to review their latest code in detail but its possible especially for Live555 to have some type of protocol bugs because of the way it implements RTSP... (http://www.live555.com/liveMedia/doxygen/html/RTSPClient_8hh_source.html) I also tried ffplay just to be sure and it seems that the streams play without artifacts using it so it just makes me very curious... You can see that there are some long pauses but there aren't really any major artifacts like there are with the server, its a result maybe of how they are holding onto packets for a longer time and re-ordering them before sending to the decoder BUT that shouldn't matter when going through the RtspServer or using the RtspClient. I will continue to look into this and let you know what I find but in the meantime you might have better quality pulling down the stream with ffmpeg / ffplay and making something like a FFMpegSource which can run from the command line and output over rtsp locally so it can be consumed from the server again with less artifacts. Thanks for your continued contributions and testing resources! |
|
A quick update before I head off for the night, I updated the source code to show you that PerPacket is working a lot better with ffplay, I see VLC also is improved but still has artifacts. It may save you having to create a FFMPEG Source after all...
Please test and let me know how you find it! |
|
I pushed up a quick few other fixes from testing, it seems that the camera sometimes would send a binary packet which was getting my logic tripped up, the packet was never parsed but it wasted some cpu... I adjusted for this logic (short packets) by ensuring the frame header was completely read before returning any frameLength, I also changed a few checks to look for frameChannel.HasValue It didn't happen a lot but when it did it seems to cause some of the pauses and artifacts. I am going to sleep now, will check back again in the morning. |
|
Things looked good from over here, I was able to run your streams for hours without any issues. I find VLC has more artifacts than ffplay but the artifacts come and go with time on VLC, when using FFplay it was much harder to notice any artifacts at all even during switching of streams. I think I am going to close this issue if I don't hear back from you with confirmation, Thank you again for your contributions and testing resources! |
|
Thank you! I'll definitely check you suggestions in a few days! |
|
Just pushed up a few more fixes, please let me know what you think asap! (Funny note, your cameras reply to a DESCRIBE * but then stop responding in the stream directly after that) You can press "X" when running the RtspClient test to see it) it will drop all RTP traffic for some reason right after it gets a DESCRIBE *.... Let me know if you notice anything else when testing! |
|
All PR's are merged and I did some basic testing, everything looks good to me especially with Thanks again for your contributions and testing resources! |
|
So I start to see some artifacts on ffplay as well... they eventually resolve but they are there especially more times than others. I will continue to look into what can be done to reduce the artifacts, it seems your cameras have a VBR and are sensitive to sequence number and timestamp changes as a result. When I play your streams individually through the server all the artifacts eventually clear up (even with VLC). and the Framerate is good. It's when we starting to use the Composite stream I notice most of the artifacts and lack of framerate and TBH its somewhat expected because of the way we are mixing streams from multiple different sources into a single stream and hoping the decoder does the right thing. Its more likely to work with less artifacts if all cameras have the same settings and (VBR) doesn't seems to lend itself to the best I Frame interval for doing this type of switching. I will continue to look into this but I will be away the next few days, let me know what you find as well! |
|
I pushed up a few fixes in anticipation of RTSP 2 support on these cameras... unfortunately they don't seem to support anything but 1.0. Still looking into why exactly the stream artifacts are so heavy when going through the rtsp server (especially on the composite stream) It would be ideal to get this to work without changing the cameras GOP size and I Frame interval. I will let you know what I can find if anything. |
|
I found that if we lower the clock rate slightly and force a timestamp jump right at the point of switching it reduces the artifacts greatly and VLC plays the stream much better. I pushed up those changes for you to take a look at. FF Play seems to be about the same but I will continue to test and look into this. Overall since the stream is stable minus these few artifacts which resolve themselves I think we can consider this issue closed after you verify the recent changes with the smaller ClockRate and TimeStamp changes right at the point of switching. Let me know what you think! |
I've a bunch of input RTSP streams, and want to export them under single RTSP stream from RtspServer.
For example
[rtsp://1, rtsp://2, rtsp://3]exported asrtsp://rtsp_server/uri.Streams should be exported as a circular buffer with some delay between. Ex 1 stream exported for 30 seconds,
then 2, etc.
RtspServer? Could you guide where to start?RtspServerinstances to scale approach above?For example:
The text was updated successfully, but these errors were encountered: