Create a Video-Stream #892

mrRedDead98 · 2018-03-20T13:18:12Z

Hey guys,

I would like to transfer one of the Camera-views of AirSim into QGroundControl like the normal Video-Stream. In QGroundControl under General I found the Video section where by default Video Stream is Disabled and I can choose UDP Video Stream, RTSP Video Stream or TCP-MPEG2 Video Stream.

Also I found MavLinkVideoClient and MavLinkVideoServer in MavLinkCom, is there any possibility to send the Video-Data to QGC with this modules so that I could write a piece of code or is anywhere a solution which is working well.

I hope you can give me some ideas and help.

sytelus · 2018-03-23T05:45:01Z

@lovettchris might be interested in this one.

sytelus · 2018-04-12T05:07:46Z

We are not focusing on this feature currently but please feel free to contribute if this is important for you!

VedantMistry13 · 2018-08-24T15:07:00Z

If you wish to stream on a url, I found a way to stream the image feed on local server using python.
Required libraries are opencv, numpy, flask and the airsim library of course.
Just run the code given below with the simulation running and access the url http://localhost:5000/video_feed for getting the live stream.

from flask import Flask, render_template_string, Response

import airsim
import cv2
import numpy as np

client = airsim.MultirotorClient()
client.confirmConnection()

CAMERA_NAME = '0'
IMAGE_TYPE = airsim.ImageType.Scene
DECODE_EXTENSION = '.jpg'

def frame_generator():
    while (True):
        response_image = client.simGetImage(CAMERA_NAME, IMAGE_TYPE)
        np_response_image = np.asarray(bytearray(response_image), dtype="uint8")
        decoded_frame = cv2.imdecode(np_response_image, cv2.IMREAD_COLOR)
        ret, encoded_jpeg = cv2.imencode(DECODE_EXTENSION, decoded_frame)
        frame = encoded_jpeg.tobytes()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

app = Flask(__name__)

@app.route('/')
def index():
    return render_template_string(
        """
            <html>
            <head>
                <title>AirSim Streamer</title>
            </head>
            <body>
                <h1>AirSim Streamer</h1>
                <hr />
                Please use the following link: <a href="/video_feed">http://localhost:5000/video_feed</a>
            </body>
            </html>
        """
        )

@app.route('/video_feed')
def video_feed():
    return Response(
            frame_generator(),
            mimetype='multipart/x-mixed-replace; boundary=frame'
        )

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True, port=5000)

Mrunalika96 · 2022-04-27T15:37:45Z

we are getting an error as exception thrown 1

ptolemyspce · 2023-08-05T13:46:39Z

Hi there.
I have run that flask app, but it just work in browser, NOT in VLC and QGCS.
checked it via wireshark and it was over TCP protocol.
how can i stream it over RTSP?

ptolemyspce · 2023-08-16T11:38:33Z

finally I can find my way !
u should get the frame from airsim in a while loop and save it(and rewrite it continuously-i know this is not standard but it works):

`import airsim #pip install airsim
import cv2
import numpy as np
import time

filename = 'file.jpg'

client = airsim.MultirotorClient()

while True:
t1=time.time()
png_image = client.simGetImage("0", airsim.ImageType.Scene)
# do something with image
pic = airsim.string_to_uint8_array(png_image)
im = cv2.imdecode(np.array(pic, dtype=np.uint8), cv2.IMREAD_UNCHANGED)

cv2.imwrite(filename, im)
_,img_encode = cv2.imencode('.jpg',im)

print(time.time()-t1)`

then u should read that saved file with Gstreamer and steam it through UDP:
gst-launch-1.0 -v multifilesrc location=file.jpg loop=true ! jpegdec ! videoconvert ! videoscale ! videorate ! video/x-raw,framerate=30/1 ! x264enc ! rtph264pay ! udpsink host=192.168.1.100 port=6000

finally set QGCS in udp port and enjoy live streameing

the question is:
Is there a better way instead of write and read in a file?

cloudlakecho · 2023-10-26T21:32:23Z

@ptolemyspce Regarding skipping saving image, have you applied the method from StackOverflow?

It looks like this (I haven't tried yet):

#Get current IP for RTSP
s = socket.socket(socket.AF_INET,  socket.SOCK_DGRAM)
s.connect(("8.8.8.8",  80))
ip_address = s.getsockname()[0]

pipe_out ='appsrc is-live=True ! queue ! videoconvert ! video/x-raw, format=RGBA ! nvvidconv\
      ! omxh264enc insert-sps-pps=true ! video/x-h264, stream-format=byte-stream! h264parse \
      ! rtph264pay name=pay0 pt=96 config-interval=1 ! udpsink port=8554 host=192.168.1.158'

rtsp_out = cv2.VideoWriter(pipe_out,  fourcc=0,  apiPreference=cv2.CAP_GSTREAMER, fps=fps, 
                           frameSize=(1980, 1080),  isColor=True) 

# For USB camera case, AirSim should be different
ok, frame = cam.read()

frame_out =  cv2.resize(frame,  (1980, 1080))

# result streaming to rtsp 
rtsp_out.write(frame_out)

rtsp_out.release()

sytelus added the px4 label Apr 12, 2018

sytelus closed this as completed Apr 12, 2018

p3jawors mentioned this issue Aug 25, 2020

How to create video stream like in airsim demo #2969

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a Video-Stream #892

Create a Video-Stream #892

mrRedDead98 commented Mar 20, 2018

sytelus commented Mar 23, 2018

sytelus commented Apr 12, 2018

VedantMistry13 commented Aug 24, 2018 •

edited

Mrunalika96 commented Apr 27, 2022

ptolemyspce commented Aug 5, 2023

ptolemyspce commented Aug 16, 2023 •

edited

cloudlakecho commented Oct 26, 2023

Create a Video-Stream #892

Create a Video-Stream #892

Comments

mrRedDead98 commented Mar 20, 2018

sytelus commented Mar 23, 2018

sytelus commented Apr 12, 2018

VedantMistry13 commented Aug 24, 2018 • edited

Mrunalika96 commented Apr 27, 2022

ptolemyspce commented Aug 5, 2023

ptolemyspce commented Aug 16, 2023 • edited

cloudlakecho commented Oct 26, 2023

VedantMistry13 commented Aug 24, 2018 •

edited

ptolemyspce commented Aug 16, 2023 •

edited