About stopping a running ffmpeg process

[RafaJurado]

Hi again.

I'm been struggling with java/Spring and Jaffree in order to understand how to implement a way to stop a ffmpeg running process at any moment. I've carefully reviewed the StopExample.java that's available in the repo, but I've failed to adapt it to my context. I've also searched extensively through the Spring documentation for any related example. The problem is that I was not sure exactly what I was looking for. Maybe I'll have more luck here.

Here's my last try:

//CONTROLLER
//returns ResponseEntity<StreamingResponseBody>
AtomicReference<FFmpegResultFuture> ffmpegResult = null;
return ResponseEntity.ok()
		.contentType(MediaType.parseMediaType("video/mp4"))
		.body(os -> {
			ffmpegResult.set(streamObject.getFfmpegStream(os, rtspUri));
		});

//SERVICE WITH JAFFREE CALL
public FFmpegResultFuture getFfmpegStream(OutputStream os, String uri) throws IOException {
		log.info("getFfmpegStream: "+uri);

		return FFmpeg.atPath()
				//.setLogLevel(LogLevel.TRACE)
				.addInput(UrlInput.fromUrl(uri)
						//.setReadAtFrameRate(true) //esto ralentiza a muerte el stream, no usar!
						.addArguments("-c:v","h264")
						.setFrameRate(10)
						.addArguments("-rtsp_transport", "tcp")
						.addArguments("-err_detect","ignore_err")
						.addArgument("-an")
						//.addArguments("-bufsize","4096k")
						)
				.addOutput(
						PipeOutput.pumpTo(os)
								.addArguments("-c:v", "h264")
								//.addArguments("-pix_fmt","yuv420p")
								.addArguments("-preset","ultrafast")
								.setFrameRate(10)
								.addArguments("-g","20")
								.addArguments("-b:v", "2048k")
								.addArguments("-maxrate","2048k")
								.addArguments("-minrate","512k")
								.addArguments("-bufsize","2M")
								.addArguments("-movflags","+faststart")
								.addArguments("-fflags","+igndts")
								//.addArguments("-acodec", "aac")
								//.addArgument("-an")
								//.addArguments("-r", "12")
								.disableStream(StreamType.AUDIO)
								.setFormat("ismv")
								//.setFrameRate(10)

				)
				//.setOverwriteOutput(true)
				.executeAsync();
	}

What's the objective of this?. I have an RTSP uri for streaming a recorded file on a remote "server". In fact it's not a server but a Network Video Recorder, installed in a remote place (right now it's like 1m from me, but the device is treated as a remote one) and acting like a file server. The uri is used as an input for the Ffmpeg process that modifies the stream in order to be playable in any player (the files directly from the NVR won't play on VLC or any other player, at least under Linux) and serves it to the frontend as a response to an ajax request.

The code I've shared it's not working as is (no image on the browser player). But with some minor changes it works:

//returns ResponseEntity<StreamingResponseBody>
return ResponseEntity.ok()
		.contentType(MediaType.parseMediaType("video/mp4"))
		.body(os -> streamObject.getFfmpegStream(os, rtspUri));


//Ffmpeg CALL
//returns FFmpegResult getFfmpegStream
...
).execute();

The problem is that this way I can't access any Ffmpeg* object and, as you may guess, I can't stop the process. Any advice?. I'm not asking for code but for a clue or concept to know what I have to look for in java/Spring docs.

[kokorin]

Never use OutputStream with Controllers. It's never a good idea. You don't know how and when outputstream will be read. There is no guarantee that content will actually be streamed (and not fully buffered first) by Spring/Tomcat/Netty/etc.

Regarding you question: at what moment you want to stop Jaffree? 10 seconds? 1 minute? Or may be when a user closes a browser? With HTTP (at least HTTP 1.1) the only way to stop downloading is closing a connection.

What is your final goal? If you want allow a user to download "fixed" file - the easiest way is to read RTSP and store fixed file on local disk. Then to serve the file from local disk.

If you want to provide live streaming functionality - check HLS-related examples/issues/discussions.

[RafaJurado]

Thanks for answering so fast! About the controllers and OutputStream yes, I know, but thanks for the advice. All the code I shared it's proof of concept, not for production. I'm just seeking the way it can work.

The application is designed for serving live stream from remote cameras, recorded videos and another functions (get video lists by day, device status, configurations...), although the signal is not received directly but through a middleware were all the remote devices are registered. This middleware serves streams, uris or json, depending on what you request is. User will see a page with place for 4 cameras for live stream, a list of recordings and a modal to watch/download recordings. The NVR that controls the cameras will communicate with a 4G modem/router, so bandwidth is important. I need a way to interrupt a stream in any given moment, for example with an stop button or when the modal closes. This way I can save precious bandwidth as the NVR/cameras share it with other devices.

The download works great right now except for the audio (g722.1) that I'm not been able to recode yet, but this is another minor problem. For the stream part I'm using the code I shared and also your HLS example, testing both ways of doing a working stream. For the pure live stream I'm using the code above and works, for the file stream I was using HLS but couldn't get it working with video.js.

[kokorin]

What do you mean by "pure live stream"? Live stream with minimal delay? I think you can reach 2-5 seconds delay with HLS depending on keyframe interval, segment duration and so on.

I think from browser you should close connection (used to watch the stream) "when the modal closes". So OutputStream will produce an exception next time Jaffree tries to write bytes to it.

With HLS you can implement some smart solution:

1. When user requests M3U8 first time start ffmpeg producing HLS segments and M3U8, then serve M3U8 to client (when it becomes available). After that register Jaffree process somewhere and set timeout to it (say x2 M3U8 duration)
2. Every time a client requests M3U8 (or segment) - prolong Jaffree timeout
3. When timeout is reached - stop Jaffree

With that approach you will not stop consuming NVR stream immediately, but when it's not in use.

[RafaJurado]

Hi!. With "pure live stream" I meant just the live stream, to differentiate it from the stream that the middleware creates for recorded videos.

I think from browser you should close connection (used to watch the stream) "when the modal closes".

Unfortunately it's not working like that. When the modal closes, the Jaffree process continues and I think its related to the response type being StreamingResponseBody, that works asynchronously. As a side note, I've read again my notes and the OutputStream is part of the StreamingResponseBody way of doing things, so it seems it won't give me problems.

My problem now is how to get a FFmpegResultFuture for calling one of the stop methods. I've tried this in my Controller method:

StreamingResponseBody stream = out -> {
			FFmpegResultFuture future = streamObject.getFfmpegStream(out, rtspUri);
			ffmpegMap.put("stream",future);
		};

log.info("getFileStreaming!");
return ResponseEntity.ok()
		.contentType(MediaType.parseMediaType("video/mp4"))
		.body(stream);

That method is called via src atttribute on video.js like this:

videoJS.src({type:'video/mp4', src: url});

With that structure, the service method that contains Jaffree code must return FFmpegResultFuture, and that means to change execute() with executeAsync(). I don't know why, but this way the Controller method is called every second, launching a bunch of identical ffmpeg processes. I cant find any info at the log with a "close" statement, so all ffmpeg processes are executing at the same time (it makes sense, as I specified executeAsync).

I'll try your solution as soon as possible, hope this time video.js finds the m3u8 files. Thanks!

[RafaJurado]

Hi again. As you suggested I tried with HLS and it worked, but with some changes. The process is somewhat similar to the one I use to download a file: A controller method launches the ffmpeg process by ajax request. That ajax request starts an interval to another js function that asks for a m83u file. When the file is created, another controller method serves the files (m3u8 and ts). The FFmpegResultFuture is stored in a global class map variable and called when the user clicks the stop button. Thank you very much for your help! (Is there any way to contribute to the project?). The code is like this:

let streamIntervalId;
/* play button launches process */
    $("#modalPlayButton").click(function(e) {
        e.preventDefault();
        var dayDeviceIdAndChannel = $("#modalData").val();
        var videoName = dayDeviceIdAndChannel +".mp4";
        var url = "/index/record/r/"+videoName;
        console.log("URL: "+url);
        $.ajax({
            type: 'GET',
            url: url,
            dataType: 'text',
            success: function(response) {
                var resp = response;
                console.log("RESPONSE: "+resp);
                streamIntervalId = setInterval(getLiveStreamFile,5000);

            },
            error: function(jqXHR, textStatus,errorThrown) {
                alert('Error: ' + jqXHR.status + ' - ' +textStatus +' - ' +errorThrown );
            }
        });
    });

/* controller method called by ajax req */
@CrossOrigin
	@RequestMapping(value = "/index/record/r/{videoName}", method = RequestMethod.GET,
			produces = MediaType.TEXT_PLAIN_VALUE)
	@ResponseBody
	public ResponseEntity<String> getFileStream(@PathVariable String videoName) throws IOException {

		String[] rtspUriStringArray = cameraService.getLiveCameraUri(videoName);
		String rtspUri = rtspUriStringArray[0] + "//foo:bar!@" +rtspUriStringArray[1];

		FFmpegResultFuture future = streamObject.getFfmpegFileStream(rtspUri);
                /* ffmpegMap global map */
		ffmpegMap.put("stream",future);

		return ResponseEntity.ok()
				.contentType(MediaType.parseMediaType("text/plain"))
				.body("0");
	}

/* javascript function in interval */
function getLiveStreamFile() {
        $.ajax({
           url: '/index/record/f/stream.m3u8',
           type: 'HEAD',
           success: function () { //checks if file exists
              // The file exist!
              clearInterval(streamIntervalId);
              streamIntervalId = null;
              /* video.js element creation */
              var videoJS = videojs('modalPlayer' , {
                                    muted:true,
                                    autoplay:true,
                                    fluid: true,
                                    controls: true,
                                    liveui: true,
                                    html5:{
                                      hls:{ //important!
                                        enableLowInitialPlaylist:true
                                      }
                                    }
                                  },function(e){
                              });
              $("#stopstream").removeClass("disabled");
              videoJS.src({type:'application/x-mpegURL', src: '/index/record/f/stream.m3u8'});
           },
           error: function () {
              console.log("waiting");
           },
        });

/* last phase, controller method serves requested files */
@CrossOrigin
	@RequestMapping(value = "/index/record/f/{hlsfilename}", method = RequestMethod.GET)
	@ResponseBody
	public ResponseEntity<Resource> getStreamFilePath(@PathVariable String hlsfilename) {
		StorageProperties sp = new StorageProperties();
		String filename = hlsfilename;
		log.info("getStreamFilePath.filename: " + filename);
		Path rootPath = Paths.get(sp.getLocation());
		Path pathToDestiny = rootPath.resolve(filename);
		try {
			Resource resource = new UrlResource(pathToDestiny.toUri());
			if (resource.exists() || resource.isReadable()) {
				log.info("getStreamFilePath.DESTINO: " + pathToDestiny.toString());
				HttpHeaders headers = new HttpHeaders();
				headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + filename + "\"");
				headers.add("Cache-Control", "no-cache, no-store, must-revalidate");
				headers.add("Pragma", "no-cache");
				headers.add("Expires", "0");
				return ResponseEntity.ok()
						.contentType(MediaType.parseMediaType("application/x-mpegURL"))
						.headers(headers)
						.body(resource);
			}
			resource = null;
			return ResponseEntity.ok()
					.contentType(MediaType.APPLICATION_OCTET_STREAM)
					.body(resource);

		} catch (IOException e) {
            throw new RuntimeException(e);
        }
}

/* the ffmpeg method*/
public FFmpegResultFuture getFfmpegFileStream(String uri) throws IOException {

		StorageProperties sp = new StorageProperties();
		Path rootPath = Paths.get(sp.getLocation());

		return FFmpeg.atPath()
				.addInput(
						UrlInput.fromUrl(uri)
								.addArguments("-c:v","h264")
								.setFrameRate(10)
								.addArgument("-an")
								.addArguments("-rtsp_transport", "tcp")
								.addArguments("-err_detect","ignore_err")
				)
				.addOutput(
						UrlOutput.toPath(rootPath.resolve(Paths.get("stream.m3u8")))
								.setFrameRate(10)
								.setFormat("hls")
								.addArguments("-c:v","libx264")
								.addArguments("-bufsize","2M")
								.addArguments("-preset","veryfast")
								.addArguments("-hls_list_size", "5")
								.addArguments("-hls_delete_threshold", "5")
								.addArguments("-hls_time", "15")
								.addArguments("-hls_flags", "delete_segments")

				)
				.setOverwriteOutput(true)
				.executeAsync();

	}