Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8277309: Add support for H.265/HEVC to HTTP Live Streaming #726

Closed
wants to merge 3 commits into from

Conversation

sashamatveev
Copy link
Member

@sashamatveev sashamatveev commented Feb 4, 2022

  • Added support for fragmented MP4 with HEVC/H.265, H.264 and AAC.
  • Added support for elementary AAC streams without any container for audio only streams.
  • Added "aacparse" plugin from GStreamer. Required on Linux, since decoder cannot handle AAC elementary streams directly. DirectShow decoder works without it.
  • DirectShow H.264 decoder on Windows and H.265/H.264 decoder on Linux will be reloaded when fMP4 stream changes resolution. Dynamic format change did not worked for these streams on Windows and Linux.

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed
  • Change requires a CSR request to be approved

Issues

  • JDK-8277309: Add support for H.265/HEVC to HTTP Live Streaming
  • JDK-8281288: Add support for H.265/HEVC to HTTP Live Streaming (CSR)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jfx pull/726/head:pull/726
$ git checkout pull/726

Update a local copy of the PR:
$ git checkout pull/726
$ git pull https://git.openjdk.java.net/jfx pull/726/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 726

View PR using the GUI difftool:
$ git pr show -t 726

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jfx/pull/726.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Feb 4, 2022

👋 Welcome back almatvee! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Ready for review label Feb 4, 2022
@mlbridge
Copy link

mlbridge bot commented Feb 4, 2022

Webrevs

@kevinrushforth
Copy link
Member

/reviewers 2

@openjdk
Copy link

openjdk bot commented Feb 4, 2022

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

@yososs
Copy link

yososs commented Feb 4, 2022

Why not use BitSet instead of ArrayList<Boolean>?


Because I forgot to add Markdown escaping, the
ArrayList <Boolean> was not being displayed.

@kevinrushforth
Copy link
Member

Why not use BitSet instead of ArrayList?

Can you be more specific? The only use of ArrayList that I see in the patch is in code that uses a few already-existing lists. Changing that to some other data structure would be an unrelated change that seems out of scope for this fix.

@yososs
Copy link

yososs commented Feb 4, 2022

modules/javafx.media/src/main/java/com/sun/media/jfxmedia/locator/HLSConnectionHolder.java

I noticed that in this change, ArrayList<Boolean> is rewritten as ArrayList<>.
The BitSet rewrite needs some thought, so it might be a good idea to make it a separate issue.
In the file I pointed out, it is used in many more places than the one I pointed out.

@openjdk openjdk bot added the csr Need approved CSR to integrate pull request label Feb 4, 2022
Copy link
Member

@kevinrushforth kevinrushforth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my testing looks good. I spot checked the new code, and it looks good as well. I left a couple questions for you, but I'll approved it as is.

@openjdk openjdk bot removed the csr Need approved CSR to integrate pull request label Mar 10, 2022
@sashamatveev
Copy link
Member Author

Fixed several issues found during additional testing:

  • sendHeader was true by default which caused sending first and second segment together for MP2TS and MP3/AAC elementary streams. This variable should be true only for fragmented MP4.
  • Removed G_SIGNAL_NO_RECURSE flag from javasource signals. From GLib doc "G_SIGNAL_NO_RECURSE - Signals being emitted for an object while currently being in emission for this very object will not be emitted recursively, but instead cause the first emission to be restarted." Basically it caused skipping data segments since in some cases loadNextSegment() was called twice. We do use signal mechanism from separate thread when reading duration. This issue is intermittent. All GStreamer provided elements do not use G_SIGNAL_NO_RECURSE flag, so not sure why javasource was using it.
  • Fixed duration calculation. We should not count duration of header, since header does not contain any data.
  • Added code which will adjust PTS during rendering to start with 0, otherwise (after duration fix) we will drop frames with PTS > duration. Note: In fMP4 HLS streams PTS starts with N and goes upto duration + N. Our rendering code expects start with 0 and stop at duration.
  • Minor code cleanup and removal of unnecessary "if (segment.start != 0)" check in hlsprogressbuffer.

@johanvos
Copy link
Collaborator

This looks really great. I'm doing a few more tests before approving, but I don't expect issues.
I wonder if we can add some simple resources that allow testing the different protocols on different platforms?

@openjdk
Copy link

openjdk bot commented Mar 23, 2022

@sashamatveev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8277309: Add support for H.265/HEVC to HTTP Live Streaming

Reviewed-by: kcr, arapte, jvos

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 27 new commits pushed to the master branch:

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Ready to be integrated label Mar 23, 2022
@kevinrushforth
Copy link
Member

I wonder if we can add some simple resources that allow testing the different protocols on different platforms?

I can't think of a good way to do that, especially for this feature, since it requires a web server.

@sashamatveev
Copy link
Member Author

/integrate

@openjdk
Copy link

openjdk bot commented Mar 23, 2022

Going to push as commit 424aeba.
Since your change was applied there have been 27 commits pushed to the master branch:

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Mar 23, 2022
@openjdk openjdk bot closed this Mar 23, 2022
@openjdk openjdk bot removed ready Ready to be integrated rfr Ready for review labels Mar 23, 2022
@openjdk
Copy link

openjdk bot commented Mar 23, 2022

@sashamatveev Pushed as commit 424aeba.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@mlbridge
Copy link

mlbridge bot commented Mar 24, 2022

Mailing list message from Scott Palmer on openjfx-dev:

On Mar 23, 2022, at 8:20 AM, Kevin Rushforth <kcr at openjdk.java.net> wrote:

On Tue, 22 Mar 2022 08:54:45 GMT, Johan Vos <jvos at openjdk.org> wrote:

I wonder if we can add some simple resources that allow testing the different protocols on different platforms?

I can't think of a good way to do that, especially for this feature, since it requires a web server.

Just wondering why that?s an issue, specifically for HLS. Java?s built-in web server, the old one at com.sun.net.httpserver.HttpServer (i.e. it wouldn?t need JDK 18), would work fine.

E.g. something not much more complicated than:

int port = 8888;
Path testFilesRoot = Paths.get("media");
HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);
server.createContext("/hls", new HttpHandler() {
@Override
public void handle(HttpExchange exchange) throws IOException {
Path path = Paths.get(exchange.getRequestURI().getPath()).getFileName();
boolean isPlaylist = path.endsWith(".m3u8");
boolean isTS = path.endsWith(".ts");
Path assetPath = testFilesRoot.resolve(path);
byte[] data = Files.readAllBytes(assetPath);
exchange.getResponseHeaders().add("Content-Type", isPlaylist
? "application/x-mpegURL"
: isTS ? "video/MP2T" : "video/mp4");
exchange.sendResponseHeaders(200, data.length);
OutputStream os = exchange.getResponseBody();
os.write(data);
os.close();
}
});
server.start();

The content would all be short static test files, easily fitting in the byte array. You could grab something from a public domain source (just edit the paths in the .m3u8 files as needed)

Cheers,

Scott

@kevinrushforth
Copy link
Member

Interesting idea. Btw, we don't have any media tests at all in our repo at present, so there is a more basic issue that would need to be solved. We could file an RFE, but there are some challenges in doing anything that is useful enough to justify the effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated
5 participants