Draft of file manipulation article. #4682

jpmedley · 2017-06-12T16:24:44Z

I don't know why this PR is showing changes to the cheatsheet. Those have already been merged to master.

kqyang · 2017-06-15T23:17:59Z

src/content/en/fundamentals/media/manipulating/cheatsheet.md

@@ -191,11 +194,6 @@ remove all whitespace including the final carriage return.
 For the `-key` flag use the key created earlier and stored in the media.key
 file. However, when entering it on the command line, be sure you've removed its
 whitespace. For the `-key_id` flag repeat the key value.


key_id is really an identifier that uniquely identifies the key. It can usually be found in the manifests and media containers in plain text/hex.

Some applications concatenate key_id and key and store them together, e.g. Apple's segmenter tool. This gives an impression that key_id is part of the key, but it is actually not.

The size of key needs to be 16 bytes because we are using AES-128 (128 bits, 16 bytes) for CENC and SAMPLE-AES.

The requirement of key-id being 16 bytes comes from CENC spec (23001-7) which requires key-id (KID) to be 16 bytes.

"It can usually be found in the manifests and media containers in plain text/hex." What manifests and media containers?

For example, DASH MPD(cenc:default_KID, see https://github.com/google/shaka-packager/blob/master/packager/app/test/testdata/bear-640x360-av-cenc-golden.mpd), fmp4 (in tenc box).

That's what I thought you meant. But I haven't created the MPD yet. Is there some place else I can get it?

You can find it in CENC spec (23001-7:2016).

The full text of the spec in $118. Can you please just give me the answer?

Not sure what answer you need. I think you can just say:

key_id is an identifier that uniquely identifies the key.

If you want, you can mention that:

It is available in packager outputs in plaintext. It will be used by the player / content decryption module to select the right key to decrypt the content.

Let me know if you need anything else.

What if I'm encrypting in but not doing DASH? Is there some other way to get the key_id?

As I mentioned earlier, it is also available in tenc box for fMP4.

kqyang · 2017-06-15T23:20:18Z

src/content/en/fundamentals/media/manipulating/cheatsheet.md

-> **Note:** Technically, the key ID is supposed to be either the first 8 OR the
-> first 16 hex digits of the key. Since packager requires the key to be 16
-> digits and does not allow a 32 digit key, both flags use the same value.
-

    packager \
      input=myvideo.mp4,stream=audio,output=glocka.m4a \


--enable_fixed_key_decryption is not needed below.

'-key_id INSERT_KEY_HERE' should be '-key_id INSERT_KEY_ID_HERE'

kqyang · 2017-06-15T23:27:35Z

src/content/en/fundamentals/media/manipulating/cheatsheet.md

      --enable_widevine_encryption \
      --key_server_url "https://license.uat.widevine.com/cenc/getcontentkey/widevine_test" \
-      --content_id "fd385d9f9a14bb09" --signer "widevine_test" \
+      --content_id "16_Rand_Hex_Chrs" --signer "widevine_test" \


'content_id' is an unique identifier identifies the content. There is no restriction on the size of content id. In fact, you can use an arbitrary size string as content_id. It does need to be hex converted before supplying it to packager.

It does not need to be randomly generated either.

kqyang · 2017-06-15T23:29:51Z

src/content/en/fundamentals/media/manipulating/files.md

+Before I start manipulating media files, I want to talk a bit about how media
+files are put together. I think of them as being like an onion. The file that
+you see in your operating system shell is a _container_, identified by a file
+extension (mp4, webm, etc.). The container houses from one to four _streams_,


Some containers allows multiple streams of the same type, e.g. webm

kqyang · 2017-06-15T23:31:43Z

src/content/en/fundamentals/media/manipulating/files.md

+    ffmpeg -i myvideo.webm -vcodec copy -an myvideo_video.webm
+    ffmpeg -i myvideo.webm -acodec copy -vn myvideo_audio.webm
+
+Just as with Shaka Packer, we have both an input and an output file. Another


s/Packer/Packager/

kqyang · 2017-06-15T23:36:18Z

src/content/en/fundamentals/media/manipulating/files.md

+the top three. If you're serving video someplace like India, you'll want to
+include 2G as well. For demonstration purposes, I'm going to target 3G.
+
+In Shaka Packager you set the bitrate by adding the bandwidth option to the


The bandwidth option in Shaka Packager does not modify the bit rate of the content. It is used to finetune the bandwidth value appears in the manifests (DASH MPDs or HLS playlists). If not provided, packager will estimate from the content.

That's not what packager --help says:

"bandwidth (bw): Optional value which contains a user-specified
content bit rate for the stream, in bits/sec."

Sorry for the misleading description. I will fix the description in Shaka packager to make it clearer.

Then how do I set the bit rate using Packager?

Packager does not do transcoding, so there is no way to set/modify the bit rate. Normally the user prepares multi-bitrate streams, e.g. using FFmpeg or another encoder, then feed the multi-bitrate streams to packager.

jpmedley · 2017-06-19T21:35:13Z

@kqyang Thanks for the feedback. I have questions about 2 of the comments. I've made changes for the others.

johnpallett · 2017-06-19T22:18:57Z

src/content/en/fundamentals/media/manipulating/files.md

+  Audio and video streams are split into separate files
+  Versions of the video file are in mp4 and webm format
+  Versions of the audio file are in m4a and webm format
+  A bitrate of 0.35 Megabits per seconde (Mbs)


seconde -> second

johnpallett · 2017-06-19T22:19:10Z

src/content/en/fundamentals/media/manipulating/files.md

+files are put together. I think of them as being like an onion. The file that
+you see in your operating system shell is a _container_, identified by a file
+extension (mp4, webm, etc.). The container houses one or more _streams_ of up to
+four types, specifically audio, video, captions, and data. Although containers


There are definitely more than four types of streams, and some media files can have 30, 40 or more streams. Sorry I missed this in 'Application Primers' earlier.

You appear to be using 'streams' and 'types of streams' interchangeably. I'm not. Can you please clarify?

johnpallett · 2017-06-19T22:19:46Z

src/content/en/fundamentals/media/manipulating/files.md

+Let's start by changing the file container. You'll recall that we're starting
+with a file that has an mov extension. I'm going to use ffmpeg to change the
+container type from mov to mp4 and webm. In actual practice, you would likely
+specify a codec at the same time. For this lesson, I'm letting ffmpeg use it's


it's -> its

johnpallett · 2017-06-19T22:20:33Z

src/content/en/fundamentals/media/manipulating/files.md

+When you package media resources, for HLS and DASH, you'll need to separate the
+video and audio streams. Splitting the audio and video streams of a file is
+often referred to as **demultiplexing** or **demuxing** for short. Professional
+content creators record audio and video to separate files to begin with. If


creators often record audio and video to separate files

Plenty of high-end devices create single files with both audio and video

johnpallett · 2017-06-19T22:21:03Z

src/content/en/fundamentals/media/manipulating/files.md

+
+## File properties
+
+I've unpealed the onion. Yet if you compare what I've done to the list of stated


unpealed -> unpeeled

johnpallett · 2017-06-19T22:21:36Z

src/content/en/fundamentals/media/manipulating/files.md

+goals you'll see that I'm not quite done. Anong the remaining items are bitrate
+and resolution. These properties correlate to the amount of data in a media
+file. It probably goes without saying, but I'm going to say it anyway, that you
+can always lower bitrate and resolution, but you can't increase them. (Well,


You can increase both (e.g. SD to HD) and there are algorithms to help with quality. Remove that bit please.

I've reworded it a bit. See what you think.

johnpallett · 2017-06-19T22:22:58Z

src/content/en/fundamentals/media/manipulating/files.md

+recommends](https://support.google.com/youtube/answer/6375112) the following
+resolutions for video uploads, all in the 16:9 aspect ratio. There's nothing
+specific to Youtube about this list, which makes it a good guide for video on
+the web in general.


These are all pretty standard 16:9 resolutions for video; I'd take out "which makes it a good guide..." and replace with "it's just a list of common 16:9 video resolutions."

johnpallett · 2017-06-19T22:23:55Z

src/content/en/fundamentals/media/manipulating/files.md

+| 360p | 640x360 |
+| 240p | 426x240 |
+
+Which one do I use? Again, the short answer is all of them. Fortunately, this is


I'd just make this functional: "You might use one or all of these depending upon your application. Fortunately, this is one of the simplest..."

johnpallett · 2017-06-19T22:24:43Z

src/content/en/fundamentals/media/manipulating/files.md

+Unless your company completes the Master License Agreement with
+[Widevine]([http://www.widevine.com/contact.html](http://www.widevine.com/contact.html))
+, this type of encryption can really only be used for testing. Since this is
+covered without much explanation on the Shaka Packager READMEhere it goes.


READMEhere -> README, here

johnpallett · 2017-06-19T22:25:35Z

src/content/en/fundamentals/media/manipulating/files.md

+
+This does not cover everything you could do to a media file before posting it to
+the web, not by a longshot. To be fair, this subject is one deserving of a
+website that is itself as large as [developers.google.com/web/](/web/). I'm


I'd take out this section; we're trying to create this sub-site for posting to the web here. Could reference developers.google.com/media/vp9 for VP9/WebM encoding if you'd like.

kqyang · 2017-06-19T23:42:15Z

src/content/en/fundamentals/media/manipulating/files.md

+the top three. If you're serving video someplace like India, you'll want to
+include 2G as well. For demonstration purposes, I'm going to target 3G.
+
+In Shaka Packager you set the bitrate by adding the bandwidth option to the


Sorry for the misleading description. I will fix the description in Shaka packager to make it clearer.

kqyang · 2017-06-19T23:44:19Z

src/content/en/fundamentals/media/manipulating/cheatsheet.md

@@ -233,7 +233,7 @@ This command will accept a key with either 16 or 32 characters.
      input=glocken.mp4,stream=audio,output=enc_audio.m4a \
      --enable_widevine_encryption \
      --key_server_url "https://license.uat.widevine.com/cenc/getcontentkey/widevine_test" \
-      --content_id "16_Rand_Hex_Chrs" --signer "widevine_test" \
+      --content_id "Hec_converted_random_ID" --signer "widevine_test" \


s/Hec/Hex/

I'd also replace random with unique.

jpmedley · 2017-06-22T14:38:52Z

@beaufortfrancois I didn't realize you were taking over from Sam. Can you please review this in the next few days?

cwilso · 2017-06-22T15:24:04Z

src/content/en/fundamentals/media/manipulating/files.md

+| Desktop web video | 2 Mbs |
+| 4G mobile video | 0.7 Mbs |
+| 3G mobile video | 0.35 Mbs |
+| 2G mobile video | ?? |


cwilso · 2017-06-22T15:25:06Z

src/content/en/fundamentals/media/manipulating/files.md

+{# wf_updated_on: 2017-06-23 #}
+{# wf_published_on: 2017-06-23 #}
+
+# Media File Manipulation {: .page-title }


I think this probably isn't "Media File Manipulation" - it's "Transforming a raw video file into an encrypted multi-bandwidth resource" or the like?

cwilso · 2017-06-22T15:26:01Z

src/content/en/fundamentals/media/manipulating/files.md

+To be more specific about the goals, the result of the procedures described here
+will be media resources with the following characteristics:
+
+  Audio and video streams are split into separate files


Why are you splitting into separate files? Is this/why is this a good idea?

See line 106. I said it's done that way for HLS and DASH. I guess I should have asked if HLS and DASH required this. (I thought it did, but maybe it doesn't.) Even if it doesn't, it would seem to me to be more efficient if the dynamic part of DASH only had to deal with one type of stream at a time.

Maybe @johnpallett or @beaufortfrancois could comment on this.

It is a requirement in DASH.

I would add that comment here, since it's 75 more lines before you get there - just a "(this is required by DASH)".

It probably just seems that way because this is where you thought of the question. This intro list contains a lot that's justified later, specifically the bitrate and resolution. If I start explaining all this stuff here, the introduction will bloat and start to lose its effectiveness.

cwilso · 2017-06-22T15:27:40Z

src/content/en/fundamentals/media/manipulating/files.md

+
+Notice that the stream descriptor has an input, an output, and a stream type.
+You might remember from [Application Primers](applications) that Shaka Packager
+can take multiple stream descriptors. This means that Shaka Packager lets me


But the final product is a single

samdutton · 2017-06-22T15:37:26Z

src/content/en/fundamentals/media/manipulating/files.md

+may chose one, several, or all. Fortunately, this is one of the simplest
+transformations you'll make with ffmpeg.
+
+    ffmpeg -i myvideo.webm -s 1920x1080 myvideo_1980x1020.webm


Should this be myvideo_1920x1080.webm?

Fixed both.

I'm still seeing 1980x1020.webm instead of 1920x1080.webm.

samdutton · 2017-06-22T15:44:50Z

src/content/en/fundamentals/media/manipulating/cheatsheet.md

@@ -2,7 +2,7 @@ project_path: /web/_project.yaml
 book_path: /web/fundamentals/_book.yaml
 description: A summary of commands used to convert a raw mov file to an encrypted full HD file for web playback.

-{# wf_updated_on: 2017-06-12 #}
+{# wf_updated_on: 2017-06-23 #}
 {# wf_published_on: 2017-06-09 #}


I think '(1920 x 1980)should be(1920 x 1080)`.

into manipulation

jpmedley · 2017-06-26T15:45:45Z

@cwilso Can you please approve the changes today or tomorrow?

beaufortfrancois

Thank you @jpmedley for writing this. I've learnt a lot ;)
See my nits.

beaufortfrancois · 2017-06-27T12:36:04Z

src/content/en/fundamentals/media/manipulating/files.md

@@ -0,0 +1,331 @@
+project_path: /web/_project.yaml
+book_path: /web/fundamentals/_book.yaml
+description: TBD.


Please add description

beaufortfrancois · 2017-06-27T12:38:30Z

src/content/en/fundamentals/media/manipulating/files.md

+
+Within the audio and video streams, the actual data is compressed using a
+_codec_. As we'll see later, the distinction between a container and a codec is
+import as files with the same container can have their contents encoded with


s/import/important

beaufortfrancois · 2017-06-27T12:51:30Z

src/content/en/fundamentals/media/manipulating/files.md

+    ffmpeg -i glocken.mov -vf setsar=1:1 glocken.webm
+
+Webm takes quite a bit longer to create than mp4. This isn't surprising when
+you look at the results. While mp4 compresses to about a quarter of the original


You may want to mention which version of ffmpeg you used.

beaufortfrancois · 2017-06-27T12:53:21Z

src/content/en/fundamentals/media/manipulating/files.md

+you look at the results. While mp4 compresses to about a quarter of the original
+file's size, webm compresses to 1% of the original file's size.
+
+    -rw-r--r--  1 jmedley  eng  12080306 May  9 14:34 glocken.mov


Here's the kind of output I'd use:

ls glocken.* -hlGrt -rw-r--r--. 1 fr 12M Jun 27 11:48 glocken.mov -rw-rw-r--. 1 fr 9.7M Jun 27 14:47 glocken.mp4 -rw-rw-r--. 1 fr 480K Jun 27 14:50 glocken.webm

This makes it easier to read.

beaufortfrancois · 2017-06-27T12:54:21Z

src/content/en/fundamentals/media/manipulating/files.md

+
+Webm takes quite a bit longer to create than mp4. This isn't surprising when
+you look at the results. While mp4 compresses to about a quarter of the original
+file's size, webm compresses to 1% of the original file's size.


I get 4% for webm, and 83% for mp4.

beaufortfrancois · 2017-06-27T13:13:51Z

src/content/en/fundamentals/media/manipulating/files.md

+## File properties
+
+I've unpeeled the onion. Yet if you compare what I've done to the list of stated
+goals you'll see that I'm not quite done. Anong the remaining items are bitrate


s/Anong/Among

beaufortfrancois · 2017-06-27T13:15:55Z

src/content/en/fundamentals/media/manipulating/files.md

+
+    ffmpeg -i glocken.mov -b:v 350K -b:a 350K glocken.mp4
+
+Notice that there are two bitrate flags, -b:a and -b:v. One is for audio and the


Please add backquotes for -b:a and -b:v

beaufortfrancois · 2017-06-27T13:38:48Z

src/content/en/fundamentals/media/manipulating/files.md

+This example implies that I've typed or pasted the key into the command line
+manually. There's no reason you can't get fancy.
+
+    packager \


This doesn't work for me as the media.key file contains characters that can't be displayed in the terminal. How did you get it to work?

Let me dig into this some more.

It should be hex encoded.

It's not central to the article, so I'm taking it out.

beaufortfrancois · 2017-06-27T13:44:35Z

src/content/en/fundamentals/media/manipulating/files.md

+      input=myvideo.mp4,stream=video,output=glockv.mp4 \
+      --enable_fixed_key_encryption --enable_fixed_key_decryption \
+      -key "$(< media.key)" -key_id "$(< media.key)"
+


You may want to add a way for user to check that the file is indeed encrypted with:

packager input=glocken_video_encrypted.mp4 --dump_stream_info

Good suggestion.

FYI, since this is useful for every step in the process, I've added a short section called "Check your work" at the beginning of the article.

beaufortfrancois · 2017-06-27T13:45:37Z

src/content/en/fundamentals/media/manipulating/files.md

+digits and does not allow a 32 digit key, both flags use the same value.
+
+    packager \
+      input=myvideo.mp4,stream=audio,output=glocka.m4a \


output=glocken_video_encrypted.mp4 and output=glocken_audio_encrypted.m4a makes it easier to read.

beaufortfrancois · 2017-06-29T07:08:27Z

@jpmedley Please resolve conflicts so that https://pr-4682-dot-web-central.appspot.com/web/fundamentals/media/manipulating/files is updated accordingly to your latest changes.

into manipulation

cwilso · 2017-06-29T17:20:52Z

src/content/en/fundamentals/media/manipulating/files.md

+To be more specific about the goals, the result of the procedures described here
+will be media resources with the following characteristics:
+
+  Audio and video streams are split into separate files


I would add that comment here, since it's 75 more lines before you get there - just a "(this is required by DASH)".

cwilso · 2017-06-29T17:21:57Z

src/content/en/fundamentals/media/manipulating/files.md

+
+This does not cover everything you could do to a media file before posting it to
+the web, not by a longshot. To be fair, this subject is one deserving of its own
+website. I'm hoping this introduction will give you enough to help you find your


I would cut the "To be fair..." sentence, and let's keep this on the list to expand.

Leaving in. I want to be open and honest about the limits of what I've done.

jpmedley · 2017-06-29T18:14:55Z

@cwilso Conversation view is showing my response to your second comment, but not your first. Please click over to Files Changed to see it.

cwilso · 2017-06-29T18:43:16Z

ok.

beaufortfrancois · 2017-06-29T19:19:52Z

LGTM @jpmedley
Thank you!

Draft of file manipulation article.

fd49453

jpmedley added the DO NOT MERGE label Jun 12, 2017

jpmedley assigned samdutton, cwilso and paullewis Jun 12, 2017

googlebot added the cla: yes label Jun 12, 2017

Pass CI tests.

39197b5

kqyang reviewed Jun 15, 2017

View reviewed changes

Feedback from kqyang.

02d7e22

johnpallett reviewed Jun 19, 2017

View reviewed changes

kqyang reviewed Jun 19, 2017

View reviewed changes

jpmedley added 3 commits June 20, 2017 07:48

Feedback from johnpallet and kqyang.

7d4e967

Remove Packager instrs re bitrate.

3a29edb

Correct streams info.

36d4fad

paullewis removed their assignment Jun 22, 2017

Merge branch 'master' into manipulation

539a55d

jpmedley assigned beaufortfrancois and unassigned samdutton Jun 22, 2017

cwilso suggested changes Jun 22, 2017

View reviewed changes

samdutton reviewed Jun 22, 2017

View reviewed changes

jpmedley added 2 commits June 22, 2017 10:47

Feedback from cwilso and samdutton.

dafa15a

Merge branch 'manipulation' of https://github.com/google/WebFundamentals

1a1caa3

into manipulation

Minor corrections.

d447a37

beaufortfrancois suggested changes Jun 27, 2017

View reviewed changes

Feedback from beaufortfrancois, pt I.

f0724e9

jpmedley added 7 commits June 29, 2017 07:16

Merge branch 'master' into manipulation

1462717

More feedback changes.

ac40e56

Merge branch 'manipulation' of https://github.com/google/WebFundamentals

2712ebc

into manipulation

One more review by me.

cc9f77f

Move glocken.mov to cloud storage.

57371b8

Move glocken.mov to cloud storage.

b74bd4f

Merge branch 'manipulation' of https://github.com/google/WebFundamentals

bd1ecad

into manipulation

cwilso suggested changes Jun 29, 2017

View reviewed changes

cwilso approved these changes Jun 29, 2017

View reviewed changes

beaufortfrancois approved these changes Jun 29, 2017

View reviewed changes

Copy edit from dgash.

d5db8cd

jpmedley merged commit 689ff84 into master Jun 29, 2017

jpmedley deleted the manipulation branch June 29, 2017 19:57

jpmedley removed the DO NOT MERGE label Jun 30, 2017


		## File properties

		I've unpealed the onion. Yet if you compare what I've done to the list of stated


		ffmpeg -i glocken.mov -b:v 350K -b:a 350K glocken.mp4

		Notice that there are two bitrate flags, -b:a and -b:v. One is for audio and the

Draft of file manipulation article. #4682

Draft of file manipulation article. #4682

Conversation

jpmedley commented Jun 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpmedley commented Jun 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpmedley commented Jun 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpmedley commented Jun 26, 2017

beaufortfrancois left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beaufortfrancois commented Jun 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpmedley commented Jun 29, 2017

cwilso commented Jun 29, 2017

beaufortfrancois commented Jun 29, 2017