Skip to content
This repository

Created tests for http://www.w3.org/TR/html-media-capture/ #306

Closed
wants to merge 5 commits into from

6 participants

Zhiqiang Zhang hoppipolla-critic-bot Dominique Hazael-Massieux cathychan Anssi Kostiainen Tobie Langel
Zhiqiang Zhang
Collaborator

No description provided.

hoppipolla-critic-bot

Critic review: https://critic.hoppipolla.co.uk/r/284

This is an external review system which you may optionally use for the code review of your pull request.

html-media-capture/capture_video_multiple_device.html
((3 lines not shown))
  3 + <head>
  4 + <meta charset='utf-8'>
  5 + <title>HTML Media Capture Test: capture_video_multiple_device</title>
  6 + <link rel='author' title='Intel' href='http://www.intel.com'>
  7 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#the-capture-attribute'>
  8 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#security'>
  9 + </head>
  10 + <body>
  11 + <p><strong>This test is only useful on devices that supports multiple cameras (e.g. a front-facing camera in addition to a primary camera).</strong></p>
  12 +
  13 + <p>Click the button, allow microphone and camera access, and start to capture a video.</p>
  14 +
  15 + <p>Test passes</p>
  16 + <ol>
  17 + <li>if there appears a video capture screen,</li>
  18 + <li>if there is a way to change camera,</li>
6
Tobie Langel Collaborator
tobie added a note

This isn't normative.

Zhiqiang Zhang Collaborator
zqzhang added a note

Thanks for the review.

http://www.w3.org/TR/html-media-capture/#security, says,
Similarly, the user agent SHOULD allow the user:

  • to select the exact media capture device to be used if there exists multiple devices of the same type (e.g. a front-facing camera in addition to a primary camera).

As the test said, it is only for > 1 cameras.

Zhiqiang Zhang Collaborator

@tobie, looking back to the spec, I find below statement. Could you please review this again? Many thanks.

[[
2. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.
]]

Tobie Langel Collaborator
tobie added a note

Sorry for dropping the ball on this.

I read the spec as: if your device has two camera's "there
may exist valid reasons in particular circumstances"[1] not to provide UI for the user to pick one,
"but the full implications must be understood and carefully weighed before choosing a different course."[1]

So basically, there might be might be situations where it's OK not to offer this option to the user and still be spec compliant. So this brings up a methodology question of how we should deal with this in practice.

This has been discussed in the past[2].

I'm going to bring it back up the public-test-infra@ mailing list and propose a resolution.

Sounds good?


[1] http://www.ietf.org/rfc/rfc2119.txt
[2] http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0053.html

Zhiqiang Zhang Collaborator

Thanks for pointing me out this. It sounds like an issue of conformance testing of RFC2119 key word 'should'. The above talk [2] sounds reasonable.

However, it is a little different from the coverage tool, that you proposed 4 tests per RFC2119 key word ('must' and 'should') by default :)

It is good to find out a resolution around this, thanks.

Tobie Langel Collaborator
tobie added a note

However, it is a little different from the coverage tool, that you proposed 4 tests per RFC2119 key word ('must' and 'should') by default :)

Hehe. Good catch. :)

Let's seek to get consensus on this in the mailing list[3] and apply the decision consistently. My personal preference would be to mark optional tests with meta-data but still include them in the repo.


[3] http://lists.w3.org/Archives/Public/public-test-infra/2013JulSep/0240.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
html-media-capture/capture_video_multiple_device.html
((5 lines not shown))
  5 + <title>HTML Media Capture Test: capture_video_multiple_device</title>
  6 + <link rel='author' title='Intel' href='http://www.intel.com'>
  7 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#the-capture-attribute'>
  8 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#security'>
  9 + </head>
  10 + <body>
  11 + <p><strong>This test is only useful on devices that supports multiple cameras (e.g. a front-facing camera in addition to a primary camera).</strong></p>
  12 +
  13 + <p>Click the button, allow microphone and camera access, and start to capture a video.</p>
  14 +
  15 + <p>Test passes</p>
  16 + <ol>
  17 + <li>if there appears a video capture screen,</li>
  18 + <li>if there is a way to change camera,</li>
  19 + <li>if there is a way to disable the audio record, and</li>
  20 + <li>if there is a way to stop the video capture.</li>
4
Tobie Langel Collaborator
tobie added a note

This isn't normative either, afaik. These are QoI issues.

Zhiqiang Zhang Collaborator
zqzhang added a note

Yes, the spec says nothing about how to stop (here I mean quit) the media capture. But as a complete test, tester should find a way. :)

Zhiqiang Zhang Collaborator

@tobie, looking back to the spec, I find below statement. Could you please review this again? Many thanks.

[[
2. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.
]]

Tobie Langel Collaborator
tobie added a note

You're right. Same issue with SHOULD vs MUST here though. Let's try to find a resolution around this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dominique Hazael-Massieux
Owner

w3c-test:mirror

html-media-capture/capture_reflect.html
((18 lines not shown))
  18 + attribute boolean capture;
  19 + };
  20 + </pre>
  21 +
  22 + <div style='display:none'>
  23 + <input id='absent' type='file' accept='image/*'>
  24 + <input id='present' type='file' accept='image/*' capture>
  25 + <input id='present-empty-string' type='file' accept='image/*' capture=''>
  26 + <input id='present-canonical-name' type='file' accept='image/*' capture=capture>
  27 + </div>
  28 +
  29 + <div id='log'></div>
  30 +
  31 + <script>
  32 + test(function() {
  33 + var inputs = document.querySelector('input');
1
cathychan
cathychan added a note

If the intention is to perform this test on all input elements, querySelectorAll should be used instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
html-media-capture/capture_reflect.html
((19 lines not shown))
  19 + };
  20 + </pre>
  21 +
  22 + <div style='display:none'>
  23 + <input id='absent' type='file' accept='image/*'>
  24 + <input id='present' type='file' accept='image/*' capture>
  25 + <input id='present-empty-string' type='file' accept='image/*' capture=''>
  26 + <input id='present-canonical-name' type='file' accept='image/*' capture=capture>
  27 + </div>
  28 +
  29 + <div id='log'></div>
  30 +
  31 + <script>
  32 + test(function() {
  33 + var inputs = document.querySelector('input');
  34 + for (var obj in inputs) {
2
cathychan
cathychan added a note

The use of for..in loop on a NodeList object is not recommended, as it would loop through other properties of the NodeList. See https://developer.mozilla.org/en-US/docs/Web/API/NodeList

A simple for loop should be used instead:
for(var i=0, obj; i<inputs.length, obj=inputs[i]; i++) {

Zhiqiang Zhang Collaborator
zqzhang added a note

@cathychan, thanks for the review and comments, updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
cathychan

The permission model being used in these tests is different from the one depicted in the spec. The spec suggests implicit consent through user interaction, whereas the test instructions presume an explicit allow/deny prompt ("Click the button, allow camera access, and start to capture an image.", "Make sure that there is no granted camera access.", "Click the button, deny camera access.")

Furthermore, the tests do not verify that the UA correctly produces a file when the user goes ahead with a capture, or correctly produces no files when the user denies the capture (implicitly by cancelling the capture).

I would re-work the capture tests as follows, using capture_image.html and capture_image_deny.html as examples.

capture_image.html

  • instructions: Click the button, and capture an image.
  • add script to verify that one (and only one) image file becomes available.

capture_image_deny.html

  • instructions: Click the button, but cancel capturing an image.
  • add script to verify that no files become available.

I would skip the check on whether an image capture screen is shown, or put that in a separate test, as that is actually a SHOULD requirement.

François Daoust tidoust referenced this pull request from a commit in tidoust/mobile-web-app-standards
François Daoust tidoust Fixed broken links to w3c-test.org (specs and test suites)
A number of specifications and test suites got moved to different folders on
w3c-test.org. This commit should fix all broken links that appear in the
final roadmap.

As an exception to the rule, I could not find the tests referenced by the
inputaccept.json file for the Media Capture specification. I see there's
an on-going pull request at:
  w3c/web-platform-tests#306
... but files do not appear under http://w3c-test.org/submissions. The
broken link remains.

Not related but note the notifications API tutorial is no longer available
on WebPlatform.org so I dropped the section from the notification.json file.
448b508
Anssi Kostiainen
anssiko commented

@zqzhang The spec was updated as follows to address the issues raised in this PR: http://dev.w3.org/cvsweb/2009/dap/camera/Overview.src.html.diff?r1=1.5;r2=1.6;f=h

Namely, the Security and privacy considerations section is now non-normative, and as such contains no test assertions: http://dev.w3.org/2009/dap/camera/#security

Could you update the PR to reflect this? Thanks!

Anssi Kostiainen
anssiko commented

LGTM

@zqzhang Thanks!

Zhiqiang Zhang
Collaborator
zqzhang commented

Squashed and landed in 5d797b8

Zhiqiang Zhang zqzhang closed this
Zhiqiang Zhang zqzhang deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
44 html-media-capture/capture_fallback_file_upload.html
... ... @@ -0,0 +1,44 @@
  1 +<!DOCTYPE html>
  2 +<html>
  3 + <head>
  4 + <meta charset='utf-8'>
  5 + <title>HTML Media Capture Test: capture_fallback_file_upload</title>
  6 + <link rel='author' title='Intel' href='http://www.intel.com'>
  7 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#the-capture-attribute'>
  8 + <link rel='help' href='http://www.w3.org/TR/html5/forms.html#file-upload-state-(type=file)'>
  9 + <meta name='flags' content='interact'>
  10 + <script src='/resources/testharness.js'></script>
  11 + <script src='/resources/testharnessreport.js'></script>
  12 + </head>
  13 + <body>
  14 + <p>This test validates that the user agent must act as if there was no capture attribute, when the accept attribute value is set to a MIME type that has no associated capture control type.</p>
  15 +
  16 + <p>Test steps:</p>
  17 + <ol>
  18 + <li>Download <a href='support/upload.txt'>upload.txt</a> to local.</li>
  19 + <li>Select the local upload.txt file to run the test.</li>
  20 + </ol>
  21 +
  22 + <form>
  23 + <input id='fileChooser' type='file' accept='text/plain' capture>
  24 + </form>
  25 +
  26 + <div id='log'></div>
  27 +
  28 + <script>
  29 + setup({explicit_done: true, explicit_timeout: true});
  30 +
  31 + var fileInput = document.querySelector('#fileChooser');
  32 +
  33 + on_event(fileInput, 'change', function(evt) {
  34 + test(function() {
  35 + var fileList = document.querySelector('#fileChooser').files;
  36 + assert_equals(fileList.length, 1, 'fileList length is 1');
  37 + assert_equals(fileList.item(0).name, 'upload.txt', 'file name string is "upload.txt"');
  38 + }, 'Check if input.capture fallback to file upload when the accept attribute value is set to a MIME type that has no associated capture control type');
  39 +
  40 + done();
  41 + });
  42 + </script>
  43 + </body>
  44 +</html>
57 html-media-capture/capture_reflect.html
... ... @@ -0,0 +1,57 @@
  1 +<!DOCTYPE html>
  2 +<html>
  3 + <head>
  4 + <meta charset='utf-8'>
  5 + <title>HTML Media Capture Test: capture_reflect</title>
  6 + <link rel='author' title='Intel' href='http://www.intel.com/'>
  7 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#the-capture-attribute'>
  8 + <link rel='help' href='http://www.w3.org/html/wg/drafts/html/CR/infrastructure.html#reflect'>
  9 + <link rel='help' href='http://www.w3.org/html/wg/drafts/html/CR/infrastructure.html#boolean-attributes'>
  10 + <meta name='flags' content='dom'>
  11 + <meta name='assert' content='Test checks that the capture IDL attribute must reflect the content attribute of the same name.'>
  12 + <script src='/resources/testharness.js'></script>
  13 + <script src='/resources/testharnessreport.js'></script>
  14 + </head>
  15 + <body>
  16 + <pre style='display:none'>
  17 + partial interface HTMLInputElement {
  18 + attribute boolean capture;
  19 + };
  20 + </pre>
  21 +
  22 + <div style='display:none'>
  23 + <input id='absent' type='file' accept='image/*'>
  24 + <input id='present' type='file' accept='image/*' capture>
  25 + <input id='present-empty-string' type='file' accept='image/*' capture=''>
  26 + <input id='present-canonical-name' type='file' accept='image/*' capture=capture>
  27 + </div>
  28 +
  29 + <div id='log'></div>
  30 +
  31 + <script>
  32 + test(function() {
  33 + var inputs = document.querySelectorAll('input');
  34 + for (var i=0, obj; i<inputs.length, obj=inputs[i]; i++) {
  35 + assert_own_property(obj, 'capture');
  36 + assert_equals(typeof obj.capture, 'boolean');
  37 + }
  38 + }, 'Element input should have own property capture');
  39 +
  40 + test(function() {
  41 + assert_false(document.querySelector('#absent').capture);
  42 + }, 'input.capture is false when the capture attribute is absent');
  43 +
  44 + test(function() {
  45 + assert_true(document.querySelector('#present').capture);
  46 + }, 'input.capture is true when the capture attribute is present');
  47 +
  48 + test(function() {
  49 + assert_true(document.querySelector('#present-empty-string').capture);
  50 + }, 'input.capture is true when the capture attribute is present as empty string');
  51 +
  52 + test(function() {
  53 + assert_true(document.querySelector('#present-canonical-name').capture);
  54 + }, 'input.capture is true when the capture attribute is present as canonical name');
  55 + </script>
  56 + </body>
  57 +</html>
50 html-media-capture/idlharness.html
... ... @@ -0,0 +1,50 @@
  1 +<!DOCTYPE html>
  2 +<html>
  3 + <head>
  4 + <meta charset='utf-8'>
  5 + <title>HTML Media Capture IDL tests</title>
  6 + <link rel='author' title='Intel' href='http://www.intel.com/'>
  7 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#conformance'>
  8 + <link rel='help' href='http://www.w3.org/TR/html-media-capture/#the-capture-attribute'>
  9 + <script src='/resources/testharness.js'></script>
  10 + <script src='/resources/testharnessreport.js'></script>
  11 + <script src='/resources/WebIDLParser.js'></script>
  12 + <script src='/resources/idlharness.js'></script>
  13 + </head>
  14 + <body>
  15 + <h1>HTML Media Capture IDL tests</h1>
  16 + <div id='log'></div>
  17 +
  18 + <pre id='untested_idl' style='display:none'>
  19 + interface HTMLInputElement {
  20 + };
  21 + interface HTMLElement {
  22 + };
  23 + interface Element{
  24 + };
  25 + </pre>
  26 +
  27 + <pre id='idl' style='display:none'>
  28 + partial interface HTMLInputElement {
  29 + attribute boolean capture;
  30 + };
  31 + </pre>
  32 +
  33 + <form style='display:none'>
  34 + <input id='testElement' type='file' accept='image/*' capture>
  35 + </form>
  36 +
  37 + <script>
  38 + (function() {
  39 + var idl_array = new IdlArray();
  40 +
  41 + idl_array.add_untested_idls(document.getElementById('untested_idl').textContent);
  42 + idl_array.add_idls(document.getElementById('idl').textContent);
  43 +
  44 + var testElement = document.getElementById('testElement');
  45 + idl_array.add_objects({HTMLInputElement: [testElement]});
  46 + idl_array.test();
  47 + })();
  48 + </script>
  49 + </body>
  50 +</html>
1  html-media-capture/support/upload.txt
... ... @@ -0,0 +1 @@
  1 +Hello

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.