From 84154b9a1f815ac9ea48e7a556f0a2224ed5808f Mon Sep 17 00:00:00 2001 From: Ilya Grigorik Date: Fri, 16 Sep 2016 09:48:26 -0700 Subject: [PATCH 1/7] allow Request to outlive environment settings object Related discussion in [1]. This exposes keepAlive flag within Request constructor and adds guards for limiting the size of such requests. [1] https://github.com/w3c/beacon/pull/27 --- Overview.src.html | 47 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 38 insertions(+), 9 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index 0cba4fafc..f76ba8e14 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -4223,7 +4223,7 @@

Body mixin

To extract a body and a `Content-Type` value from -object, run these steps: +object with optional maximum-body-size, run these steps:

  1. Let stream be the result of @@ -4232,6 +4232,7 @@

    Body mixin

  2. Let Content-Type be null.

  3. Let action be null. +

  4. Let body-size be zero.

  5. Switch on object's type: @@ -4293,12 +4294,20 @@

    Body mixin

    If action is non-null, run action in parallel:

      -

    1. Whenever one or more bytes are available, let bytes be the bytes and +

    2. Whenever one or more bytes are available, let bytes be the bytes, enqueue a Uint8Array object - wrapping an ArrayBuffer containing bytes to stream. If - creating the ArrayBuffer threw an exception, - error stream with that exception - and cancel running action. + wrapping an ArrayBuffer containing bytes to stream, and + increment body-size by the number of consumed bytes. + +

        +

      1. If creating the ArrayBuffer threw an exception, + error stream with that exception + and cancel running action. + +

      2. If maximum-body-size is provided and body-size is equal or greater + than the provided value, throw (TODO: ???)Error and cancel running action. + TODO: is there a way to bound how long the stream is allowed to stay alive? +

    3. When running action is done, close stream. @@ -4486,6 +4495,7 @@

      Request class

      readonly attribute RequestCache cache; readonly attribute RequestRedirect redirect; readonly attribute DOMString integrity; + readonly attribute boolean keepAlive; [NewObject] Request clone(); }; @@ -4505,6 +4515,7 @@

      Request class

      RequestCache cache; RequestRedirect redirect; DOMString integrity; + boolean keepAlive; any window; // can only be set to null }; @@ -4629,10 +4640,14 @@

      Request class

      cache mode, redirect mode is request's - redirect mode, and + redirect mode, integrity metadata is request's - integrity metadata. + integrity metadata, and + keep-alive flag is + request's + keep-alive flag + .

    4. If any of init's members are present, run these substeps: @@ -4737,6 +4752,10 @@

      Request class

      request's integrity metadata to it. +

    5. If init's keepAlive member is present, set + request's + keepAlive flag to it. +

    6. If init's method member is present, let method be it and run these substeps: @@ -4805,7 +4824,17 @@

      Request class

      1. Let Content-Type be null. -

      2. Set inputBody and Content-Type to the result of +

      3. If init's keepAlive member is present and is set to + true, then set inputBody and Content-Type to the + result of extracting init's + body member with maximum-body-size set by user agent policy. + Rethrow any exceptions. +

        This step ensures that requests that are allowed to outlive the + environment settings object and contain a body have bounded size and are not allowed to stay + alive indefinitely. The maximum body size for such requests, and the timeout, is subject to + user agent policy. + +

      4. Otherwise, set inputBody and Content-Type to the result of extracting init's body member. Rethrow any exceptions. From 164aaa07b4ed13e8174fb638644a54ab37e92cbf Mon Sep 17 00:00:00 2001 From: Ilya Grigorik Date: Thu, 29 Sep 2016 09:35:04 -0700 Subject: [PATCH 2/7] disallow ReadableStream if keepAlive flag is set --- Overview.src.html | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index f76ba8e14..45624a5be 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -4223,7 +4223,7 @@

        Body mixin

        To extract a body and a `Content-Type` value from -object with optional maximum-body-size, run these steps: +object with optional keepalive-body-size, run these steps:

        1. Let stream be the result of @@ -4232,7 +4232,6 @@

          Body mixin

        2. Let Content-Type be null.

        3. Let action be null. -

        4. Let body-size be zero.

        5. Switch on object's type: @@ -4290,24 +4289,22 @@

          Body mixin

          Set stream to object. +

        6. +

          If keepalive-body-size is provided and object's type is + ReadableStream, or object's size is greater + than keepalive-body-size, throw a TypeError and abort the + remaining steps. +

        7. If action is non-null, run action in parallel:

            -

          1. Whenever one or more bytes are available, let bytes be the bytes, +

          2. Whenever one or more bytes are available, let bytes be the bytes and enqueue a Uint8Array object - wrapping an ArrayBuffer containing bytes to stream, and - increment body-size by the number of consumed bytes. - -

              -

            1. If creating the ArrayBuffer threw an exception, - error stream with that exception - and cancel running action. - -

            2. If maximum-body-size is provided and body-size is equal or greater - than the provided value, throw (TODO: ???)Error and cancel running action. - TODO: is there a way to bound how long the stream is allowed to stay alive? -
            + wrapping an ArrayBuffer containing bytes to stream. If + creating the ArrayBuffer threw an exception, + error stream with that exception + and cancel running action.

          3. When running action is done, close stream. @@ -4827,7 +4824,7 @@

            Request class

          4. If init's keepAlive member is present and is set to true, then set inputBody and Content-Type to the result of extracting init's - body member with maximum-body-size set by user agent policy. + body member with keepalive-body-size set by user agent policy. Rethrow any exceptions.

            This step ensures that requests that are allowed to outlive the environment settings object and contain a body have bounded size and are not allowed to stay From 9a9847b3ecc0c94e932a010be4818afeba3aa27b Mon Sep 17 00:00:00 2001 From: Ilya Grigorik Date: Fri, 30 Sep 2016 10:31:41 -0700 Subject: [PATCH 3/7] integrate Anne's feedback --- Overview.src.html | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index 45624a5be..e557be39e 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -4223,7 +4223,7 @@

            Body mixin

            To extract a body and a `Content-Type` value from -object with optional keepalive-body-size, run these steps: +object with optional keepaliveBodySize, run these steps:

            1. Let stream be the result of @@ -4290,10 +4290,9 @@

              Body mixin

            2. -

              If keepalive-body-size is provided and object's type is - ReadableStream, or object's size is greater - than keepalive-body-size, throw a TypeError and abort the - remaining steps. +

              If keepaliveBodySize is provided and object's type is a + ReadableStream object, or object's size + is greater than keepaliveBodySize, then throw a TypeError.

            3. If action is non-null, run action in @@ -4492,7 +4491,7 @@

              Request class

              readonly attribute RequestCache cache; readonly attribute RequestRedirect redirect; readonly attribute DOMString integrity; - readonly attribute boolean keepAlive; + readonly attribute boolean keepalive; [NewObject] Request clone(); }; @@ -4512,7 +4511,7 @@

              Request class

              RequestCache cache; RequestRedirect redirect; DOMString integrity; - boolean keepAlive; + boolean keepalive; any window; // can only be set to null }; @@ -4749,9 +4748,9 @@

              Request class

              request's integrity metadata to it. -

            4. If init's keepAlive member is present, set +

            5. If init's keepalive member is present, then set request's - keepAlive flag to it. + keepalive flag to it.

            6. If init's method member is present, let @@ -4820,16 +4819,18 @@

              Request class

              1. Let Content-Type be null. +

              2. +

                Let keepaliveBodySize be set by user agent policy. + +

                This variable ensures that requests that are allowed to outlive the + environment settings object and contain a body have bounded size and are not allowed to stay + alive indefinitely. The maximum body size for such requests is subject to user agent policy. -

              3. If init's keepAlive member is present and is set to - true, then set inputBody and Content-Type to the - result of extracting init's - body member with keepalive-body-size set by user agent policy. - Rethrow any exceptions. -

                This step ensures that requests that are allowed to outlive the - environment settings object and contain a body have bounded size and are not allowed to stay - alive indefinitely. The maximum body size for such requests, and the timeout, is subject to - user agent policy. +

              4. +

                If init's keepalive member is present and is set to + true, then set inputBody and Content-Type to the + result of extracting init's + body member with keepaliveBodySize. Rethrow any exceptions.

              5. Otherwise, set inputBody and Content-Type to the result of extracting init's From 1ce51bfe1c5a9d66271c6a2b8d4b766245ff20f1 Mon Sep 17 00:00:00 2001 From: Ilya Grigorik Date: Mon, 3 Oct 2016 08:54:44 -0700 Subject: [PATCH 4/7] s/keep-alive/keepalive + extended flag description --- Overview.src.html | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index e557be39e..b5e1df802 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -680,11 +680,12 @@

                Requests

                (null or a browsing context). Unless stated otherwise it is null.

                A request has an associated -keep-alive flag. Unless stated otherwise it is unset. +keepalive flag. Unless stated otherwise it is unset. -

                This is used by navigator.sendBeacon and the HTML -img element to outlive the -environment settings object. +

                This can be used to allow the request to outlive the +environment settings object - e.g. +navigator.sendBeacon and the HTML img element set this flag. Requests +with this flag set are subject to additional processing requirements.

                A request has an associated skip-service-worker flag. Unless stated otherwise it is @@ -1335,7 +1336,7 @@

                Fetch groups

                terminated, for each associated fetch record whose request's done flag or -keep-alive flag is unset, +keepalive flag is unset, terminate the fetch record's fetch with reason fatal. @@ -4640,9 +4641,9 @@

                Request class

                integrity metadata is request's integrity metadata, and - keep-alive flag is + keepalive flag is request's - keep-alive flag + keepalive flag .
              6. From 97c6ecac803b60450ea7db9a84799528f0eb1cb9 Mon Sep 17 00:00:00 2001 From: Ilya Grigorik Date: Tue, 4 Oct 2016 09:25:01 -0700 Subject: [PATCH 5/7] add body size check in HTTP-network-or-cache fetch --- Overview.src.html | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index b5e1df802..c914195c1 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -3100,6 +3100,15 @@

                HTTP-network-or-cache fetch

                httpRequest's header list. +

              7. If contentLengthValue is non-null, + the httpRequest's keepalive flag is set, + and contentLengthValue is greater than user-agent-defined maximum body size, + then return a network error. + +

                The above user-agent-defined value ensures that requests that are + allowed to outlive the environment settings object and + contain a body have bounded size and are not allowed to stay alive indefinitely. +

              8. If httpRequest's referrer is a URL, then append @@ -4224,7 +4233,7 @@

                Body mixin

                To extract a body and a `Content-Type` value from -object with optional keepaliveBodySize, run these steps: +object with optional keepalive flag, run these steps:

                1. Let stream be the result of @@ -4291,9 +4300,9 @@

                  Body mixin

                2. -

                  If keepaliveBodySize is provided and object's type is a - ReadableStream object, or object's size - is greater than keepaliveBodySize, then throw a TypeError. +

                  If keepalive flag is set and object's type is a + ReadableStream object, then throw a + TypeError.

                3. If action is non-null, run action in @@ -4641,10 +4650,7 @@

                  Request class

                  integrity metadata is request's integrity metadata, and - keepalive flag is - request's - keepalive flag - . + keepalive flag is request's keepalive flag.

                4. If any of init's members are present, run these substeps: @@ -4750,8 +4756,7 @@

                  Request class

                  integrity metadata to it.

                5. If init's keepalive member is present, then set - request's - keepalive flag to it. + request's keepalive flag to it.

                6. If init's method member is present, let @@ -4820,18 +4825,11 @@

                  Request class

                  1. Let Content-Type be null. -

                  2. -

                    Let keepaliveBodySize be set by user agent policy. - -

                    This variable ensures that requests that are allowed to outlive the - environment settings object and contain a body have bounded size and are not allowed to stay - alive indefinitely. The maximum body size for such requests is subject to user agent policy. -

                  3. If init's keepalive member is present and is set to true, then set inputBody and Content-Type to the result of extracting init's - body member with keepaliveBodySize. Rethrow any exceptions. + body member with keepalive. Rethrow any exceptions.

                  4. Otherwise, set inputBody and Content-Type to the result of extracting init's @@ -4940,6 +4938,9 @@

                    Request class

                    return request's redirect mode. +

                    The keepalive attribute's getter +must return request's keepalive flag. +

                    The integrity attribute's getter must return request's integrity metadata. From 75f19828cfe3c67afe2eaf51b8331d1f49c69661 Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Thu, 13 Oct 2016 09:49:27 +0200 Subject: [PATCH 6/7] nits --- Overview.src.html | 59 ++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 29 deletions(-) diff --git a/Overview.src.html b/Overview.src.html index c914195c1..566191dd9 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -679,13 +679,13 @@

                    Requests

                    target browsing context (null or a browsing context). Unless stated otherwise it is null. -

                    A request has an associated -keepalive flag. Unless stated otherwise it is unset. +

                    A request has an associated keepalive flag. Unless +stated otherwise it is unset.

                    This can be used to allow the request to outlive the -environment settings object - e.g. -navigator.sendBeacon and the HTML img element set this flag. Requests -with this flag set are subject to additional processing requirements. +environment settings object, e.g., +navigator.sendBeacon and the HTML img element set this flag. Requests with +this flag set are subject to additional processing requirements.

                    A request has an associated skip-service-worker flag. Unless stated otherwise it is @@ -1336,8 +1336,7 @@

                    Fetch groups

                    terminated, for each associated fetch record whose request's done flag or -keepalive flag is unset, -terminate the +keepalive flag is unset, terminate the fetch record's fetch with reason fatal. @@ -3100,14 +3099,13 @@

                    HTTP-network-or-cache fetch

                    httpRequest's header list. -

                  5. If contentLengthValue is non-null, - the httpRequest's keepalive flag is set, - and contentLengthValue is greater than user-agent-defined maximum body size, - then return a network error. +

                  6. If contentLengthValue is non-null, httpRequest's + keepalive flag is set, and contentLengthValue is greater than a + user-agent-defined maximum, then return a network error.

                    The above user-agent-defined value ensures that requests that are - allowed to outlive the environment settings object and - contain a body have bounded size and are not allowed to stay alive indefinitely. + allowed to outlive the environment settings object and contain a + body, have a bounded size and are not allowed to stay alive indefinitely.

                  7. If httpRequest's referrer is a URL, then @@ -4233,7 +4231,7 @@

                    Body mixin

                    To extract a body and a `Content-Type` value from -object with optional keepalive flag, run these steps: +object, with an optional keepalive flag, run these steps:

                    1. Let stream be the result of @@ -4241,6 +4239,7 @@

                      Body mixin

                      ReadableStream object.

                    2. Let Content-Type be null. +

                    3. Let action be null.

                    4. @@ -4299,14 +4298,14 @@

                      Body mixin

                      Set stream to object. -

                    5. -

                      If keepalive flag is set and object's type is a - ReadableStream object, then throw a - TypeError. +

                    6. If keepalive flag is set and object's type is a + ReadableStream object, then throw a + TypeError.

                    7. -

                      If action is non-null, run action in - parallel: +

                      If action is non-null, run action + in parallel: +

                      1. Whenever one or more bytes are available, let bytes be the bytes and enqueue a Uint8Array object @@ -4756,7 +4755,8 @@

                        Request class

                        integrity metadata to it.

                      2. If init's keepalive member is present, then set - request's keepalive flag to it. + request's keepalive flag if init's keepalive + member is true, and unset it otherwise.

                      3. If init's method member is present, let @@ -4825,11 +4825,11 @@

                        Request class

                        1. Let Content-Type be null. -

                        2. -

                          If init's keepalive member is present and is set to - true, then set inputBody and Content-Type to the - result of extracting init's - body member with keepalive. Rethrow any exceptions. + +

                        3. If init's keepalive member is present and is true, then set + inputBody and Content-Type to the result of + extracting init's body + member, with keepalive flag set. Rethrow any exceptions.

                        4. Otherwise, set inputBody and Content-Type to the result of extracting init's @@ -4938,13 +4938,14 @@

                          Request class

                          return request's redirect mode. -

                          The keepalive attribute's getter -must return request's keepalive flag. -

                          The integrity attribute's getter must return request's integrity metadata. +

                          The keepalive attribute's getter +must return true if request's keepalive flag +is set, and false otherwise. +

                          The clone() method, when invoked, must From 630758191501dad390844886f1a447d9f3832f32 Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Thu, 13 Oct 2016 09:59:42 +0200 Subject: [PATCH 7/7] preserve old keep-alive ID plus more nits plus generate --- Overview.html | 59 ++++++++++++++++++++++++++++++++++++----------- Overview.src.html | 25 ++++++++++---------- 2 files changed, 58 insertions(+), 26 deletions(-) diff --git a/Overview.html b/Overview.html index 013c51240..5ac036e2d 100644 --- a/Overview.html +++ b/Overview.html @@ -7,7 +7,7 @@

                          Fetch

                          -

                          Living Standard — Last Updated 12 October 2016

                          +

                          Living Standard — Last Updated 13 October 2016

                          Participate: @@ -749,12 +749,13 @@

                          3.1.5 Requests

                          target browsing context (null or a browsing context). Unless stated otherwise it is null. -

                          A request has an associated -keep-alive flag. Unless stated otherwise it is unset. +

                          A request has an associated +keepalive flag. Unless stated otherwise it is unset. -

                          This is used by navigator.sendBeacon and the HTML -img element to outlive the -environment settings object. +

                          This can be used to allow the request to outlive the +environment settings object, e.g., +navigator.sendBeacon and the HTML img element set this flag. Requests with +this flag set are subject to additional processing requirements.

                          A request has an associated skip-service-worker flag. Unless stated otherwise it is @@ -1405,8 +1406,7 @@

                          3.3 Fetch groups

                          terminated, for each associated fetch record whose request's done flag or -keep-alive flag is unset, -terminate the +keepalive flag is unset, terminate the fetch record's fetch with reason fatal. @@ -3169,6 +3169,15 @@

                          5.5 HTTP-network httpRequest's header list. +
                        5. +

                          If contentLengthValue is non-null, httpRequest's + keepalive flag is set, and contentLengthValue is greater than a + user-agent-defined maximum, then return a network error. + +

                          The above user-agent-defined maximum ensures that requests that are + allowed to outlive the environment settings object and contain + a body, have a bounded size and are not allowed to stay alive indefinitely. +

                        6. If httpRequest's referrer is a URL, then append @@ -4293,7 +4302,7 @@

                          6.2 Body mixin

                          To extract a body and a `Content-Type` value from -object, run these steps: +object, with an optional keepalive flag, run these steps:

                          1. Let stream be the result of @@ -4301,6 +4310,7 @@

                            6.2 Body mixin

                            ReadableStream object.

                          2. Let Content-Type be null. +

                          3. Let action be null.

                          4. @@ -4359,9 +4369,14 @@

                            6.2 Body mixin

                            Set stream to object.

                          7. +

                        7. If keepalive flag is set and object's type is a + ReadableStream object, then throw a + TypeError. +

                        8. -

                          If action is non-null, run action in - parallel: +

                          If action is non-null, run action + in parallel: +

                          1. Whenever one or more bytes are available, let bytes be the bytes and enqueue a Uint8Array object @@ -4556,6 +4571,7 @@

                            6.3 Request class

                            readonly attribute RequestCache cache; readonly attribute RequestRedirect redirect; readonly attribute DOMString integrity; + readonly attribute boolean keepalive; [NewObject] Request clone(); }; @@ -4575,6 +4591,7 @@

                            6.3 Request class

                            RequestCache cache; RequestRedirect redirect; DOMString integrity; + boolean keepalive; any window; // can only be set to null }; @@ -4699,10 +4716,11 @@

                            6.3 Request class

                            cache mode, redirect mode is request's - redirect mode, and + redirect mode, integrity metadata is request's - integrity metadata. + integrity metadata, and + keepalive flag is request's keepalive flag.

                          2. If any of init's members are present, run these substeps: @@ -4807,6 +4825,10 @@

                            6.3 Request class

                            request's integrity metadata to it. +

                          3. If init's keepalive member is present, then set request's + keepalive flag if init's keepalive member is true, and unset + it otherwise. +

                          4. If init's method member is present, let method be it and run these substeps: @@ -4875,7 +4897,12 @@

                            6.3 Request class

                            1. Let Content-Type be null. -

                            2. Set inputBody and Content-Type to the result of +

                            3. If init's keepalive member is present and is true, then set + inputBody and Content-Type to the result of + extracting init's body + member, with keepalive flag set. Rethrow any exceptions. + +

                            4. Otherwise, set inputBody and Content-Type to the result of extracting init's body member. Rethrow any exceptions. @@ -4986,6 +5013,10 @@

                              6.3 Request class

                              must return request's integrity metadata. +

                              The keepalive attribute's getter +must return true if request's keepalive flag +is set, and false otherwise. +

                              The clone() method, when invoked, must diff --git a/Overview.src.html b/Overview.src.html index 566191dd9..5bddb49ae 100644 --- a/Overview.src.html +++ b/Overview.src.html @@ -679,8 +679,8 @@

                              Requests

                              target browsing context (null or a browsing context). Unless stated otherwise it is null. -

                              A request has an associated keepalive flag. Unless -stated otherwise it is unset. +

                              A request has an associated +keepalive flag. Unless stated otherwise it is unset.

                              This can be used to allow the request to outlive the environment settings object, e.g., @@ -3099,13 +3099,14 @@

                              HTTP-network-or-cache fetch

                              httpRequest's header list. -

                            5. If contentLengthValue is non-null, httpRequest's - keepalive flag is set, and contentLengthValue is greater than a - user-agent-defined maximum, then return a network error. +

                            6. +

                              If contentLengthValue is non-null, httpRequest's + keepalive flag is set, and contentLengthValue is greater than a + user-agent-defined maximum, then return a network error. -

                              The above user-agent-defined value ensures that requests that are - allowed to outlive the environment settings object and contain a - body, have a bounded size and are not allowed to stay alive indefinitely. +

                              The above user-agent-defined maximum ensures that requests that are + allowed to outlive the environment settings object and contain + a body, have a bounded size and are not allowed to stay alive indefinitely.

                            7. If httpRequest's referrer is a URL, then @@ -4754,9 +4755,9 @@

                              Request class

                              request's integrity metadata to it. -

                            8. If init's keepalive member is present, then set - request's keepalive flag if init's keepalive - member is true, and unset it otherwise. +

                            9. If init's keepalive member is present, then set request's + keepalive flag if init's keepalive member is true, and unset + it otherwise.

                            10. If init's method member is present, let @@ -4826,7 +4827,7 @@

                              Request class

                              1. Let Content-Type be null. -

                              2. If init's keepalive member is present and is true, then set +

                              3. If init's keepalive member is present and is true, then set inputBody and Content-Type to the result of extracting init's body member, with keepalive flag set. Rethrow any exceptions.