Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when parsing XML ebuttm:documentStartOfProgramme #492

Closed
danielthepope opened this issue Jul 18, 2019 · 1 comment
Closed

Error when parsing XML ebuttm:documentStartOfProgramme #492

danielthepope opened this issue Jul 18, 2019 · 1 comment

Comments

@danielthepope
Copy link
Collaborator

I am trying to parse an EBU-TT-Live document using EBUTT3Document.create_from_xml(xml). The xml variable is a string read in from a file.

The XML contains the following in the <ebuttm:documentMetadata> block:

<ebuttm:documentStartOfProgramme>10:00:00:00</ebuttm:documentStartOfProgramme>

However I'm getting a KeyError in ebu_tt_live/bindings/_ebuttdt.py:73: No such key timing_attribute_name in the XML parsing context.

@nigelmegitt
Copy link
Collaborator

Thanks @danielthepope great issue! I can see that there are two problems (at least):

  1. The XSD doesn't match the Tech3390 spec for that element type, which allows either ebuttdt:smpteTimingType or ebuttdt:clockTimingType.
  2. The code has a mix-in for conversion/semantic validation of time expressions that both a) seems to assume that time expressions are always in attributes (which in this case is untrue - it is in an element) and b) runs despite no particular validation or conversion really being needed.

nigelmegitt added a commit that referenced this issue Oct 16, 2019
* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).
nigelmegitt added a commit that referenced this issue Oct 18, 2019
* Fix #492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.
danielthepope pushed a commit to bbc/ebu-tt-live-toolkit that referenced this issue Oct 30, 2019
* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Nov 23, 2021
* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly

Closes ebu#490.

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Issue 447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* Issue 507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.

* Up version to 2.1.2
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Dec 7, 2021
Pulls in upstream fixes from Release/2.1.2 (ebu#522) and fixes an issue where, on conversion to EBU-TT-D, `<br>` children of `<p>` elements that have child `<span>` elements with differing timing also get wrapped in appropriately timed `<span>`s and therefore are not active always. Fixes a presentation error for some content.

### New stuff

* Add and pass tests

Drive-by discovery and fix of a bug in `denester.py` that did not handle `span` elements with an indefinite computed end time correctly.

In `ebuttd_nested_elements.feature`, the new code bumps the span that gets style S2S6 along one, because the `br` element before it now gets wrapped in a span.

Add a BDD for timings present on both p and consecutive span elements with an intervening br element.

* Remove Pipfile and don't need pytest-catchlog

* Up version to 3.0.1

* Remove restriction on hyperlink

Appears that python-hyper/hyperlink#16 has been resolved.


### Notes from pulled in changes:

* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly (ebu#490).

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* ebu#447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* ebu#507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Dec 7, 2023
* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly

Closes ebu#490.

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Issue 447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* Issue 507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.

* Up version to 2.1.2
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Dec 25, 2023
* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly

Closes ebu#490.

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Issue 447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* Issue 507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.

* Up version to 2.1.2
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Dec 25, 2023
* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly

Closes ebu#490.

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Issue 447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* Issue 507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.

* Up version to 2.1.2
nigelmegitt added a commit to bbc/ebu-tt-live-toolkit that referenced this issue Dec 25, 2023
* Add service_identity module dependency

Removes a test warning, improves twisted's ability to verify certificates.

* Fix ebu#474

Use relative links to templates.
Put message payload into the UIP display so that the receiver can read it without having to open the message XML up.

* Remove requirement for pytest-capturelog, removed from

No longer needed because it has been merged into the core.

* Refactor proxy handling for client

Track a change to proxy handling in Autobahn / twisted for ebu#477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.

* More elegant fix for proxy

We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.

* Don't call fixtures directly

Closes ebu#490.

Calling fixtures directly is deprecated, this solution described at
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly
seems to work, creating a named fixture rather than defining the "then"
step as a fixture directly.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Restrict autobahn to versions before 18

Temporarily fixes ebu#496, prior to addressing the problem properly anyway.

* Issue 447 resequencer documentation (ebu#515)

* Fix indentation warning

* Fix underline length warning

* Fix document generation warning

* Add Resequencer Node documentation

Closes ebu#447 .

* Address review comment

Merging now because this affects documentation only, and this allows progress to continue on other pull requests.

* Handle timing types in elements without crashing (ebu#516)

* Fix ebu#492

* Define the type of `ebuttm:documentStartOfProgramme` correctly.
* When processing a timing type outside of the context of an attribute, bypass the timebase validation.

This is possibly a temporary hack, since element timing validation might be needed one day, but since we don't do anything with metadata elements, this isn't a disaster (yet).

* Add tests

Checks that including a valid time in a `ebuttm:documentStartOfProgramme` element does not cause a processing or validation error.

* Issue 507 timedelta (ebu#508)

* Correctly interpret decimal fractions of seconds

Fixes ebu#507 by interpreting decimal fractions of seconds as milliseconds and adding tests to verify this behaviour for 3 and 4 decimal places.

Removes `_int_or_none` which doesn't seem to be needed

* Specify incompatible attributes (ebu#518)

Fixes ebu#514.

* Resequencer: allow to immediately issue documents (ebu#510)

So far the resequencer starts to regularly issue documents only after
the first EBU-TT Live document has been received. However for some use
cases this might be inconvenient as an active document must exist at all
times e.g. when creating segments for an MPEG-DASH stream. The reason
for documents being issued only after the first received EBU-TT Live
document is that certain parameters of that document are used for
initialisation.

This commit adds a new configuration parameter that specifies a document
which will be used for initialisation (instead of the first received
EBU-TT Live document). Therefore the resequencer will immediately (be
able to) start issuing documents after its creation.

Closes ebu#505.

* Resequencer: fix sequence number 1 if no content (ebu#509)

While the EBUTT3Splicer is provided with the current sequence number
stored/incremented by the Resequencer, the `create_compatible_document`
method so far refers to the sequence number stored internally by
EBUTT3DocumentSequence instead. This leads to a sequence number of 1 in
the issued output document if no subtitle content is available.

To align with the correct (strictly monotonic increasing) sequence
number, always use the one stored by the Resequencer.

Fixes ebu#502.

* Up version to 2.1.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants