Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

choosing alternatives #133

Closed
mattgarrish opened this issue Oct 23, 2019 · 12 comments
Closed

choosing alternatives #133

mattgarrish opened this issue Oct 23, 2019 · 12 comments
Assignees

Comments

@mattgarrish
Copy link
Member

mattgarrish commented Oct 23, 2019

I raised this in a previous issue, but it got lost in the shuffle: what is the process for selecting alternatives? For example:

  • does the UA have to stop at the first it can handle?
  • can it select from any it can handle?
  • what if there are multiple alternatives and no encodingFormat? Does it try one and fail, or does it have to keep trying all listed?
@mattgarrish
Copy link
Member Author

Stepping back a bit, is alternate supposed to be a general mechanism for providing any kind of alternative for a resource, or has it gone beyond its original ambition and is only intended for specifying synchronized media?

If it's only for the latter, would a more specific name be in order, and can we leave the details of how it works to the sync media document?

/cc @marisademeglio

@iherman
Copy link
Member

iherman commented Oct 31, 2019

If it is in the pub manifest specification then it is not for sync media only imho.

@mattgarrish
Copy link
Member Author

mattgarrish commented Oct 31, 2019

If it is in the pub manifest specification then it is not for sync media only imho.

Right, but the request only came for this for defining the sync media alternatives. We're getting into EPUB's world of manifest fallbacks with this (and HTML sources, etc.), but haven't really discussed the expectations. Was it the intention in adding to get into general alternatives, or a byproduct? Will it cause confusion if it's used for both purposes?

I'm not completely sure how we decide the selection process, but even epub defines rules for the fallback chain.

If I were to try and formulate rules for selection, I suppose it would be along these lines:

  • check each alternative's encodingFormat and determine if it is usable, if so add to the set of possible alternatives
  • if encodingFormat is not set on any alternatives, try to load the first any one of the resources (maybe prioritize by file extension?). If it is unusable, provide an error to the user. Don't attempt to load any others.
  • if there is a set of usable alternatives, user agent can select from among them whichever it determines is optimal
  • if the (selected) resource is unusable (doesn't match its encodingFormat, is corrupt, etc.), provide an error to the user. Don't attempt to load any others.

@iherman
Copy link
Member

iherman commented Oct 31, 2019

I do not have enough experience to answer this...

@wareid @marisademeglio @llemeurfr @TzviyaSiegman @dauwhe are much better informed than I am...

My technical question is much more down-to-Earth: at the moment, the alternate value in the context file is set to be a 'set', ie, order does not count. If there is a priority, then the order does matter, and that must then be changed...

@mattgarrish
Copy link
Member Author

If there is a priority, then the order does matter, and that must then be changed...

Right, that's hard to say.If we want to emulate EPUB, then yes as EPUB you have a very specific chain you have to follow.

But I don't think order matters for HTML sources. We don't have to say "select first" as much as "select one", which could be based on file extension, if we opt for that approach.

@marisademeglio
Copy link
Contributor

My technical question is much more down-to-Earth: at the moment, the alternate value in the context file is set to be a 'set', ie, order does not count. If there is a priority, then the order does matter, and that must then be changed...

We've been talking about two concrete use cases for alternate:

  • sync media as an alternate to either audio or html reading order entries
  • html as an alternate to audio reading order entries

I don't think you can really order these. The most-preferred alternate, in any case, depends on what the user wants for their experience (to quickly scan an html transcript? to get a synchronized presentation?).

I'd rather say that if a reading system supports both encoding formats, then it should let the user decide. And if the encoding format isn't specified, then it's up to the reading system's ability to sniff it out or not. So: I vote to keep the list unordered.

Here's an interesting scenario related to relying on encoding format (but unrelated to ordered lists) - say there is a book with many reading order entries, and each one has two alternate values, and they both are of encoding format sync media. There are two, because each sync media file uses audio with different voices (w3c/sync-media-pub#27). So how does the reading system maintain continuity from chapter to chapter? This would be an issue for any alternate list containing multiples of the same encoding format. We don't currently put any restrictions on this. Should we prohibit multiples w. same encoding format? Or we could introduce a property (a sort of role value maybe) to help control the flow? Or, leave it out and see oddball naming conventions develop...

@mattgarrish
Copy link
Member Author

The most-preferred alternate, in any case, depends on what the user wants for their experience

Isn't this where personalization starts to come into play? I don't expect a user agent would actually provide an interface for the user to select from options in any given publication, or at least I've never seen anything like that pan out on the web (it's part of what killed longdesc).

So how does the reading system maintain continuity from chapter to chapter?

My guess is personalization again. Does the W3C work on this get into things like voice preference? @clapierre?

If you extended LinkedResource to add readBy, for example, you could indicate the narrator's gender on a file-by-file basis.

@mattgarrish
Copy link
Member Author

mattgarrish commented Nov 4, 2019

Before we can decide how to select resources, I suppose the more pressing problem is what properties we expect to be used for selecting resources. Options I can see here include:

  1. only defining encodingFormat at the pub manifest level, but allow profiles to extend LinkedResource with additional properties
  2. add height, width and bitrate from mediaobject in addition to encodingFormat, with same proviso that profiles may add others
  3. define our own property for media queries in addition to encodingFormat and possibly also keep bitrate

Any thoughts?

@clapierre
Copy link

My guess is personalization again. Does the W3C work on this get into things like voice preference? @clapierre?

Currently our specification does not get into these kinds of personalization options (voices, font sizing, etc.). We are primarily focused on adding additional semantics to web content so that it can be personalized by a UA.

@iherman
Copy link
Member

iherman commented Nov 5, 2019

This issue was discussed in a meeting.

  • No actions or resolutions
View the transcript Matt Garrish: #133
Matt Garrish: this is about what to make about alternate resources… saying they have to be unique on one property and the UA can make the choice… but we don’t talk about what the unique properties are, how the UA goes about picking them… essentially, it’s two questions: we have encoding format, but no properties for selecting alternatives? what do we need here? … what are we selecting from and what is the process for selecting them?
… should figure this out before going to CR… unfortunately, i don’t know what the best resolution is… seems like a case for media queries, but it might be too deep in the weeds… for selecting them, my general thought is we follow HTML–the UA picks the one it can handle, and if it can’t, you get an error message… without going resource to resource one by one
… those are the questions
Dave Cramer: using this as a general purpose mechanism makes me a little nervous… reminds me of multiple renditions in EPUB… and how to present a bunch of complex choices to users…
Ivan Herman: +1 to dauwhe
Rachel Comerford: +1 dauwhe
Dave Cramer: this seems like an entirely good thing to point to media overlays, which sort of augment rather than replace the underlying content, but defining a mechanism to provide an entirely different primary content experience seems like a world of complexity
Mateus Teixeira: +1
Garth Conboy: you had me at multiple renditions
Marisa DeMeglio: originally, with alternate, we didn’t think about offering more than one… then with audiobooks it came up that you can have a pure HTML alternate or synched text alternate…
… there’s sort of an edge case here, which is if you have two alternates… like deep voice or high pitched voice… and you want a consistent experience, so the selection should be the default… and the encoded format wouldn’t work in this case because it would be the same for either option
… the other thing was for it to be consistent with the way we write about URLs, to be consistent with the rest of our data values… i don’t know what we should choose… but i’m wary about inventing a selection mechanism… HTML-style audio fallback wouldn’t really work here either… seems like a very complicated issue
Ivan Herman: to add to the complication, we have to remind ourselves that the alternate originally appeared for audiobooks, and we decided other types of publication might use it, so we pushed it to pub manifest
… so any general mechanism defined here would be an absolute nightmare… so it should be strictly profile-specific, because otherwise there is no end to it
Garth Conboy: continuing from ivan… avoiding the nightmare seems like a good thing… is making it profile-specific a way to get around it for now?
Ivan Herman: yes, but we will need to say something about audiobooks… the nightmare is then more manageable in the general case, but we still need to define it… also depends on the kind of resource… audio is one thing, but also cover page alternates (images), which would possibly be a different kind of mechanism… like picking based on resolution
… so not only dependent on profile, but also the role of the resource for which the alternate is defined… it’s a multiple-dimension thing
Matt Garrish: this is sort of what my question is… does it belong in pub manifest or can it be defined for synched media? if that’s the primary use case, wouldn’t it be better to define it in that spec?
Ivan Herman: +1 to Avneesh
Avneesh Singh: i think that we should keep it as simple as possible, not get into a situation where we get into a nasty loop to define more and more things
Wendy Reid: +1
Avneesh Singh: we should keep the audiobook case simple
Ivan Herman: +10000 to Avneesh
Garth Conboy: so should we keep it only relevant to synched media?
Avneesh Singh: yes, and keep it specific to audio
Marisa DeMeglio: I don’t mind extending synched media to incorporate this, but i’m afraid it would be a very narrow solution, because as Avneesh said, you can have both HTML and audio, and it might get awkward on making that call… would rather not do anything… if there are so many books that can’t be disambiguated, then we realize that the standard has to accommodate more, but it doesn’t seem that pressing
Matt Garrish: I misunderstood, didn’t know there was an audiobook case here too… one thing we can do with it is say that alternate only provides different encoding formats, and leave it open to profiles to do more interesting things, and that’s it… it’s sort of open-ended at the moment what you’re supposed to make with it… the uses are not very well defined
Ivan Herman: we can make what Avneesh proposed to say that this version of this spec does not define anything, put it in a note in the document… and make it clear that future versions of the specs may come up with a more complex mechanism if there’s a need for it
… inventing things for use cases we don’t know about is not a good idea, but we know that this may require at some point a more complex mechanism, so we can make this note that we wait for use cases to come up
Garth Conboy: mattg, are you comfortable with that direction?
Matt Garrish: yes, just saying that we’re not defining anything at this point in time, and leave it to future versions and profiles
Garth Conboy: is that acceptable?
Marisa DeMeglio: yes, it’s ok. there is one accessibility use case about switching voices, but we need to do more research about it, and future versions of the spec could introduce a solution to that

@iherman
Copy link
Member

iherman commented Nov 5, 2019

Related PR: #146

@mattgarrish
Copy link
Member Author

Closing with #146

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants