Change the default value for start_url

[mgiuca]

Breaking: Changed the default start_url to the parent path of the manifest.

Previously, it defaulted to the document URL, which was not stable (the same manifest produces different results in different contexts) and is not guaranteed to exist.

This also makes the document URL optional for the "processing the manifest" algorithm, since it is now only used to check that the document and start_url are same origin.

---

Preview | Diff

[kenchris]

Lgtm

[mgiuca]

Lgtm

Really? This was meant as a straw-man (to show the difficulty of having no default --- see the issue box added by this patch). Marcos expressed uncomfortability with this solution also.

Let's sleep on it over the weekend.

[kenchris]

Yes, I wanted to say that I am OK with it, but of course you shouldn't merge it as there are still concerns from Marcos. Maybe I should have made that clear

[kenchris]

I am also not such a fan of making it required for the reasons you stated. But then again scope really only make sense in combination with start_url. So maybe we should make some properties dependent on others instead

[mgiuca]

On the bug I proposed making . the default instead (with a somewhat forward-compatibility breaking thing). I'm liking this option more and more. WDYT?

[kenchris]

It sounds like a sane default to me. But maybe some sites actually don't want their site so show up as a PWA even when using a service worker, so maybe we need a way to opt out of that

[mgiuca]

But maybe some sites actually don't want their site so show up as a PWA even when using a service worker, so maybe we need a way to opt out of that

Practically speaking, the best signal an app can provide that it doesn't want to be a PWA is "display: browser" (which is the default), not omitting the start_url.

Any manifest with "display: browser" (or missing display) will not be installable in Chrome, and should generally be taken as a signal that it doesn't want to be a PWA.

Any manifest with "display: standalone" is sending a signal that it wants to be run in a standalone window (and thus should be installable).

[marcoscaceres]

@mgiuca wrote:

Practically speaking, the best signal an app can provide that it doesn't want to be a PWA is "display: browser" (which is the default), not omitting the start_url.

Agree. It's also why we made it the default.

On the bug I proposed making . the default instead (with a somewhat forward-compatibility breaking thing). I'm liking this option more and more. WDYT?

That sounds good to me.

[mgiuca]

Updated to set the default to ".".

This will be a behavioural change. I'll ask internally whether people are OK with this.

[nhoizey]

Any manifest with "display: browser" (or missing display) will not be installable in Chrome, and should generally be taken as a signal that it doesn't want to be a PWA.

@mgiuca why will it not be installable in Chrome? What if I want the browser to show people the install banner to suggest them to put a link to my site on their homescreen, but make it "only" a link that opens in the browser, not an app without browser chrome?

[mgiuca]

@mgiuca why will it not be installable in Chrome? What if I want the browser to show people the install banner to suggest them to put a link to my site on their homescreen, but make it "only" a link that opens in the browser, not an app without browser chrome?

This is a great question. Ultimately, it's a product decision: Chrome doesn't want to promote browser-tab sites for "installation" because we view that install banner as a privilege granted to apps that have put in some effort towards a true app-like experience. However, this is a single browser's individual product decision; another browser could promote all sites with a valid manifest as installable, for example.

So, to go back to the original question, how does a site developer opt out of receiving the PWA treatment? Well, it depends what you're trying to opt out of:

• If you're trying to opt out of being shown in a stand-alone window (you want to always be shown in a browser tab, even when installed), then you should use "display: browser". That's literally what it means.
• If you're trying to opt out of being installable, there's no real way to do this. Even if a site has no service worker or manifest, a user agent can still add UI to install the app to home screen (and of course, many browsers, including Chrome, do have this UI for all sites). This is purely a user agent decision. This is a bit like asking, "how do I opt out of being bookmarkable?" --- you can't, and I don't think we should be providing ways for developers to opt out of such affordances.

Either way, the answer is not to give developers an opt-out mechanism of leaving start_url blank. So changing the default start_url should be fine.

[pkotwicz]

I do not have opinion on which signal we should use to determine whether a site is a PWA
I do have an opinion about the default start_url

I think the default start_url should be the origin of the manifest
If the Web Manifest URL is
https://weather.com/weather/assets/manifest.507fcb498f4e29acfeed7596fe002857.json
The default start_url should be
https://weather.com

Rationale:
When a user chooses to add a site to their home screen, the browser should try to give the user best possible experience. If the Web Manifest does not provide a start_url, in my opinion, "the best possible experience" is a start_url which is least likely to be a 404.

Here's an example Web Manifest in the wild which does not have a start_url (and has display=standalone) where a default start_url of "." would result in a 404.
https://weather.com/weather/assets/manifest.507fcb498f4e29acfeed7596fe002857.json
https://weather.com/weather/assets/ is a 404.

[mgiuca]

@pkotwicz:

I think the default start_url should be the origin of the manifest
Rationale:
When a user chooses to add a site to their home screen, the browser should try to give the user best possible experience. If the Web Manifest does not provide a start_url, in my opinion, "the best possible experience" is a start_url which is least likely to be a 404.

(I'll refer to your proposal as being that the default start_url is "/", as opposed to mine where it is ".".)

Let's cover the most common cases for an app directory hierarchy:

• a) Most common: A start page at the top level domain, with a manifest in the top level directory: e.g., the start page is "https://example.com/" and the manifest is at "https://example.com/manifest.json".
• b) The start page is at the top level, with the manifest in a subdirectory: e.g., start page is "https://example.com/" and the manifest is at "https://example.com/assets/manifest.json".
• c) The entire app is in a subdirectory, with the manifest in that same directory: e.g., start page is "https://example.com/myapp/" and the manifest is at "https://example.com/myapp/manifest.json".
• There are even rarer cases, such as having manifest off-origin, which don't make sense to handle with the default.

So (a) will be covered no matter whether the default is "/" or ".", and that's the most important. A default of "/" covers (b) but not (c). A default of "." covers (c) but not (b).

So I can see arguments for and against "/" and "." being the default. However, I don't think (b) is necessarily more common than (c); I'd guess they're within the same order of magnitude, with (a) an order of magnitude more common than either of them.

I have two main arguments for "." over "/":

1. All explicit URLs in the manifest spec are resolved relative to the manifest URL: scope, start_url, src (of icons and screenshots), src and scope of a serviceworker. It makes sense for the defaults to also be relative to the manifest URL.
2. By having the default be relative to the manifest URL, the app's directory hierarchy is "portable" in that it can be moved between directories. You can pick up the entire directory tree and move it into a sub-directory and things will keep working, which is true of the entire web platform unless path-absolute URLs are used.

I don't think it matters particularly much, but those two reasons cause me to lean towards "." instead of "/".

Finally, I don't think we should be looking at this "probabilistically" (you used the phrase "which is least likely to be a 404"). Since, at least, Chrome does not currently honour a manifest without a start_url, we aren't breaking existing site expectations. Since no existing site will break, it isn't necessary to try and maximize the number of situations in which the default is appropriate (it doesn't matter if 10% of sites can use the default and 90% have to specify it explicitly). Consistency / logic is more important than sheer numbers.

Here's an example Web Manifest in the wild which does not have a start_url (and has display=standalone) where a default start_url of "." would result in a 404.
https://weather.com/weather/assets/manifest.507fcb498f4e29acfeed7596fe002857.json
https://weather.com/weather/assets/ is a 404.

Yep, that's an example of (b). But there are also lots of others of case (c), notably every PWA hosted on github.io, since your app needs to be hosted in a sub-directory (an example I just found: https://gauravchl.github.io/secure-wallet/).

[pkotwicz]

Matt sorry for the earlier confusion. Chrome currently DOES honour a manifest without a start_url.

If a user adds a page with a Web Manifest with no start_url to the homescreen, Chrome will currently add a shortcut to a fullscreen webapp to the homescreen and uses the document URL as the start_url

[dominickng]

@pkotwicz this is something I've wanted to get rid of for a long time. The right to a standalone app treatment should be reserved for PWAs only since devs have gone to the extra effort there to make a good experience.

[mgiuca]

Yeah, in the non-PWA case, we use the document URL, but we also do that when there is no manifest whatsoever.

I think we can consider that to be just ignoring the manifest, rather than honouring the current language that says to use the document URL. Note that the manifest spec says: "The start_url member is purely advisory, and a user agent MAY ignore it or provide the end-user the choice not to make use of it." So even if we deleted the language around the document URL being the default start URL, Chrome can still use the document URL in the non-PWA case.

[marcoscaceres]

I'm happy to go with whatever @dominickng and @mgiuca come up with here (and removing the MAY text around start url).

[mgiuca]

I think the MAY text is helpful actually. It means we can decide to override the developer's wishes (e.g., in the case where the site is not a PWA, and we deliberately want to disrespect the manifest). However, having a specified default behaviour is useful.

[pkotwicz]

If the client can ignore the manifest as you mentioned in a previous comment, I am ok with using "." as a default start URL.

I am concerned a bit that a user will add a Web Manifest to their site for a reason other than the "add-to-homescreen" behavior. (and would not have tested add-to-homescreen behavior). It feels like a bug to allow a user to use the browser to add a website to the homescreen if the start_url points to a 404 page.

[dominickng]

@pkotwicz this can already happen today - developers can explicitly specify a start_url that 404s.

If a dev is adding a web manifest for reasons other than add to homescreen, they probably won't have met the other requirements for being a PWA, so we would ignore the start_url anyway.

[mgiuca]

I think the above comments by @marcoscaceres @dominickng and @pkotwicz represent sufficient consensus around the issue.

I will land the change as-is on Monday (unless there is an objection here). This will be a breaking change, of sorts. Specifically:

• Technically, this changes the start URL for all manifests missing a start_url member.
• User agents following the current spec will still technically be compliant with the new spec, under the clause in §7.3: "Please note that the start URL is not necessarily the value of the start_url member: the user or user agent could have changed it when the application was added to home-screen or otherwise bookmarked." They'll essentially be user agents that override the start_url member with custom logic, which is allowed by the spec.
• User agents that do update to the new behaviour are not required to migrate existing app shortcuts, per the above clause.
• Google Chrome currently does not allow installation if start_url is missing, so any site that wants to be installed should already have an explicit start_url. (This won't break any sites that previously worked, at least in Chrome.)
  • Having said that, Chrome will want to update its implementation to allow manifests without a start_url to be installable, under the new rules.
• Potential compat risk if Chrome updates to respect the new default but other browsers do not. Developers may rely on this behaviour in Chrome. User agents ideally would not distinguish between an explicit "start_url": "." and a missing start_url.

I have added a new (temporary) non-normative note to the spec in the latest commit:

"This default was changed in June 2018. Manifest authors might wish to avoid relying on the default behaviour (and explicitly specify this member), as user agents might still respect the previous default (the document URL), or ignore the manifest entirely if start_url is missing."

[dominickng]

Discussed offline: this is broken for the case where you have a manifest with no start_url hosted on originA, which is a different origin to your document URL (originB). In that case, the start_url will be set to something on originA, which is still a different origin to the document URL.

We need to check the default value when creating it to make sure it is same-origin with document URL.

[mgiuca]

This is correct, and quite annoying as it seems we can't (yet) detach our dependency on the document URL.

I've added a new step which falls back to document URL as the default, if manifest is not same-origin as the document. This will be cleaned up once we fully detach the document URL.

[boyofgreen]

Hello Folks,
Kirupa and I looked over this and compared it to our current patterns and think this is a good change. From a Microsoft implementation, there are some edge cases when the document domain is actually the app package name, not the site URL. For that reason, when no start_url is present, or if it is relative to manifest, we use the base URL of the domain of the site the manifests resides. If I understand the proposal properly, this will align with our current approach.

[mgiuca]

Hi Jeff,

When you say the "app package name", do you mean that's the unique identifier for the app? This spec doesn't really have an opinion about what is the "unique key" for the app (though maybe it should). This proposal is more about just what is the start_url (i.e., what page opens when you launch the app).

Aside: I think the best thing to use as a "unique key" is the manifest URL itself, not the start URL or origin. While it's unlikely that two apps will share the same start URL, they can. And it's certainly possible for two apps to have the same origin (e.g., all apps for a single user under github.io). I don't think you should be using the origin when the start_url is relative to the manifest.

I guess pertinent to this PR, what is the default page that you open for sites that have no start_url in their manifest? Is it just the top-level "/" page of the site? Note that this proposal is changing it to be the parent directory of the manifest, which doesn't sound exactly like what you do. Happy to work out a different default, but the proposed one makes the most sense since all URLs in the Manifest spec are resolved relative to the Manifest URL.

[boyofgreen]

Matt, We are on the same page. Our package name is basically a generated GUID that the dev has no control over, so we don't think it's a good use case either. In our case, if there is no start URL, we do assume the top level "/" of the site to which the manifest is served from. I think that is aligned unless I'm missing something.

[mgiuca]

@boyofgreen:

In our case, if there is no start URL, we do assume the top level "/" of the site to which the manifest is served from. I think that is aligned unless I'm missing something.

Sorry, but that's not what we're proposing here. This proposal makes the default start URL the directory containing the manifest. So if a site has its manifest URL at https://example.com/myapp/manifest.json with a missing start_url member:

• The current proposal makes it so that the start URL is https://example.com/myapp/.
• It sounds like Edge currently sets the start URL to https://example.com/.
• Note that Chrome currently considers a missing start_url an error.

So we aren't aligned. Do you think you could change Edge behaviour to pick the directory containing the manifest in the future? Or should we change the proposal here?

Note that there was a discussion about "." vs "/" above. I gave some arguments for why we prefer "." in this comment, summarized as "URLs on the Web are always relative by default", so I would prefer ".", but if we have to change to "/" for compat with Edge, we can do so. Please let me know what you'd like to do.

[dominickng]

@boyofgreen can you follow up on Matt's post here? Otherwise I'd like to propose landing this change.

[aarongustafson]

@mgiuca This is getting stale. Do you think we can land it soon?

[philloooo]

If the document URL is not given, and the start_url has a different origin from the manfiest_url, should we require/recommend the parser of the manifest to verify that the start_url's origin acknowledge this manifest_url?

[mgiuca]

@mgiuca This is getting stale. Do you think we can land it soon?

Sorry @aarongustafson ; this has really slipped off my radar, to the point that I just don't have context on it (I'm not working anywhere near this area). I've essentially handed off the responsibility for this to @philloooo and @dmurph so I think they can take it and land it.

Apologies for the churn / confusion.

[marcoscaceres]

Ok, conflict we have here now is that both Safari and Firefox derive the start URL from document URL.

I'm inclined to keep with the status quo.

Did Chrome ever ship this?

[mgiuca]

Like my previous comment said, I haven't been keeping up with this space and @philloooo and @dmurph should weigh in on it.

Ok, conflict we have here now is that both Safari and Firefox derive the start URL from document URL.

And Chrome.

This change is not about changing Safari and Firefox to be aligned with Chrome. This is about changing Safari, Firefox and Chrome to have reliable behaviour.

From memory, the problem is that document URL isn't stable. It means that if you have a manifest without an explicit start_url, then you essentially don't know what the start URL is going to be when the app is installed, because it will depend on which page the user was on when they clicked the install button. You don't really have an app... you just have an ability to install whatever web page you happen to be on at the time.

My argument is that if the spec (basically) says "if start_url is omitted, then the site has no control over the installed app's start URL", then there is no contract established between the site and the user agent here. The site cannot expect any particular URL is the start URL, therefore, if the user agent changes the default, it isn't violating any expectation or contract. Therefore, it's safe to change the spec from "no guarantees" to "this is the behaviour", as long as all vendors are willing to change.

[marcoscaceres]

I haven't been keeping up with this space and @philloooo and @dmurph should weigh in on it.

No problem. I'm happy to work with @philloooo and @dmurph on it... just following up on the id idea (#988) which relies on start_url, which brought me back here.

This change is not about changing Safari and Firefox to be aligned with Chrome. This is about changing Safari, Firefox and Chrome to have reliable behaviour.

At this point, I'm mostly just specifying what's shipping. It's not ideal, but I'm not optimistic about changes to engines.

From memory, the problem is that document URL isn't stable. It means that if you have a manifest without an explicit start_url, then you essentially don't know what the start URL is going to be when the app is installed, because it will depend on which page the user was on when they clicked the install button. You don't really have an app... you just have an ability to install whatever web page you happen to be on at the time.

Correct.

My argument is that if the spec (basically) says "if start_url is omitted, then the site has no control over the installed app's start URL", then there is no contract established between the site and the user agent here. The site cannot expect any particular URL is the start URL, therefore, if the user agent changes the default, it isn't violating any expectation or contract. Therefore, it's safe to change the spec from "no guarantees" to "this is the behaviour", as long as all vendors are willing to change.

That's mostly true, but if the site owner wants start_url stability they can add the explicit start_url - or just add "start_url": "." to get the same result as here. I'm thinking that's good enough for now in capturing cross browser behavior.

[alancutter]

just following up on the id idea (#988) which relies on start_url, which brought me back here.

If id relies on the parsed value of start_url then it's important to ensure it is well defined otherwise the value of an app's identity isn't well defined.