1.12.0 vs 1.11.2 selectors with # broken #2824

Closed
cosmicnet opened this Issue Jan 14, 2016 · 48 comments

Projects

None yet
@cosmicnet

The following is a perfectly valid selector under 1.11.2:
$('.class a[href=#anchor]');

But fails under 1.12.0, and requires quotes:
$('.class a[href="#anchor"]');

Likely related to:
41f522a

@dmethvin
Member

Yes, but it was never a valid standard CSS selector, anything with special characters or spaces has to be quoted in the selector. Accepting it in previous versions was a bug.

Attribute values must be CSS identifiers or strings. -- https://www.w3.org/TR/css3-selectors/#attribute-selectors

@dmethvin dmethvin closed this Jan 14, 2016
@cosmicnet

Whether you regard this as a previous bug or not, it is a breaking change.

@timmywil
Member

Fixing a bug almost always breaks something.

@cosmicnet

The release notes specify this is a non-breaking change. But you are changing the way the selector syntax works and has worked for a long time. There will be a LOT of code in the wild that uses this construct.

Your own test suite previous proved the accepted # syntax:
41f522a
You had to update it to work with the new syntax. Yet nowhere do you indicate that the selector syntax has now changed. Where is the issue for the bug? Where is the release note stating the change in behaviour?

@mgol
Member
mgol commented Jan 18, 2016

@cosmicnet Every behavior change may break code that relies on unspecified behavior; if we treated all of them as breaking changes we couldn't have non-major releases at all. Semver only applies to what's documented, i.e. what's on https://api.jquery.com/. We've never promised there that such a selector works, hence by definition changing that is not a breaking change. We did think that it might break some incorrect code, though and that's why we didn't introduce this change in a patch release but in a minor one, even though semver permits us to do it in a patch release.

@dmethvin
Member

@cosmicnet specifically what do you think we need to do here?

Are you advocating that we revert this change? I don't think we can do that. We are trying to parse valid CSS selectors per the spec and allowing these kind of exceptions (even undocumented) makes that difficult if not impossible. It also causes the selector to be rejected by the querySelectorAll path which results in performance degradation.

If you're just saying you're frustrated that this was a breaking change to your code, we're sorry for that. There were several other undocumented features that we did mention in the release notes, it's just hard to know which ones the world may be depending upon. We're always glad to take feedback on our WIP builds, note that this was committed long ago and using the WIP build would have revealed the compatibility issue.

@cosmicnet

@dmethvin The fact that your own test suite was proving the "unspecified behaviour" indicates to me just how common this usage will be, I also see that as documenting the feature. After all, the test suite should be the definitive definition of what is valid and what is not. If it's proving "undocumented features" that are erroneous then that indicates a serious problem with the tests.

I fail to see how not mentioning it at all in the release notes is anything other than an omission. Instead, at the very least a note with the explanation you have given would be far more appropriate.

@dmethvin
Member

@cosmicnet specifically what do you think we need to do here?

@cosmicnet

@dmethvin I fail to see how not mentioning it at all in the release notes is anything other than an omission. Instead, at the very least a note with the explanation you have given would be far more appropriate.

@dmethvin
Member

Thanks. I agree. All we can do is try to track these better in the future.

@drbyte drbyte added a commit to drbyte/zencart that referenced this issue Feb 8, 2016
@drbyte drbyte Fix for compatibility with jQuery 1.12.0 924dfdd
@ericandrewlewis ericandrewlewis referenced this issue in jquery/api.jquery.com Apr 13, 2016
Closed

Attribute Equals Selector docs may be vague #910

@aaronjorbin aaronjorbin added a commit to aaronjorbin/api.jquery.com that referenced this issue Apr 14, 2016
@aaronjorbin aaronjorbin Link to W3C for unquoted single word
Unquoted single word has a specific definition in this case that is not
succinct. A link to the spec helps developers understand what is meant.

fixes gh-910
ref jquery/jquery#2824
6468c15
@danieliser
danieliser commented Apr 18, 2016 edited

@dmethvin I think this needs a serious revisit. Its too late now, but in the future this is really relevant to how you make decisions.

WordPress included this in v4.5 and broke dozens of themes if not hundreds of really popular themes which means this bug is now in the wild on a very large scale.

Problem is that my products are unrelated and have no issues at all but are reliant on our users theme not throwing stupid errors.

So no my entire support queue is full due to problems I can't do anything about. Seems like the simple solution would have been to simply make it still recognize the older ones and throw a silent error while still doing what was expected originally. Deprecate it, document it, revist it later to remove it entirely.

Progress in the name of progress, is anything but.

@mgol
Member
mgol commented Apr 18, 2016

When WordPress updates a jQuery version it may always break themes,
especially if it's a major upgrade of jQuery. This wasn't a major upgrade
but previously we weren't following semver and minor upgrades did have
breaking changes from time to time, e.g. in 1.9 there were many of them. I'm
sure it broke a lot of code back then as well. This is unavoidable,
otherwise we couldn't make any changes in the library.

This is completely normal if after a new jQuery version is released some
plugins/themes don't work with it. They should adjust to the changes over
time.

Michał Gołębiowski

@markelog
Member
markelog commented Apr 18, 2016 edited

Totally agree with @mgol, but i think we should discuss this on today meeting anyway.

@markelog markelog reopened this Apr 18, 2016
@dmethvin
Member

To put all the discussion in one place, I agree with @danieliser that it is too late to change at this point. The original change was made to master last year and it was shipped in early January. The reason it has resurrected this week is because Wordpress did an update, some Wordpress themes were impacted, and there appears to be no auto-update for themes. Had any of this come to light last year or even right after the 1.12.0 release we could have done something different, but let's not revisit this change now. Better to have people update their themes.

The going-forward question is how Wordpress will ever update to jQuery 3.0. Right now WP includes both 1.12.x and Migrate 1.x in order to let themes be sloppy. If they were to upgrade to jQuery 3.0 the themes would need to be able to run with jQuery 1.12 without Migrate before a jQuery3+Migrate3 combination would allow them to work. Given all the uproar about changing an invalid undocumented thing like unquoted attributes I worry how something more substantial will fare.

Seems like the main problem is that Wordpress themes are either not being updated by the users or no theme update is available, either of which makes a jQuery update risky. Not sure how to deal with that other than never updating jQuery.

@timmywil
Member

Could they just stop auto-updating jQuery and let theme builders decide which version of jQuery to use?

@markelog
Member
markelog commented Apr 18, 2016 edited

There is always a revert and release this thing in major, not saying that is what we should do, just saying this is an option

@FagnerMartinsBrack
FagnerMartinsBrack commented Apr 18, 2016 edited

Just my 2c here:

It seems the main problem is that jQuery throws an error that, if not caught, makes all scripts in the page stop working.

jQuery has historically been treating wrong inputs as noop, returning an empty list instead of throwing. That is assumed by the consumers as an implicit behavior inherent from jQuery philosophy. Therefore not many people bothered in surrounding anything in a throw/catch clause. A minor update is not expected to break everything. For that reason, any fix that changes the behavior from noop to throw an Error will have a higher impact than a fix that breaks a behavior but doesn't stops all scripts in the page.

My suggestion in the future to prevent this kind of problem is to treat any behavior change from noop to Error as a major breaking change worth changelog, even if it is not undocumented. This way there will be no unintended breakage.

I hope it makes sense.

@markelog
Member
markelog commented Apr 18, 2016 edited

I hope it makes sense.

It makes a lot of sense to me.

/cc @gibson042

@dmethvin
Member

My suggestion in the future to prevent this kind of problem is to treat any behavior change from noop to Error as a major breaking change worth changelog, even if it is not undocumented. This way there will be no unintended breakage.

Although I agree something should have been in the changelog, I'm not sure it would have changed anything for this case. The thing we did not know at the time was that it seems several popular Wordpress themes used the feature. Due to the way jQuery gets updated on many/most sites (via a WP update and not an explicit decision to update jQuery by the site owner) we hadn't gotten much feedback about the scale of the problem until last week's WP update.

There is always a revert and release this thing in major, not saying that is what we should do, just saying this is an option

Major is a great place for a breaking change if this is considered one, and as I mentioned it's not clear WP has a path to jQuery 3.0 right now anyway. If they do, this will be one of the smaller problems.

Could they just stop auto-updating jQuery and let theme builders decide which version of jQuery to use?

It's trickier than that as I understand. Wordpress by default always puts in a relatively recent jQuery+Migrate, and updates these as part of its own auto-update cycle. Some themes and plugins outside the theme use this one, since it's there. Others are more paranoid and load their own via noConflict, which of course gets wasteful since the page is loading multiple copies. However in my experience many if not most sites do the latter, for example http://dominiqueansel.com/ loads both 1.8.2 and 1.11.2 with Migrate 1.2.1. That allows it to run Thickbox from 2007. 😭

@markelog
Member
markelog commented Apr 18, 2016 edited

Although I agree something should have been in the changelog, I'm not sure it would have changed anything for this case.

I think idea here is to consider any change that starts to throw as breaking change.

Major is a great place for a breaking change if this is considered one

If we consider this as breaking and release another 1.x with proposed revert, then wordpress users would deal with this issue only at jQuery major update not at patch version

@gibson042
Member

treat any behavior change from noop to Error as a major breaking change worth changelog, even if it is not undocumented

That is in explicit contradiction with our API design guidelines:

Undocumented inputs result in unpredictable output. - The API may throw an error inside jQuery, do nothing, or have some unpredictable behavior. This may change without warning across versions, even minor patches.

I see merits to the proposal, but am very reluctant to switch.

@markelog
Member

switch

Yeah, that is what proposed here, that is why i summoned you :)

@dmethvin
Member

I think idea here is to consider any change that starts to throw as breaking change.

Ah, okay understood although that means our hands are essentially tied to avoid any invalid input from becoming a thrown error. That's pretty restrictive. We'd need to check the effects with inputs of null, undefined, window, document.

If we consider this as breaking change and release another 1.x with proposed revert, then wordpress users would deal with this issue only at jQuery major update not at patch version.

I don't think a switch back after 4 months is a good idea.

@markelog
Member
markelog commented Apr 18, 2016 edited

That's pretty restrictive.

Indeed, but it seems that is the only action that could be taken that could protect us from such things.

I don't think a switch back after 4 months is a good idea.

Just to reiterate - not saying we should or shouldn't do it, just saying it is an option. But to quote you -

we hadn't gotten much feedback about the scale of the problem until last week's WP update.

Now we do, before we didn't, so in any case, i think we should do something :), to prevent such things to happen in the future

@dmethvin
Member

One other solution to this specific case is to add more logic to Migrate, since Wordpress includes it. For example if we find the [name=#...] pattern we first try qSA to see if it's valid and if that throws we try a string replace to add quotes. That does break in some cases but I think they are only pathological and it's a question whether the rare pathological case should outweigh the benefit of fixing the common case.

@markelog
Member

One other solution to this...

Didn't we try this before and failed? In any case, wordpress breakage revealed the consequences of this change, jquery-migrate might help for wordpress case, but not for the rest of them.

@timmywil
Member

but not for the rest of them.

Those cases can include migrate and change their code themselves. I like the migrate idea to fix wordpress. I don't think we should revert.

@markelog
Member

So yeah, meeting is in 40 minutes we can continue there i think

@dmethvin
Member

Right now Migrate only warns on naked hashes because there are cases like a[href="[dog=#bark]"] where the regexp would incorrectly update the [dog=#bark] part and actually break a valid-but-bizarre selector. Running through qSA first would fix that particular one although a[href="[dog=#bark]"]:first would still be mangled by the regexp.

@danieliser

@dmethvin - Thanks for revisiting the issue. You are pretty close to spot on the last part about themes loading extra jQuery.

@FagnerMartinsBrack - Well said.

@timmywil - You are partially correct, they can do that. But there are 2 ways to do it, WP uses an asset queing system (see wp_enqueue_script). It allows for dynamic inclusion of scripts from themes and plugins including dependency management. IE my plugin uses jQuery so I can tell WP to load it before mine.

So if a theme needs an older version they are either replacing it for every script that needed jquery. The alternative would be them enqueing a custom version of jquery separately and using noConflct.

@gibson042 - That may be a bit presumptuous. WP Core philosophies are a bit different than yours of course, but when 25%+ of the internet runs your software in an extendable way like it is, your hands are bound as well. Their are breaking changes quite regularly, but they are well documented, get distributed directly to developers via many channels well before release etc.

@markelog, @dmethvin - I wouldn't start backtracking, that wasn't the point of my initial comment. But had this been well documented the WP Core team would have made sure to both distribute the changes and info to all, but their automated theme/plugin scanners in the wordpress.org repository would have also been able to flag themes using it. Before each release of WP an email goes to every dev telling them to check their plugins against a list of issues, themes are automatically scanned (hopefully plugins soon too).

I am glad to see the conversation moving, but I would suggest that since WP uses jQuery as a core dependency and a large percentage rely on that it may behoove you and the WP Core team to work out a direct communication channel to discuss these things in detail. I'm sure they would be happy to discuss both a path to v3 as well as push authors to update their themes ahead of time.

@timmywil
Member

@danielliser Thank you for the responses. You're right that we need better communication with the WP team. Perhaps an irc channel.

@danieliser

@timmywil WP has a Slack group where nearly all core communication goes on. May be an option to just make a new channel there if you guys do slack. In any case I will tell you as both a dev in the WP community and a WordCamp organizer that the WP community is very open and love collaboration so I think this would be a win for both WP & jQuery.

@timmywil
Member

I'm also on Slack, but the jQuery team is not. Maybe I could squat in the WP team.

@danieliser

@timmywil If nothing else you will be able to use it to get in touch with the right people and find another communication channel for you all.

WP Slack: https://make.wordpress.org/chat/
Make WP: https://make.wordpress.org/ - Describes the different core groups and their slack channel info is provided as well as scheduled meeting times etc.
Make WP Core: https://make.wordpress.org/core/ - Most likely channel/group to talk to. There is scheduled meeting info and member info here you can use to get in touch.

@dmethvin dmethvin referenced this issue in jquery/jquery-migrate Apr 18, 2016
Closed

Try to fix the [href=something#hashed] case #174

@markelog
Member

Okay, so we discuss this in length at the meeting, we decide to do the following -

  1. We not gonna revert
  2. We will try to mitigate this in jquery-migrate, since wordpress includes it, it should help
  3. We changed our policy guideline to prevent such changes happening in patch/minor versions on case-by-case bases
  4. We will discuss this with wordpress team

Thank you for your thoughts everyone!

@markelog markelog closed this Apr 18, 2016
@markelog markelog removed the Needs review label Apr 18, 2016
@danieliser

@markelog - Thanks guys, I think that will go a long way. I am going to discuss with some fellow WP devs the possibility of both automated compatibility checks for plugins and themes as well as the possibility to do version-ed loading of jQuery in the future, allowing plugins & themes to load a select version if required with its own unique namespace. I think with those 2 their could be a path to jQuery 3 over a few years.

@AurelioDeRosa AurelioDeRosa added a commit to jquery/api.jquery.com that referenced this issue Apr 19, 2016
@aaronjorbin @AurelioDeRosa aaronjorbin + AurelioDeRosa Link to W3C for unquoted single word
Unquoted single word has a specific definition in this case that is not
succinct. A link to the spec helps developers understand what is meant.

Fixes gh-910
Ref jquery/jquery#2824
Closes gh-911
8c1b607
@aaronjorbin

RE: WordPress jQuery communication.
I'm one of the WordPress core committers. Happy to help encourage communication in any way I can. I'm now idling in both the jquery and jquery-dev rooms as jorbin. Feel free to contact me if you ever need anything from WordPress. Y'all are also welcome to come hang out in our slack.

@danieliser

@aaronjorbin - Thanks for jumping in here. I think this is a strategic partnership for both WP & jQuery.

@scottgonzalez
Member

FWIW, I've been in the WordPress Slack channel since it was first created. There are multiple WordPress committers who ping me when things come up related to jQuery projects. I'm not sure how this helps with a change like this though, unless @timmywil is going to inform WordPress devs of every change.

One part of the solution is being available. This is easy. And @aaronjorbin just did that for the WP -> jQuery side.

Another part is making your presence known. This is not so easy. The people involved in this thread now know that @aaronjorbin is available to ping in our channels if needed. However, others do not know. And over time, some people in this thread will forget.

The bigger part is being active. As @aaronjorbin said, "I'm now idling in both the jquery and jquery-dev rooms as jorbin." But idling is not the same as paying attention to everything that's going on. I'm not saying that this is bad, it's exactly what I've been doing for the jQuery -> WP side for years (going back to before Slack). But without the active part of this, I'm not sure how issues like this will be avoided.

I'm all for increasing communication even more. Certainly the more people we have on each side that are idling in various channels, the more people will know to ping them at the right times. But I'm interested to hear what the plans are for how this type of problem will be avoided in the future.

I used to take a much more active role in this across several projects, but it's been a few years since I switched to a passive role. This page is now very out of date, but we used to track this information on the jQuery UI wiki: http://wiki.jqueryui.com/w/page/54364943/Project-Collaboration though we never published individual names for the people we worked with.

@markelog
Member
markelog commented Apr 21, 2016 edited

@scottgonzalez

But I'm interested to hear what the plans are for how this type of problem will be avoided in the future

Improving communication between projects would definitely help (personally, i'm aware of our connection with the wordpress team), but wordpress wasn't exclusively affected by this problem, we received issues about this before.

Scale of it, however, was revealed only in this ticket, that is why it was re-open and added back to the agenda of the weekly meeting.

In monday, we had extensive discussion on not letting these kind of issues rise again in patch and minor versions, which resulted in guideline changes.

So from now on, we will be more careful with these kind of changes and even though they will still be evaluated on case by case basis (as guideline says), likelihood of such event should be very low. Not only for the wordpress themes though but for all users.

Maybe @dmethvin or @timmywil have more info about this.

@scottgonzalez
Member

That sounds great @markelog. Does this also mean that changes like this, even though the feature was undocumented, will be listed in the change log for major releases?

@markelog
Member

Yeah, see definition on https://github.com/jquery/jquery/wiki/API-design-guidelines under "Undocumented inputs result in unpredictable output."

@scottgonzalez
Member

Perfect. Thanks.

@danieliser

@markelog Thanks for the special attention you guys have shown here. I think even a subtle guideline change like that could make all the difference in terms of keeping scale of issues down.

Next step would be to get the WP Make: Meta team involved to update the automated scanners to look for these types of things and notify plugin & theme authors automatically.

@dmethvin
Member
dmethvin commented May 2, 2016

There is a patch going into the next jQuery Migrate (both 1.x and 3.x) to detect and fix most of these cases. There are still situations where the selector is too complex for the regexp to fix it, but it seems to handle the cases that have been reported in the past months. jquery/jquery-migrate#184

@dougbeal dougbeal added a commit to foolscapcon/affix-sidebar-bootstrap-sass that referenced this issue May 9, 2016
@dougbeal dougbeal jquery - must escape # cd7f170
@dmethvin
Member

jQuery Migrate 1.4.1 has just been released, it fixes the problem with unquoted selectors in the cases that have been reported.

http://blog.jquery.com/2016/05/19/jquery-migrate-1-4-1-released-and-the-path-to-jquery-3-0/

@art-mickiewicz art-mickiewicz added a commit to art-mickiewicz/django-suit that referenced this issue Nov 17, 2016
@art-mickiewicz art-mickiewicz Update suit.js
Fixes tab activation with invalid jquery selector # emerged on the updated jquery.
Related jquery issue: jquery/jquery#2824
b90a739
@art-mickiewicz art-mickiewicz referenced this issue in darklow/django-suit Nov 17, 2016
Merged

Update suit.js #554

@alijamal14

Salaamun Alekum Can Anyone Tell me What Can I Write Alternate To This

linkElement: '#primary-menu ul li a:not([target="_blank"]):not([href*=#]):not([data-lightbox])',

@dmethvin
Member

@alijamal14 please ask for help on StackOverflow

@dmethvin dmethvin locked and limited conversation to collaborators Dec 17, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.