Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

bug 1331643, 1330357: Avoid duplicate tags with TAGGIT_CASE_INSENSITIVE=True #4103

Merged
merged 5 commits into from Feb 7, 2017

Conversation

jwhitlock
Copy link
Contributor

@jwhitlock jwhitlock commented Jan 20, 2017

With bug 1293749 fixed (Add missing UNIQUE KEY indexes in production), some new issues have appeared:

  • bug 1330357 - ISE if profile interest tag is not lowercase in the database
  • bug 1331643 - ISE when adding two document tags that only differ in capitalization

Both appear to be caused by an issue with django-taggit and MySQL with a case-insensitive collation and a unique index. Adding two "case similar" tags, like "WEB" and "web", will result in attempting to add both, and raise an IntegrityError.

The issue is handled in this PR by querying the database for existing tags with "case similar" names, and creating new ones as needed, and then passing the unique set of Tag instances to taggit to create the many-to-many relations, rather than passing a list of names.

Additionally, user profile interests and expertise were restricted to lower-case tags to avoid triggering the creation of duplicate tags. This removes the restriction and UI changes, so that non-lowercase interests, like "HTML" and "CSS", can be added. This change allows mixed-case profile tags, but existing tags will need to be updated in the database as desired.

This is an alternate for PR #4100 that uses TAGGIT_CASE_INSENSITIVE=True, and also avoids django-taggit issue jazzband/django-taggit#460.

@jwhitlock
Copy link
Contributor Author

I've opened PR jazzband/django-taggit#461 to fix the issue upstream, and will apply the fixes to this branch with the assumption that a future version of django-taggit will fix the issue.

@codecov-io
Copy link

codecov-io commented Jan 20, 2017

Current coverage is 86.20% (diff: 100%)

Merging #4103 into master will decrease coverage by <.01%

@@             master      #4103   diff @@
==========================================
  Files           143        143          
  Lines          8887       8886     -1   
  Methods           0          0          
  Messages          0          0          
  Branches       1192       1192          
==========================================
- Hits           7661       7660     -1   
  Misses          989        989          
  Partials        237        237          

Powered by Codecov. Last update 359f55b...1d0f303

@safwanrahman
Copy link
Contributor

Need to update django-taggit==0.21.5 as the fix has been merged in the version
https://github.com/alex/django-taggit/releases

django-taggit==0.18.0 \
--hash=sha256:7010a1f7597d954aaf8b8174563f2e3455520db949d9c5d360dd9e4f906dd32a \
--hash=sha256:4ddd61928a4fc6ba7ccb1145d8d3a6d5c2829de8d8738eea49924ade8969f0ce
django-taggit==0.21.3 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with @jwhitlock patch, 0.21.5 has been released

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and now released

@jwhitlock
Copy link
Contributor Author

Unfortunately, I introduced a bug in django-taggit 0.21.5. I've opened an issue (jazzband/django-taggit#463) and a PR (jazzband/django-taggit#464) to fix it. Meanwhile, the manual tag creation continues to bypass the buggy code, so leaving it in.

I'd like to merge and deploy this PR, fixing the ISE bugs, and follow up with a new PR when django-taggit 0.21.6 is released.

@safwanrahman
Copy link
Contributor

Its better to use your fork until the patch is merged rather than using custom code for manual tag creating

@frewsxcv
Copy link

Upstream patch is merged and published

Tell django-taggit to use case insensitive lookups for existing tags.
We're using MySQL with a case insensitive collation, so this is already
done at the database level, but this may help the code work correctly
with the database.
Add tests for bug 1331643, where Revisions with existing "duplicate"
tags (equal after adjusting for case insensitivity) set the Document
tags.
Test bug 1330357, where Kuma code lowercases profile tags and tries to
add a tag that collides with the mixed case tag.
django-taggit 0.18.0->0.21.5: Many changes, including:
* Deprecate old and add new Django version support
* Implement m2m_changed signal
* Improvements for use in Django admin
* Fix case-insensitive duplicates when creating tags

This fixes the xfailed tests.
@jwhitlock
Copy link
Contributor Author

jwhitlock commented Jan 26, 2017

@frewsxcv thanks for the quick merge and publish!

Rebased on current master, removed manual tag creation, crossing fingers and waiting for TravisCI 🤞

Update: Tests pass!

Copy link
Contributor

@escattone escattone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I've verified (locally) that this fixes:

@escattone escattone merged commit 0f972a3 into mdn:master Feb 7, 2017
@jwhitlock jwhitlock deleted the taggit_ci_1331643 branch April 4, 2017 02:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants