Hashtags parser fixes and improvements

[leio10]

🎩 What? Why?

When creating a proposal, the hashtag #acción_mutante is parsed as #acci and the rest of the characters are left as plain text. When looking for fixing this, I've found other aspects of the code that I tried to improve:

1. As said, HASHTAG_REGEX was not taking unicode characters into account. Also, it was defined in the model, while it is mainly related to the parser.
2. There was one find_or_create_by executed for each hashtag detected. I've replaced with a query for every hashtag found in the content, and a creation for each non-existing hashtag.
3. I added some test for additional scenarios: hashtag creation, repeated tags in the content, unicode chars and non-hashtag characters
4. Code comments was copy&pasted from the UserParser class, I've updated them.

I think that, at least, improvements 1 and 2 could be applied to the UserParser class, I can add them in this PR if you agree with these improvements.

📌 Related Issues

📋 Subtasks

• Add CHANGELOG entry

📷 Screenshots (optional)

[mrcasals]

Wohooo! Looks real good @leio10, thanks!! Can you add a CHANGELOG entry? I'd add it under Fixed, saying that hashtags with unicode characters are now parsed correctly.

I think it'd be awesome if improvements 1 & 2 were applied to the UserParser too, what do you think @decidim/lot-mods?

[leio10]

@mrcasals great!

By the way, I was researching a little and found that Decidim allows unicode chars on usernames (I have a username @león now), while it seems that other services like Twitter don't (I didn't tested it, but I couldn't find a username like that after performing a random search). Maybe we should evaluate to stop allowing it? I think that usernames should be as simple and possible, allowing unicode we open the door to impersonation attacks as in domain names.

UPDATE: with this message we can confirm that github doesn't allow unicode in usernames, I've changed it to avoid the mention to the user @le. 😄

[mrcasals]

Damn, I guess you're right... 😞

[leio10]

@mrcasals Done, I've improved only the users query, without touching the unicode issue for users, as I understand that we don't want to allow unicode on usernames. Also, I've added the changelog entry.

Also, I've remembered another improvement I made to both parsers: I changed the boundaries for the regular expression. Instead of searching for the line starting or an space, I search for any word boundary. This allow to detect hashtags and users in contents like "blablabla (#hashtag) and:@user" that weren't detected before.