unicode emoji in repo description results in server error

[markuman]

• Gitea version (or commit ref): 1.12.5
• Git version:
• Operating system: Docker Container
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
  • Mariadb
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
  • Not relevant
• Log gist:

Description

When using emoji in the repository description, gitea throws an error (500).

Oct 14 07:26:31 osuv gitea[803]: #033[36m2020/10/14 05:26:31 #033[0m#033[32m...ters/repo/setting.go:117:#033[32mSettingsPost()#033[0m #033[1;31m[E]#033[0m #033[1mUpdateRepository#033[0m: #033[1mupdateRepository: update: Error 1366: Incorrect string value: '\xF0\x9F\x9A\x80' for column `gitea`.`repository`.`description` at row 1#033[0m
Oct 14 07:26:31 osuv gitea[803]: [Macaron] 2020-10-14 05:26:31: Completed POST /m/nextcloud_collection/settings 500 Internal Server Error in 18.947038ms

Screenshots

[Masgalor]

Your database uses the wrong charset, "utf8" instead of "utf8mb4".

You probably didnt follow the docs when preparing your database, those clearly state that "utf8mb4" is needed for emoji support.
https://docs.gitea.io/en-us/database-prep/#mysql

[markuman]

@Masgalor

Of course my database is utf8mb4

MariaDB [(none)]> show create database gitea;
+----------+-------------------------------------------------------------------+
| Database | Create Database                                                   |
+----------+-------------------------------------------------------------------+
| gitea    | CREATE DATABASE `gitea` /*!40100 DEFAULT CHARACTER SET utf8mb4 */ |
+----------+-------------------------------------------------------------------+
1 row in set (0.001 sec)

[markuman]

And the repository table

+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table      | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| repository | CREATE TABLE `repository` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `owner_id` bigint(20) DEFAULT NULL,
  `lower_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `description` mediumtext COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `website` varchar(2048) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `default_branch` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `num_watches` int(11) DEFAULT NULL,
  `num_stars` int(11) DEFAULT NULL,
  `num_forks` int(11) DEFAULT NULL,
  `num_issues` int(11) DEFAULT NULL,
  `num_closed_issues` int(11) DEFAULT NULL,
  `num_pulls` int(11) DEFAULT NULL,
  `num_closed_pulls` int(11) DEFAULT NULL,
  `num_milestones` int(11) NOT NULL DEFAULT 0,
  `num_closed_milestones` int(11) NOT NULL DEFAULT 0,
  `is_private` tinyint(1) DEFAULT NULL,
  `is_mirror` tinyint(1) DEFAULT NULL,
  `is_fork` tinyint(1) NOT NULL DEFAULT 0,
  `fork_id` bigint(20) DEFAULT NULL,
  `size` bigint(20) NOT NULL DEFAULT 0,
  `is_fsck_enabled` tinyint(1) NOT NULL DEFAULT 1,
  `topics` mediumtext COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `created_unix` bigint(20) DEFAULT NULL,
  `updated_unix` bigint(20) DEFAULT NULL,
  `is_empty` tinyint(1) DEFAULT NULL,
  `close_issues_via_commit_in_any_branch` tinyint(1) NOT NULL DEFAULT 0,
  `is_archived` tinyint(1) DEFAULT NULL,
  `avatar` varchar(64) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `original_url` varchar(2048) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `status` int(11) NOT NULL DEFAULT 0,
  `original_service_type` int(11) DEFAULT 0,
  `is_template` tinyint(1) NOT NULL DEFAULT 0,
  `template_id` bigint(20) DEFAULT NULL,
  `owner_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `UQE_repository_s` (`owner_id`,`lower_name`),
  KEY `IDX_repository_is_archived` (`is_archived`),
  KEY `IDX_repository_owner_id` (`owner_id`),
  KEY `IDX_repository_lower_name` (`lower_name`),
  KEY `IDX_repository_is_mirror` (`is_mirror`),
  KEY `IDX_repository_is_private` (`is_private`),
  KEY `IDX_repository_is_empty` (`is_empty`),
  KEY `IDX_repository_template_id` (`template_id`),
  KEY `IDX_repository_original_service_type` (`original_service_type`),
  KEY `IDX_repository_name` (`name`),
  KEY `IDX_repository_updated_unix` (`updated_unix`),
  KEY `IDX_repository_is_template` (`is_template`),
  KEY `IDX_repository_created_unix` (`created_unix`),
  KEY `IDX_repository_fork_id` (`fork_id`),
  KEY `IDX_repository_is_fork` (`is_fork`)
) ENGINE=InnoDB AUTO_INCREMENT=283 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci |
+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.001 sec)

MariaDB [gitea]> 

[markuman]

It's MariaDB 10.4.13

[markuman]

I can insert without any problems

MariaDB [gitea]> update repository set description = 'bring up your nextcloud usage to the next level 💩' where id = 280;
Query OK, 1 row affected (0.003 sec)
Rows matched: 1  Changed: 1  Warnings: 0

but gitea displays it wrong

[lunny]

@markuman You should also change your app.ini in [database], set charset=utf8mb4.

[markuman]

@markuman You should also change your app.ini in [database], set charset=utf8mb4.

@lunny uppercase CHARSET and it works. Thx!

[zeripath]

On master and 1.13 - once it is released - #12563 will set the default CHARSET as utf8mb4 for MySQL so this will no longer be necessary.