New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

500 HTTP error adding a new article #2976

Closed
ngosang opened this Issue Mar 13, 2017 · 19 comments

Comments

Projects
None yet
8 participants
@ngosang
Copy link
Contributor

ngosang commented Mar 13, 2017

Issue details

  1. Add => http://lamiradadelreplicante.com/2017/03/12/15-anos-de-arch-linux/
  2. Error happens;

500: Internal Server Error

An exception occurred while executing 'INSERT INTO `wallabag_entry` (uid, title, url, is_archived, is_starred, content, created_at, updated_at, mimetype, language, reading_time, domain_name, preview_picture, is_public, http_status, user_id) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [null, "15 a\u00f1os de Arch Linux", "http:\/\/lamiradadelreplicante.com\/2017\/03\/12\/15-anos-de-arch-linux\/", 0, 0, "<section cl

...

o.jpg", null, 200, 1]: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xF0\x9F\x98\x89</...' for column 'content' at row 1

Environment

  • Wallabag 2.2.2
  • Mysql 5.7.17
  • PHP: 7.0.15
  • wallabag_entry table encoding => utf8mb4_unicode_ci (not changed)

Update: I recently upgrade from 2.2.1 without problems. This is happening only in this article.

@tcitworld

This comment has been minimized.

Copy link
Member

tcitworld commented Mar 13, 2017

@nicosomb Please compare the config to the one used on wallabag.it since it doesn't happen there.

@tcitworld tcitworld added the Bug label Mar 13, 2017

@ngosang

This comment has been minimized.

Copy link
Contributor

ngosang commented Mar 13, 2017

Please run this in your mysql server:

SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%';

My results:

character_set_client	utf8mb4	
character_set_connection	utf8mb4	
character_set_database	latin1	
character_set_filesystem	binary	
character_set_results	utf8mb4	
character_set_server	latin1	
character_set_system	utf8	
collation_connection	utf8mb4_unicode_ci	
collation_database	latin1_swedish_ci	
collation_server	latin1_swedish_ci	
@ngosang

This comment has been minimized.

Copy link
Contributor

ngosang commented Mar 13, 2017

I changed the mysql server configuration adding this:

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Now the previous sql query retuns:

character_set_client	utf8mb4	
character_set_connection	utf8mb4	
character_set_database	utf8mb4	
character_set_filesystem	binary	
character_set_results	utf8mb4	
character_set_server	utf8mb4	
character_set_system	utf8	
collation_connection	utf8mb4_unicode_ci	
collation_database	utf8mb4_unicode_ci	
collation_server	utf8mb4_unicode_ci	

Now I can add the article correctly, but since this is not the default mysql server configuration I think you should add this in wallabag. This is the configuration used by mysql if you don't add the encoding when you stablish the connection in PHP.

More info => https://mathiasbynens.be/notes/mysql-utf8mb4

@shikasta-net

This comment has been minimized.

Copy link

shikasta-net commented Apr 12, 2017

I have had this problem with https://food52.com/blog/17210-how-to-make-any-kind-of-amaro-at-home
My MySQL 5.6 is running the default configuration, which for me is

character_set_client          latin1
character_set_connection      latin1
character_set_database        latin1
character_set_filesystem      binary
character_set_results         latin1
character_set_server          latin1
character_set_system          utf8
collation_connection          latin1_swedish_ci
collation_database            latin1_swedish_ci
collation_server              latin1_swedish_ci 

I'm not sure we can rely on the database config being the same so wallabag should have some way to deal with the different encodings.

@nicosomb

This comment has been minimized.

Copy link
Member

nicosomb commented Apr 15, 2017

@shikasta-net could you change your database configuration, like @ngosang?

@shikasta-net

This comment has been minimized.

Copy link

shikasta-net commented Apr 16, 2017

Hi @nicosomb, from what I read, the change affects the global configuration. I have several services using the same engine so I'm a bit hesitant in case changing this breaks them. I'll have another look to see if the effects can be constrained to just the Wallabag schema.

@shikasta-net

This comment has been minimized.

Copy link

shikasta-net commented Apr 16, 2017

Default behaviour of mysql is latin1 unless the client request otherwise (https://dev.mysql.com/doc/refman/5.6/en/charset-applications.html).

I have tried adding character-set-server=utf8mb4 and collation-server=utf8mb4_unicode_ci to mysqld startup (I'm running in docker without cnf file). I've not added a flag to mysqld to ignore the client character set handshake because I don't want other services failing when they negotiate for different behaviour. The configuration reported by mysql is now as follows (when I start the client with --default-character-set=utf8):

character_set_client          utf8 *
character_set_connection      utf8 *
character_set_database        utf8mb4
character_set_filesystem      binary
character_set_results         utf8 *
character_set_server          utf8mb4
character_set_system          utf8 *
collation_connection          utf8_general_ci *
collation_database            utf8mb4_unicode_ci
collation_server              utf8mb4_unicode_ci

If I change to --default-character-set=utf8mb4 the astarisk lines reflect this character set; if I omit the character set flag, those lines become latin1 as before.

The Wallabag tables appear to have been created with utf8 collation (except lexik_maintenance which was latin1_swedish_ci) so the client must not be specifying the encoding when it creates the connection.

@aaa2000

This comment has been minimized.

Copy link
Contributor

aaa2000 commented May 8, 2017

@shikasta-net The encoding of the connection is defined in https://github.com/wallabag/wallabag/blame/master/app/config/parameters.yml.dist#L23 Can you check your parameters.yml file ?

@shikasta-net

This comment has been minimized.

Copy link

shikasta-net commented May 18, 2017

@aaa2000, my parameters.yml says database_charset: utf8

@j0k3r

This comment has been minimized.

Copy link
Member

j0k3r commented May 19, 2017

As stated in the doc: https://doc.wallabag.org/en/admin/parameters.html

For PostgreSQL & SQLite you should use utf8, for MySQL use utf8mb4 to handle emoji and other special characters

@nicosomb

This comment has been minimized.

Copy link
Member

nicosomb commented Dec 16, 2017

@ngosang Do you still have a bug here?

@jjelev

This comment has been minimized.

Copy link

jjelev commented Dec 17, 2017

Same bug from me on this link https://github.com/ctjhoa/rust-learning

Installed from Docker Compose locally.

version: '3'
services:
  wallabag:
    image: wallabag/wallabag
    environment:
      - MYSQL_ROOT_PASSWORD=wallaroot
      - SYMFONY__ENV__DATABASE_DRIVER=pdo_mysql
      - SYMFONY__ENV__DATABASE_HOST=db
      - SYMFONY__ENV__DATABASE_PORT=3306
      - SYMFONY__ENV__DATABASE_NAME=wallabag
      - SYMFONY__ENV__DATABASE_USER=wallabag
      - SYMFONY__ENV__DATABASE_PASSWORD=wallapass
      - SYMFONY__ENV__MAILER_HOST=127.0.0.1
      - SYMFONY__ENV__MAILER_USER=~
      - SYMFONY__ENV__MAILER_PASSWORD=~
      - SYMFONY__ENV__FROM_EMAIL=wallabag@example.com
    ports:
      - "80:80"
    volumes:
      - ./images:/var/www/wallabag/web/assets/images
  db:
    image: mariadb
    environment:
      - MYSQL_ROOT_PASSWORD=wallaroot
    volumes:
      - /opt/wallabag/data:/var/lib/mysql
  redis:
    image: redis:alpine 

Environment: Manjaro Linux x64, kernel 4.14

@nicosomb

This comment has been minimized.

Copy link
Member

nicosomb commented Dec 18, 2017

@jjelev Please open a new issue in our docker repository https://github.com/wallabag/docker/issues/new because I think that it's more docker-related.

@pfa230

This comment has been minimized.

Copy link

pfa230 commented Dec 19, 2017

I'm having the same issue.

Example article: https://justpaste.it/1dq1o

My MySQL variables:

character_set_client | utf8mb4
character_set_connection | utf8mb4
character_set_database | utf8mb4
character_set_filesystem | binary
character_set_results | utf8mb4
character_set_server | utf8mb4
character_set_system | utf8
character_sets_dir | /usr/share/mysql/charsets/
collation_connection | utf8mb4_general_ci
collation_database | utf8mb4_unicode_ci
collation_server | utf8mb4_unicode_ci

database_charset is set to utf8mb4 in parameters.yml.

I was able to fix it only by passing --skip-character-set-client-handshake to the mariadb, which obviously is a dirty hack.

@nicosomb

This comment has been minimized.

Copy link
Member

nicosomb commented Dec 19, 2017

@pfa230 you use docker?

@ngosang

This comment has been minimized.

Copy link
Contributor

ngosang commented Dec 19, 2017

Do you still have a bug here?

@nicosomb Everything is working fine after months and several updates.

This is the MySQL default configuration => #2976 (comment)

You can try to change the connection encoding when you stablish the php=>mysql connection. I don't know if it's possible but you can read about here => http://php.net/manual/en/mysqli.set-charset.php#121647

I fixed it changing MySQL configuration => #2976 (comment)

By now, you can include the MySQL configuration in MySQL installation guide.

@pfa230

This comment has been minimized.

Copy link

pfa230 commented Dec 20, 2017

@nicosomb I do

@nicosomb

This comment has been minimized.

Copy link
Member

nicosomb commented Dec 20, 2017

@pfa230 Please open a new issue in our docker repository https://github.com/wallabag/docker/issues/new

@shikasta-net

This comment has been minimized.

Copy link

shikasta-net commented Aug 11, 2018

For people like me who keep finding this thread because they get General error: 1366 when trying to add certain articles to a docker wallabag using mysql:
firstly, make sure your mysql wallabag schema is set so that character_set_database = utf8mb4 as above;
and add SYMFONY__ENV__DATABASE_CHARSET=utf8mb4 to your wallabag docker environment so that ansible builds the correct value in parameters.yml. This isn't documented and took ages to find.

@j0k3r j0k3r referenced this issue Aug 20, 2018

Merged

Add common errors #72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment