get fewer message when using MySQL compared to SQLite

[blackarcher21]

Brief description of the issue.

http://okistudio.com/feed/
Since yesterday with two times doing update, I got 4 messages when using MySQL and 15 messages when using SQLite
Using MySQL 5.6.29, RSS Guard 3.3.6-ee9e2bc-nowebengine dev build, Windows 8.1 x64

What is the expected result?

I should got same amount messages

Other information (logs, see Wiki)

log.txt

[martinrotter]

Yes, I see the problem. I will test with the feed.

BTW: I plan to address rather wider spectrum of problems which are touching DB usage in RSS Guard. #71 is related to DB to some extent too. #70 too.

[martinrotter]

Your MySQL storage will get updated to new DB schema, read CHANGELOG when new version will get released. You can try latest development build which will be compiled shortly, see Wiki on info how to get latest dev build.

[blackarcher21]

just tested with that feed using v3.3.6-ec16fca-nowebengine, clean database, both SQLite and MySQL, Using MySQL, I get 9 messages, compared to SQLite I get 10 messages. Could you identify what could be the cause?

[martinrotter]

Please post here log.txt.

[martinrotter]

The issue is that some characters in UTF-8 encoding were not supported by so far user MySQL character set "utf8", so I switched to newer MySQL character set which should support all UTF-8 characters, so that you should no longer see messages:

[rssguard] DEBUG: Failed to insert message to DB: 'Incorrect string value: '\xF0\x9F\x99\x82" ...' for column 'contents' at row 1 QMYSQL3: Unable to execute statement'.

in log file. Post it here so that we can see the reason why some messages were not added to DB.

[martinrotter]

Right now, XML online feed contains NINE messages. RSS Guard here fetches ALL nine messages.

EDIT: Misread, really, it has 10 items. Let me check.

[martinrotter]

OK, double checked. Both MySQL & SQLite download all 10 messages. Are you really sure, that some of those messages was not "permanently deleted" from recycle bin or is not in recycle bin right now?

[blackarcher21]

I don't use my existing RSS Guard, but test it with the portable, no database yet.

SQLite log
log-sqlite.txt

MySQL log
log-mysql.txt

[martinrotter]

What is the version of your MySQL server. Problem here is with message 'Gambar Arema untuk Wallpaper, Dp BBM Bergerak, Kata Kata Aremania Terbaru' .

[blackarcher21]

Server version: 5.6.29 MySQL Community Server (GPL)

[martinrotter]

Problem is with character http://www.fileformat.info/info/unicode/char/1F600/index.htm

This is what LOG is saying.

[martinrotter]

Here, the message gets inserted but those characters get messed. But it "works".

[blackarcher21]

Here message is not inserted. So this is limitation of MySQL? or it has something to do with my MySQL setting?

[martinrotter]

Well, latest dev version sets character set of DATABASE, TABLE (the one with messages) and COLUMNS (ones inside table with messages) to correct set now. The only character set remaining is the character set of the server itself, that must be changed in settings.

Check this.

You need utf8mb4 character set, because that is the one which supports characters witch really big underlying numerical values (4-bytes).

Try to run show variables like 'char%'; SQL query to see, what values are currently set.

[martinrotter]

Thus, setting character set of DB and table is (almost) everything RSS Guard can do, all other settings must be done by system admin. The issue still might be on the side of RSS Guard, I do not guarantee it is not. I will try to investigate the issue further, sent your settings (see previous post).

[martinrotter]

Particularly interesting line will be:

character_set_connection | XXXXX

[blackarcher21]

my default setting result
http://i.imgur.com/QcudNux.png

anyway I do search for some times, and found this, follow the guide
here's the result
http://i.imgur.com/MVqIUTS.png

After that, I try again with no database and still miss a message

[martinrotter]

Okay, please send me link to the exact MySQL mutation/version installer you are using, pls. I will test it myself.

[blackarcher21]

I upgraded from wamp 2.5 to 3, but upgrade installer is not available anymore on server (I just checked that) and I can't upload due to my speed limitation.

But you might want to try this

Version I have

WampServer Version 3.0.7 64bit
Created by Romain Bourdon (2005)
Maintainer / Upgrade to 2.5 by Herve Leclerc
Upgrade to 3 by Otomatic (wampserver@otomatic.net)
Multi styles for homepage by Jojaba
Installer by Inno Setup: http://www.jrsoftware.org/isinfo.php
Forum Wampserver: http://forum.wampserver.com/index.php
______________________ Versions used ______________________
Apache 2.4.18 - PHP 5.6.18 - MySQL 5.6.29
PHP 5.5.12 for CLI (Command-Line Interface)

my my.ini (just in case you want to check this also)
my.zip

[martinrotter]

OK, thx.

[martinrotter]

I managed to replicate the issue with WAMP. It is really weird, because with XAMPP (I used for testing.) it "works".

I will try to find the cause of the problem with WAMP.

[martinrotter]

OK, I think I fixed it. Wait for dev release dc6b5b6 to compile and then try it pls.

[blackarcher21]

Yes, now it works. The message that was not inserted, now it is inserted. I was thinking it MySQL setting fault. Anyway, thanks a lot, now it works well

[martinrotter]

Well, partially it was some fault inside Qt library, partially MySQL annoyances. Good. :)