New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
errors inserting to the database #1436
Comments
Make sure that the php extension mbstring is loaded by doing php -m from
the command line. As I don't know what operating system "Alpine" is a
version of, I can't help you set up the init file to load it.
Also, accessing /phpinfo.php from the network should reveal if mbstring
is loaded.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/03/2016 07:06 AM, James Eckersall wrote:
* Ampache version: 3.8.2
* Web server + version: Nginx 1.10 + PHP 5.6.28
* Server operating system: Alpine
* Client operating system: NA
Hi,
I'm having a problem trying to create a catalog from a bunch of mp3
albums. I have enabled debug logging in ampache and see the following
repeated over and over again:
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFED\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
I've checked the mysql db charset and collate settings and they are:
mysql> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME
-> FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = 'ampache';
+----------------------------+------------------------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME |
+----------------------------+------------------------+
| utf8 | utf8_unicode_ci |
+----------------------------+------------------------+
1 row in set (0.00 sec)
The tables also have the appropriate utf8 params.
Can anyone help me to work out where I've gone wrong here?
Thanks
J
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1436>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1QbjLGG8ubwcFq0EM0dDUrjZobyMk3ks5rEYWDgaJpZM4LDUVH>.
|
Hi, Thanks |
OK, assuming you tried it with the develop branch, could the titles
have some strange characters? From what I can see on
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html MySQL
doesn't handle the entire utf8 character set. It only handles characters
up to three bytes wide.
Just throwing it out there for your consideration.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/03/2016 12:44 PM, James Eckersall wrote:
Hi,
Alpine is Linux-based. I have checked and I do have the mbstring
extention loaded.
I should also point out that I've tried this with PHP 5.6 and PHP 7.0
and the result is the same.
Thanks
J
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1QbiGs-DWlxbyxtp4QRXc84iMnZQmIks5rEdSUgaJpZM4LDUVH>.
|
Thanks for the advice. I have 200+ albums but only 40 or so get imported,
the rest are all reporting those errors. Looking at the unicode strings
the error message reports, they only seem to be two bytes. I wonder if
it's possible to get the entire error from mysql as it is truncated in the
ampache log.
…On Sat, 3 Dec 2016 at 22:22 Ernest Wagner ***@***.***> wrote:
OK, assuming you tried it with the develop branch, could the titles
have some strange characters? From what I can see on
https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html MySQL
doesn't handle the entire utf8 character set. It only handles characters
up to three bytes wide.
Just throwing it out there for your consideration.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
On 12/03/2016 12:44 PM, James Eckersall wrote:
>
> Hi,
> Alpine is Linux-based. I have checked and I do have the mbstring
> extention loaded.
> I should also point out that I've tried this with PHP 5.6 and PHP 7.0
> and the result is the same.
>
> Thanks
> J
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1436 (comment)>,
> or mute the thread
> <
https://github.com/notifications/unsubscribe-auth/AF1QbiGs-DWlxbyxtp4QRXc84iMnZQmIks5rEdSUgaJpZM4LDUVH
>.
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE1ckdeakXsSQg_UkfbOGbfotEfUHezaks5rEeu1gaJpZM4LDUVH>
.
|
What bothers me is how the BOM (Byte Order Mark) is displayed:
'\xFF\xFED\x00o\x00...'. The "D" shouldn't be there unless ampache isn't
displaying the data correctly. The BOM should be "\xFF\xFE".
Also, "\x00o" I believe indicates 4bytes as the "o" is 2 bytes.
Have you checked mysqld.log to see if it shows anything?
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/03/2016 02:45 PM, James Eckersall wrote:
Thanks for the advice. I have 200+ albums but only 40 or so get imported,
the rest are all reporting those errors. Looking at the unicode strings
the error message reports, they only seem to be two bytes. I wonder if
it's possible to get the entire error from mysql as it is truncated in the
ampache log.
On Sat, 3 Dec 2016 at 22:22 Ernest Wagner ***@***.***>
wrote:
> OK, assuming you tried it with the develop branch, could the titles
> have some strange characters? From what I can see on
> https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html MySQL
> doesn't handle the entire utf8 character set. It only handles characters
> up to three bytes wide.
>
> Just throwing it out there for your consideration.
>
>
> 73 From AA1AD
> Ernie D
> Federal Way, WA | CN87ug
>
> On 12/03/2016 12:44 PM, James Eckersall wrote:
> >
> > Hi,
> > Alpine is Linux-based. I have checked and I do have the mbstring
> > extention loaded.
> > I should also point out that I've tried this with PHP 5.6 and PHP 7.0
> > and the result is the same.
> >
> > Thanks
> > J
> >
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub
> >
<#1436 (comment)>,
> > or mute the thread
> > <
>
https://github.com/notifications/unsubscribe-auth/AF1QbiGs-DWlxbyxtp4QRXc84iMnZQmIks5rEdSUgaJpZM4LDUVH
> >.
> >
>
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#1436 (comment)>,
> or mute the thread
>
<https://github.com/notifications/unsubscribe-auth/AE1ckdeakXsSQg_UkfbOGbfotEfUHezaks5rEeu1gaJpZM4LDUVH>
> .
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1Qblmu4ukOx2UutwkIsE8VXEEgd2i8ks5rEfD7gaJpZM4LDUVH>.
|
One more thing: is php-iconv extension loaded?
There is a php file in Ampache's bin folder called fix_filenames.inc. It
will get the catalog paths and check all the files for characters that
don't conform to the system characterset.
Maybe that will help on those 40 files.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/03/2016 02:45 PM, James Eckersall wrote:
Thanks for the advice. I have 200+ albums but only 40 or so get imported,
the rest are all reporting those errors. Looking at the unicode strings
the error message reports, they only seem to be two bytes. I wonder if
it's possible to get the entire error from mysql as it is truncated in the
ampache log.
On Sat, 3 Dec 2016 at 22:22 Ernest Wagner ***@***.***>
wrote:
> OK, assuming you tried it with the develop branch, could the titles
> have some strange characters? From what I can see on
> https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html MySQL
> doesn't handle the entire utf8 character set. It only handles characters
> up to three bytes wide.
>
> Just throwing it out there for your consideration.
>
>
> 73 From AA1AD
> Ernie D
> Federal Way, WA | CN87ug
>
> On 12/03/2016 12:44 PM, James Eckersall wrote:
> >
> > Hi,
> > Alpine is Linux-based. I have checked and I do have the mbstring
> > extention loaded.
> > I should also point out that I've tried this with PHP 5.6 and PHP 7.0
> > and the result is the same.
> >
> > Thanks
> > J
> >
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub
> >
<#1436 (comment)>,
> > or mute the thread
> > <
>
https://github.com/notifications/unsubscribe-auth/AF1QbiGs-DWlxbyxtp4QRXc84iMnZQmIks5rEdSUgaJpZM4LDUVH
> >.
> >
>
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#1436 (comment)>,
> or mute the thread
>
<https://github.com/notifications/unsubscribe-auth/AE1ckdeakXsSQg_UkfbOGbfotEfUHezaks5rEeu1gaJpZM4LDUVH>
> .
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1Qblmu4ukOx2UutwkIsE8VXEEgd2i8ks5rEfD7gaJpZM4LDUVH>.
|
Interesting. I tried running the fix_filenames.inc script and got the
following error in the ampache log:
2016-12-04 03:45:22 [ampache] (PHP) -> [Runtime Error] iconv() expects
exactly 3 parameters, 2 given in file /www/bin/fix_filenames.inc(51)
That's running against the 3.8.2 tag. If I run fix_filenames.inc against
the develop branch, I get no output at all, nothing in the log either.
Here is the output of php -m for reference:
$ php -m
[PHP Modules]
Core
curl
date
ereg
fileinfo
filter
gd
gettext
hash
iconv
json
libxml
mbstring
mysqlnd
openssl
pcre
PDO
pdo_mysql
Phar
readline
Reflection
session
SimpleXML
SPL
standard
tokenizer
xml
xmlwriter
zlib
[Zend Modules]
$ php -v
PHP 5.6.28 (cli) (built: Nov 18 2016 09:37:16)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
$
…On Sun, 4 Dec 2016 at 02:26 Ernest Wagner ***@***.***> wrote:
One more thing: is php-iconv extension loaded?
There is a php file in Ampache's bin folder called fix_filenames.inc. It
will get the catalog paths and check all the files for characters that
don't conform to the system characterset.
Maybe that will help on those 40 files.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
On 12/03/2016 02:45 PM, James Eckersall wrote:
> Thanks for the advice. I have 200+ albums but only 40 or so get imported,
> the rest are all reporting those errors. Looking at the unicode strings
> the error message reports, they only seem to be two bytes. I wonder if
> it's possible to get the entire error from mysql as it is truncated in
the
> ampache log.
>
> On Sat, 3 Dec 2016 at 22:22 Ernest Wagner ***@***.***>
> wrote:
>
> > OK, assuming you tried it with the develop branch, could the titles
> > have some strange characters? From what I can see on
> > https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html
MySQL
> > doesn't handle the entire utf8 character set. It only handles
characters
> > up to three bytes wide.
> >
> > Just throwing it out there for your consideration.
> >
> >
> > 73 From AA1AD
> > Ernie D
> > Federal Way, WA | CN87ug
> >
> > On 12/03/2016 12:44 PM, James Eckersall wrote:
> > >
> > > Hi,
> > > Alpine is Linux-based. I have checked and I do have the mbstring
> > > extention loaded.
> > > I should also point out that I've tried this with PHP 5.6 and PHP 7.0
> > > and the result is the same.
> > >
> > > Thanks
> > > J
> > >
> > > —
> > > You are receiving this because you commented.
> > > Reply to this email directly, view it on GitHub
> > >
> <#1436 (comment)>,
> > > or mute the thread
> > > <
> >
>
https://github.com/notifications/unsubscribe-auth/AF1QbiGs-DWlxbyxtp4QRXc84iMnZQmIks5rEdSUgaJpZM4LDUVH
> > >.
> > >
> >
> >
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> > <#1436 (comment)
>,
> > or mute the thread
> >
> <
https://github.com/notifications/unsubscribe-auth/AE1ckdeakXsSQg_UkfbOGbfotEfUHezaks5rEeu1gaJpZM4LDUVH
>
> > .
> >
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1436 (comment)>,
> or mute the thread
> <
https://github.com/notifications/unsubscribe-auth/AF1Qblmu4ukOx2UutwkIsE8VXEEgd2i8ks5rEfD7gaJpZM4LDUVH
>.
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE1ckWF2IlwuhH85E9N6sNWtqE6n2xHjks5rEiTigaJpZM4LDUVH>
.
|
For me there is output and the same error for both php-5.6 and 7.0. There is a bug in line 51 which I may put in a pull request, but there seems to be another problem with the code which I may debug. Sorry about that. There is an undocumented debug level of 6 that displays the data being sent to the database, that might help. Beware, that it adds quite a bit to the log. |
Hi thanks for the tip. I've tried setting the debug level to 6, and whilst it gives some more output, none of it is really any further help I feel. Here is an example: 2016-12-04 19:19:14 [ampache] (Query) -> SELECT |
James,
I have some more ideas, but I need to replicate the problem here. Could
you send me a problem mp3 file off line? I have a php development
environment and debug on my system.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/04/2016 11:20 AM, James Eckersall wrote:
Hi thanks for the tip. I've tried setting the debug level to 6, and
whilst it gives some more output, none of it is really any further
help I feel.
Here is an example:
2016-12-04 19:19:14 [ampache] (Query) -> SELECT |id|, |mbid| FROM
|artist| WHERE |name| LIKE ?
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Query) -> INSERT INTO |artist| (|name|,
|prefix|, |mbid|) VALUES(?, ?, ?)
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1Qbjt1A89wL-OfLgxS0IuCHkKHFLavks5rExJ2gaJpZM4LDUVH>.
|
I was able to replicate (sort of) by inserting a supplemental unicode
character in the album tag. Fix_filenames only looks at the filename,
not the id3tags.
I suggest that you use easytag to look at the id3v2.* data of the
problem mp3 files. It is very versatile: letting you save to different
character sets among other features. You might have to manually change
id3 metadata for all the problem files: which EasyTag will do in batch mode.
I think that those problem files have metadata that is of mixed
character sets so MySQL can't deal with the strings.
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/04/2016 11:20 AM, James Eckersall wrote:
Hi thanks for the tip. I've tried setting the debug level to 6, and
whilst it gives some more output, none of it is really any further
help I feel.
Here is an example:
2016-12-04 19:19:14 [ampache] (Query) -> SELECT |id|, |mbid| FROM
|artist| WHERE |name| LIKE ?
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Query) -> INSERT INTO |artist| (|name|,
|prefix|, |mbid|) VALUES(?, ?, ?)
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
2016-12-04 19:19:14 [ampache] (Dba) -> Database connection...
2016-12-04 19:19:14 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect
string value: '\xFF\xFEJ\x00e\x00...' for column 'name' at row 1"]
2016-12-04 19:19:14 [ampache] (Dba) -> Database disconnection.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1Qbjt1A89wL-OfLgxS0IuCHkKHFLavks5rExJ2gaJpZM4LDUVH>.
|
I use Tag&Rename to set the tags in my mp3s. This can either save the tags as ascii or unicode. It looks like it uses UTF16 and there is no option for anything else. I have been trying to get all of this to work in my docker image for ampache available at https://github.com/jameseck/docker-ampache/ The docker-compose file creates a mysql container as well as the ampache one. It's a bit rough and ready and there are things that don't work, but I'd appreciate it if anyone is able to take a look and advise on why this isn't working as expected. J |
I should also add that I had previously tried https://github.com/ampache/ampache-docker and this seemed to pick up all files in my catalog correctly, so it's likely that something in my docker image is wrong, but I have no idea what. |
You might want to create a separate issue on the docker: I don't know
anything about it. Also would a docker forum be more appropriate for
that issue?
73 From AA1AD
Ernie D
Federal Way, WA | CN87ug
…On 12/10/2016 11:53 AM, James Eckersall wrote:
I use Tag&Rename to set the tags in my mp3s. This can either save the
tags as ascii or unicode. It looks like it uses UTF16 and there is no
option for anything else.
I have it set to save all tags are unicode, which are then UTF16.
I picked a single mp3 to test with and ampache failed to add this to
the catalog with the errors previously provided in this issue.
I found taglib-ruby and whipped up a quick script to verify this. The
tags are indeed saved as UTF16.
I then used taglib-ruby to change all the tags in this file to UTF8
and ran the update_catalog.inc script. This time, ampache added it to
the database correctly and it's available for playing.
I have been trying to get all of this to work in my docker image for
ampache available at https://github.com/jameseck/docker-ampache/
The docker-compose file creates a mysql container as well as the
ampache one. It's a bit rough and ready and there are things that
don't work, but I'd appreciate it if anyone is able to take a look and
advise on why this isn't working as expected.
J
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1436 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF1QbuwyH-mcrGbGy5mvTdqea2PQyZdwks5rGwNAgaJpZM4LDUVH>.
|
@jameseck could it be related to https://github.com/cantino/huginn/blob/master/.env.example#L30-L33 ? |
default collation is going to change on the database so i'll close this issue when develop makes that change. |
default collation has changed in the develop branch and can be changed with |
Hi,
I'm having a problem trying to create a catalog from a bunch of mp3 albums. I have enabled debug logging in ampache and see the following repeated over and over again:
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect string value: '\xFF\xFED\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
2016-12-03 15:03:46 [ampache] (Dba) -> Error: ["HY000",1366,"Incorrect string value: '\xFF\xFEG\x00o\x00...' for column 'name' at row 1"]
I've checked the mysql db charset and collate settings and they are:
mysql> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME
-> FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = 'ampache';
+----------------------------+------------------------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME |
+----------------------------+------------------------+
| utf8 | utf8_unicode_ci |
+----------------------------+------------------------+
1 row in set (0.00 sec)
The tables also have the appropriate utf8 params.
Can anyone help me to work out where I've gone wrong here?
Thanks
J
The text was updated successfully, but these errors were encountered: