New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SphinxQL Query Charset #2384

Closed
coolswitcher opened this Issue Jan 25, 2016 · 9 comments

Comments

Projects
None yet
2 participants
@coolswitcher

When i'm run sphinxql query, i'm got output in wrong charset:
screenshot 2016-01-25 11 52 58
Server variables of sphinx server:
screenshot 2016-01-25 11 59 59
Source mysql database in utf8 with collocation utf8_general_ci

@dmoagx

This comment has been minimized.

Show comment
Hide comment
@dmoagx

dmoagx Jan 25, 2016

Member

Open Console.app and search for output from Sequel Pro.

Member

dmoagx commented Jan 25, 2016

Open Console.app and search for output from Sequel Pro.

@coolswitcher

This comment has been minimized.

Show comment
Hide comment
@coolswitcher

coolswitcher Jan 25, 2016

I found these errors:

25.01.16 12:42:20,477 Sequel Pro[11570]: SecTaskLoadEntitlements failed error=22

25.01.16 11:57:59,000 kernel[0]: Sandbox: Mail(556) deny(1) file-read-data /Applications/Sequel Pro.app/Contents/Library/QuickLook/Sequel Pro.qlgenerator/Contents

25.01.16 11:34:42,915 Sequel Pro[10995]: -[NSAlert(SPAlertDebug) sp_buttonPressed:] of <NSAlert: 0x10b5d58a0> title=
MySQL Error
text=
Unknown collation: ''utf8_general_ci''

25.01.16 11:43:14,757 Sequel Pro[10995]: -[NSAlert(SPAlertDebug) sp_buttonPressed:] of <NSAlert: 0x11790d8c0> title=
Ошибка
text=
sphinxql: syntax error, unexpected IDENT, expecting ATTRIBUTES or RAMCHUNK or RTINDEX near 'PRIVILEGES'

I found these errors:

25.01.16 12:42:20,477 Sequel Pro[11570]: SecTaskLoadEntitlements failed error=22

25.01.16 11:57:59,000 kernel[0]: Sandbox: Mail(556) deny(1) file-read-data /Applications/Sequel Pro.app/Contents/Library/QuickLook/Sequel Pro.qlgenerator/Contents

25.01.16 11:34:42,915 Sequel Pro[10995]: -[NSAlert(SPAlertDebug) sp_buttonPressed:] of <NSAlert: 0x10b5d58a0> title=
MySQL Error
text=
Unknown collation: ''utf8_general_ci''

25.01.16 11:43:14,757 Sequel Pro[10995]: -[NSAlert(SPAlertDebug) sp_buttonPressed:] of <NSAlert: 0x11790d8c0> title=
Ошибка
text=
sphinxql: syntax error, unexpected IDENT, expecting ATTRIBUTES or RAMCHUNK or RTINDEX near 'PRIVILEGES'

@dmoagx

This comment has been minimized.

Show comment
Hide comment
@dmoagx

dmoagx Jan 29, 2016

Member

Do you have anything set in Database > Charset for display?
Also look into Sequel Pros Query console and check for the most recent SET NAMES... query that appears before loading your table.

Member

dmoagx commented Jan 29, 2016

Do you have anything set in Database > Charset for display?
Also look into Sequel Pros Query console and check for the most recent SET NAMES... query that appears before loading your table.

@coolswitcher

This comment has been minimized.

Show comment
Hide comment
@coolswitcher

coolswitcher Jan 29, 2016

In Database > Charset is set UTF-8 Unicode BMP and i can't change this
In console i haven't SET NAMES queries, i'm try run:
SET NAMES utf8;
SET CHARACTER SET utf8;

No effect:(

In Database > Charset is set UTF-8 Unicode BMP and i can't change this
In console i haven't SET NAMES queries, i'm try run:
SET NAMES utf8;
SET CHARACTER SET utf8;

No effect:(

@dmoagx

This comment has been minimized.

Show comment
Hide comment
@dmoagx

dmoagx Jan 29, 2016

Member

I guess because Sphinx does not return data for the regular charset queries, Sequel Pro assumes that it is a very old MySQL version without charset support and falls back to latin1.

Member

dmoagx commented Jan 29, 2016

I guess because Sphinx does not return data for the regular charset queries, Sequel Pro assumes that it is a very old MySQL version without charset support and falls back to latin1.

@dmoagx

This comment has been minimized.

Show comment
Hide comment
@dmoagx

dmoagx Feb 5, 2016

Member

Thinking about it, I already mentioned in c7b5e88 that SP will treat Sphinx as a MySQL 2.x server because of the version number.

But we have this code:

- (BOOL)setEncoding:(NSString *)theEncoding
{

    // MySQL versions prior to 4.1 don't support encoding changes; return NO on those
    // versions.
    if (![self serverVersionIsGreaterThanOrEqualTo:4 minorVersion:1 releaseVersion:0]) {
        return NO;
    }

+ the fallback encoding is latin1 for backwards compatibility.

Sequel Pro also reads the character_set_results and character_set server variables to determine the encoding but it looks like Sphinx uses the other two.

Member

dmoagx commented Feb 5, 2016

Thinking about it, I already mentioned in c7b5e88 that SP will treat Sphinx as a MySQL 2.x server because of the version number.

But we have this code:

- (BOOL)setEncoding:(NSString *)theEncoding
{

    // MySQL versions prior to 4.1 don't support encoding changes; return NO on those
    // versions.
    if (![self serverVersionIsGreaterThanOrEqualTo:4 minorVersion:1 releaseVersion:0]) {
        return NO;
    }

+ the fallback encoding is latin1 for backwards compatibility.

Sequel Pro also reads the character_set_results and character_set server variables to determine the encoding but it looks like Sphinx uses the other two.

@dmoagx

This comment has been minimized.

Show comment
Hide comment
@dmoagx

dmoagx Feb 5, 2016

Member

I've added some code in 6457da2 which might fix this.
Please try a recent nightly build.

Member

dmoagx commented Feb 5, 2016

I've added some code in 6457da2 which might fix this.
Please try a recent nightly build.

@dmoagx dmoagx added the WaitingOnUser label Feb 5, 2016

@coolswitcher

This comment has been minimized.

Show comment
Hide comment
@coolswitcher

coolswitcher Feb 6, 2016

Thanks for fix :) Now all works fine!
I'm have testing on sphinx ver 2.2.9

Thanks for fix :) Now all works fine!
I'm have testing on sphinx ver 2.2.9

@dmoagx dmoagx added this to the 1.1.1 milestone Feb 7, 2016

@dmoagx

This comment has been minimized.

Show comment
Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment