Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[9.x] Fix Queue Failed_jobs insert issue with Exception contain UNICODE #41020

Merged
merged 1 commit into from
Feb 16, 2022

Conversation

boomhq
Copy link
Contributor

@boomhq boomhq commented Feb 15, 2022

Can't Insert failed job exception with UNICODE character.
Add an "utf8_encode" front of exception.

If you have a better way, my ears is open
I deal manually with this case from laravel 5.5

Issues :
#24263
#41016

@boomhq boomhq changed the title [9.x] Fix Queue failed logging exception UNICODE [9.x] Fix Queue Failed_jobs insert issue with Exception contain UNICODE Feb 15, 2022
@driesvints driesvints linked an issue Feb 15, 2022 that may be closed by this pull request
@driesvints
Copy link
Member

As I said in #41016, I don't think there's a bug here. This PR will also be breaking for anyone using queues.

@boomhq
Copy link
Contributor Author

boomhq commented Feb 15, 2022

#24263

In this case this error thorwing by a failed job (Swift_TransportException). Not a manual throwing.
If you have a lead, I would like to know

@driesvints
Copy link
Member

@boomhq can you link to the code that causes this?

@taylorotwell
Copy link
Member

Can we use mb_convert_encoding($string, 'UTF-8');?

@taylorotwell
Copy link
Member

@driesvints imagine an error response from an external API (that is included in a Guzzle exception message) that is not in an encoding your database can store. That is the bug.

@boomhq
Copy link
Contributor Author

boomhq commented Feb 15, 2022

@boomhq can you link to the code that causes this?

For easy reproduce juste create a job with

    public function handle()
    {
        throw (new \Exception(utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ')));
    }

Can we use mb_convert_encoding($string, 'UTF-8');?

Yes mb_convert_encoding($string, 'UTF-8'); fix the issue too.
I think DatabaseUuidFailedJobProvider need to be fixed in the same way.

I would like your opinion on whether I should continue on this path.

@mfn
Copy link
Contributor

mfn commented Feb 15, 2022

The mb_convert_encoding() approach also works with pgsql in my tests:

$ ./artisan tinker
Psy Shell v0.11.1 (PHP 7.4.27 — cli) by Justin Hileman
>>> $m = \App\Model::findOrFail(123);null;
=> null
>>> $m->title = utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ');
=> b"ÐÑÙ0E\xE2\x?98\xA0World??7B¹!þÿ"
>>> $m->save();
Illuminate\Database\QueryException with title 'SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0xd0 0xd1 (SQL: update "model" set "title" = ���0E\xE2\x?98\xA0World??7B�!��, "modified" = 2022-02-15 20:26:18 where "id" = 123)'
>>> $m->title = mb_convert_encoding(utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ'), 'UTF-8');
=> "???0E\xE2\x?98\xA0World??7B?!??"
>>> $m->save();
=> true

Speaking of which: tests would be nice before merging :)

@boomhq
Copy link
Contributor Author

boomhq commented Feb 15, 2022

I'm currently trying to write the tests.

Sorry, I don't want to waste your time. I just hesitate because I want to do it right.

@boomhq boomhq force-pushed the patch-1 branch 2 times, most recently from 89aad5c to d9d9f05 Compare February 16, 2022 06:04
@boomhq
Copy link
Contributor Author

boomhq commented Feb 16, 2022

Test added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Queue Failed_jobs issue with Exception contain UNICODE
4 participants