Skip to content

[9.x] Fix Queue Failed_jobs insert issue with Exception contain UNICODE#41020

Merged
taylorotwell merged 1 commit into
laravel:9.xfrom
boomhq:patch-1
Feb 16, 2022
Merged

[9.x] Fix Queue Failed_jobs insert issue with Exception contain UNICODE#41020
taylorotwell merged 1 commit into
laravel:9.xfrom
boomhq:patch-1

Conversation

@boomhq

@boomhq boomhq commented Feb 15, 2022

Copy link
Copy Markdown
Contributor

Can't Insert failed job exception with UNICODE character.
Add an "utf8_encode" front of exception.

If you have a better way, my ears is open
I deal manually with this case from laravel 5.5

Issues :
#24263
#41016

@boomhq boomhq changed the title [9.x] Fix Queue failed logging exception UNICODE [9.x] Fix Queue Failed_jobs insert issue with Exception contain UNICODE Feb 15, 2022
@driesvints driesvints linked an issue Feb 15, 2022 that may be closed by this pull request
@driesvints

Copy link
Copy Markdown
Member

As I said in #41016, I don't think there's a bug here. This PR will also be breaking for anyone using queues.

@boomhq

boomhq commented Feb 15, 2022

Copy link
Copy Markdown
Contributor Author

#24263

In this case this error thorwing by a failed job (Swift_TransportException). Not a manual throwing.
If you have a lead, I would like to know

@driesvints

Copy link
Copy Markdown
Member

@boomhq can you link to the code that causes this?

@taylorotwell

Copy link
Copy Markdown
Member

Can we use mb_convert_encoding($string, 'UTF-8');?

@taylorotwell

Copy link
Copy Markdown
Member

@driesvints imagine an error response from an external API (that is included in a Guzzle exception message) that is not in an encoding your database can store. That is the bug.

@boomhq

boomhq commented Feb 15, 2022

Copy link
Copy Markdown
Contributor Author

@boomhq can you link to the code that causes this?

For easy reproduce juste create a job with

    public function handle()
    {
        throw (new \Exception(utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ')));
    }

Can we use mb_convert_encoding($string, 'UTF-8');?

Yes mb_convert_encoding($string, 'UTF-8'); fix the issue too.
I think DatabaseUuidFailedJobProvider need to be fixed in the same way.

I would like your opinion on whether I should continue on this path.

@mfn

mfn commented Feb 15, 2022

Copy link
Copy Markdown
Contributor

The mb_convert_encoding() approach also works with pgsql in my tests:

$ ./artisan tinker
Psy Shell v0.11.1 (PHP 7.4.27 — cli) by Justin Hileman
>>> $m = \App\Model::findOrFail(123);null;
=> null
>>> $m->title = utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ');
=> b"ÐÑÙ0E\xE2\x?98\xA0World??7B¹!þÿ"
>>> $m->save();
Illuminate\Database\QueryException with title 'SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding "UTF8": 0xd0 0xd1 (SQL: update "model" set "title" = ���0E\xE2\x?98\xA0World??7B�!��, "modified" = 2022-02-15 20:26:18 where "id" = 123)'
>>> $m->title = mb_convert_encoding(utf8_decode ('ÐÑÙ0E\xE2\x�98\xA0World��7B¹!þÿ'), 'UTF-8');
=> "???0E\xE2\x?98\xA0World??7B?!??"
>>> $m->save();
=> true

Speaking of which: tests would be nice before merging :)

@boomhq

boomhq commented Feb 15, 2022

Copy link
Copy Markdown
Contributor Author

I'm currently trying to write the tests.

Sorry, I don't want to waste your time. I just hesitate because I want to do it right.

@boomhq boomhq force-pushed the patch-1 branch 2 times, most recently from 89aad5c to d9d9f05 Compare February 16, 2022 06:04
@boomhq

boomhq commented Feb 16, 2022

Copy link
Copy Markdown
Contributor Author

Test added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Queue Failed_jobs issue with Exception contain UNICODE

4 participants