Error parsing plex tv library - Malformed UTF-8 characters, possibly incorrectly encoded '}' At position 0.

[MagnuM2K]

Hi

When I try to execute the first command to pull watchstate.. I get the following error. Unable to proceed further.

docker-compose exec watchstate console state:import -vvrm -s plx
[2022-04-22T01:41:59+00:00] logger.INFO: [M: 3.10M] Preloading all mapper data.
[2022-04-22T01:41:59+00:00] logger.INFO: [M: 7.33M] Finished preloading mapper data.
[2022-04-22T01:41:59+00:00] logger.INFO: [M: 7.33M] Single transaction mode
[2022-04-22T01:41:59+00:00] logger.NOTICE: [M: 9.19M] Importing 'plx' play state changes since beginning.
[2022-04-22T01:41:59+00:00] logger.NOTICE: [M: 9.40M] Waiting on (3) HTTP Requests.
[2022-04-22T01:42:00+00:00] logger.INFO: [M: 9.69M] Parsing Successful plx - (Movies) - (movie:3) response.
[2022-04-22T01:42:01+00:00] logger.INFO: [M: 11.20M] Finished Parsing plx - (Movies) - (movie:3) (1011 objects) response.
[2022-04-22T01:42:01+00:00] logger.INFO: [M: 11.19M] Parsing Successful plx - (Movies 4K) - (movie:2) response.
[2022-04-22T01:42:01+00:00] logger.INFO: [M: 11.21M] Finished Parsing plx - (Movies 4K) - (movie:2) (7 objects) response.
[2022-04-22T01:42:03+00:00] logger.INFO: [M: 11.23M] Parsing Successful plx - (TV Shows) - (episode:4) response.

In Parser.php line 368:
                                                                               
  [JsonMachine\Exception\SyntaxErrorException]                                 
  Malformed UTF-8 characters, possibly incorrectly encoded '}' At position 0.  
                                                                               

Exception trace:
  at /app/vendor/halaxa/json-machine/src/Parser.php:368
 JsonMachine\Parser->error() at /app/vendor/halaxa/json-machine/src/Parser.php:220
 JsonMachine\Parser->getIterator() at /app/src/Libs/Servers/PlexServer.php:631
 App\Libs\Servers\PlexServer->App\Libs\Servers\{closure}() at /app/src/Commands/State/ImportCommand.php:252
 App\Commands\State\ImportCommand->process() at /app/src/Commands/State/ImportCommand.php:96
 App\Commands\State\ImportCommand->App\Commands\State\{closure}() at /app/src/Command.php:84
 App\Command->single() at /app/src/Commands/State/ImportCommand.php:96
 App\Commands\State\ImportCommand->runCommand() at /app/src/Command.php:22
 App\Command->execute() at /app/vendor/symfony/console/Command/Command.php:291
 Symfony\Component\Console\Command\Command->run() at /app/vendor/symfony/console/Application.php:989
 Symfony\Component\Console\Application->doRunCommand() at /app/vendor/symfony/console/Application.php:299
 Symfony\Component\Console\Application->doRun() at /app/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /app/src/Libs/Initializer.php:117
 App\Libs\Initializer->runConsole() at /app/console:66

state:import [-r|--redirect-logger] [-m|--memory-usage] [-f|--force-full] [--proxy PROXY] [--no-proxy NO-PROXY] [--timeout TIMEOUT] [-s|--servers-filter [SERVERS-FILTER]] [--import-unwatched] [--stats-show] [--stats-filter [STATS-FILTER]] [--mapper-direct] [-c|--config CONFIG]

[arabcoders]

Hmm, it seems to be able to parse your movies libraries fine and choke on your tv library, is your frontend server for plex doing compression? could you try running the following commands and then try to sync

docker exec -ti watchstate console servers:edit --key 'options.client.headers.Accept-Encoding' --set '' -- plx
docker exec -ti watchstate console servers:edit --key 'options.client.http_version' --set '1.0' -- plx

[MagnuM2K]

My server sits behind a reverse proxy, Caddy.
But when I tested the server I used both the direct http address and also the https link.

My current set is as below using http and when I ran those commands, I get the same error.

Here is my servers.yaml

plx:
  type: plex
  url: 'http://192.168.2.107:32400'
  token: <removed>
  uuid:  <removed>
  user: <removed>
  import:
    enabled: true
  export:
    enabled: true
  webhook:
    import: true
    push: true
    match:
      user: true
      uuid: true
    token: <removed>
  options:
    client:
      headers:
        Accept-Encoding: ''
      http_version: '1.0'

[2022-04-22T17:10:21+00:00] logger.INFO: [M: 38.71M] Parsing Successful plx - (Movies) - (movie:3) response.
[2022-04-22T17:10:22+00:00] logger.INFO: [M: 38.07M] Finished Parsing plx - (Movies) - (movie:3) (1011 objects) response.
[2022-04-22T17:10:22+00:00] logger.INFO: [M: 38.06M] Parsing Successful plx - (Movies 4K) - (movie:2) response.
[2022-04-22T17:10:22+00:00] logger.INFO: [M: 38.07M] Finished Parsing plx - (Movies 4K) - (movie:2) (7 objects) response.
[2022-04-22T17:10:22+00:00] logger.INFO: [M: 38.06M] Parsing Successful plx - (TV Shows) - (episode:4) response.

In Parser.php line 368:
                                                                               
  [JsonMachine\Exception\SyntaxErrorException]                                 
  Malformed UTF-8 characters, possibly incorrectly encoded '}' At position 0.  
                                                                               

Exception trace:
  at /app/vendor/halaxa/json-machine/src/Parser.php:368
 JsonMachine\Parser->error() at /app/vendor/halaxa/json-machine/src/Parser.php:220
 JsonMachine\Parser->getIterator() at /app/src/Libs/Servers/PlexServer.php:631
 App\Libs\Servers\PlexServer->App\Libs\Servers\{closure}() at /app/src/Commands/State/ImportCommand.php:252
 App\Commands\State\ImportCommand->process() at /app/src/Commands/State/ImportCommand.php:96
 App\Commands\State\ImportCommand->App\Commands\State\{closure}() at /app/src/Command.php:84
 App\Command->single() at /app/src/Commands/State/ImportCommand.php:96
 App\Commands\State\ImportCommand->runCommand() at /app/src/Command.php:22
 App\Command->execute() at /app/vendor/symfony/console/Command/Command.php:291
 Symfony\Component\Console\Command\Command->run() at /app/vendor/symfony/console/Application.php:989
 Symfony\Component\Console\Application->doRunCommand() at /app/vendor/symfony/console/Application.php:299
 Symfony\Component\Console\Application->doRun() at /app/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /app/src/Libs/Initializer.php:117
 App\Libs\Initializer->runConsole() at /app/console:66

state:import [-r|--redirect-logger] [-m|--memory-usage] [-f|--force-full] [--proxy PROXY] [--no-proxy NO-PROXY] [--timeout TIMEOUT] [-s|--servers-filter [SERVERS-FILTER]] [--import-unwatched] [--stats-show] [--stats-filter [STATS-FILTER]] [--mapper-direct] [-c|--config CONFIG]

[arabcoders]

Alright, could you please try running this command to see plex response, it seems something happening in tv response,

curl -v -H "Accept: application/json" -H "X-Plex-Token: YOUR_PLEX_TOKEN_HERE" 'http://192.168.2.107:32400/library/sections/4/all?type=4&sort=addedAt:asc&includeGuids=1' 2>&1 | tee response.log

it should save a log in your directory called response.log, please edit the log to remove your plex token, after that please attached it here or send it to me at (ws-contact at arabcoders.net)

i need to see the response headers and body from plex to determine why the json parser is failing on this specific library.

[MagnuM2K]

Ok. Just sent you the response log via email.

[arabcoders]

I've successfully parsed your given response, the only thing i noticed that might be a problem is there is invalid UTF-8 characters in your payload. i can probably fix that, and you are still using the old plex agents. which most likely will cause you problems in the future however they are "kinda" supported right now i'll see what i can do. i'll let you know once i have something hopefully by later today or tomorrow.

[arabcoders]

i pushed possible fix for this issue, in an hour i want you to switch to arabcoders/watchstate:dev-latest pull the new container and restart after that try importing

[MagnuM2K]

Ok. Will try and report back in an hour.

I checked my plex agents and they seem to be using the new agents. There is no option to upgrade like stated in the plx server site. My agents are.. The TV agent is : Plex TV Series and movies is : Plex Movie.

Are these are the correct agents?

Thx

[arabcoders]

Indeed those are correct new agents, however it seems you possibly forgot to run full meta refresh on library. Anyway, the dev build is out try now and report back if it fixes your problem.

[MagnuM2K]

I've refreshed my Plex libraries and downloaded new dev image. But still same.

Also ran db list against plx server, and see a small set of watched status.

Any way to get a count from db and compare plx vs jellyfin?

[2022-04-22T19:10:52+00:00] logger.DEBUG: Ignoring plx - (TV Shows) - (episode:4) - [Star Trek: Voyager - (3x11) - The Q and the Grey]. Not watched.
[2022-04-22T19:10:52+00:00] logger.DEBUG: Ignoring plx - (TV Shows) - (episode:4) - [Star Trek: Voyager - (3x7) - Sacred Ground]. Not watched.
[2022-04-22T19:10:52+00:00] logger.DEBUG: Ignoring plx - (TV Shows) - (episode:4) - [Star Trek: Voyager - (4x6) - The Raven]. Not watched.
[2022-04-22T19:10:52+00:00] logger.DEBUG: Ignoring plx - (TV Shows) - (episode:4) - [Star Trek: Voyager - (4x8) - Year of Hell, Part 1]. Not watched.
[2022-04-22T19:10:52+00:00] logger.DEBUG: Ignoring plx - (TV Shows) - (episode:4) - [Warehouse 13 - (1x5) - Elements]. Not watched.
[2022-04-22T19:10:52+00:00] logger.ERROR: Unable to parse plx - (TV Shows) - (episode:4) response. [(file: /app/vendor/halaxa/json-machine/src/Parser.php), (line: 368), (kind: JsonMachine\Exception\SyntaxErrorException), (error: Malformed UTF-8 characters, possibly incorrectly encoded '}' At position 0.)]
[2022-04-22T19:10:52+00:00] logger.NOTICE: Finished waiting HTTP Requests.
[2022-04-22T19:10:52+00:00] logger.INFO: No changes detected.
┌─────────┬───────┬─────────┬────────┐
│ Type    │ Added │ Updated │ Failed │
├─────────┼───────┼─────────┼────────┤
│ Movie   │ 0     │ 0       │ 0      │
├─────────┼───────┼─────────┼────────┤
│ Episode │ 0     │ 0       │ 0      │
└─────────┴───────┴─────────┴────────┘

[arabcoders]

Ok, i pushed another fix to dev-latest should be done building in <30min after that pull restart and try again. if that also does not work, im afraid im out of ideas it's clearly one an item that is not valid UTF as you can see from the log it can parse some items and then bails.

[arabcoders]

it seems this file "/home/media/TVShows/Show.Title/Season.04/ShowTitle s04e20 - Vis � Vis NTSC DVD x264 DD5.1-JCH.mkv" causing the problems. it's the character between vis and Vis it's incorrectly encoded do you not use media manger like sonarr ?

Star Trek: Voyager S04E20

[arabcoders]

nevermind, please try next build. it seems i messed up.

[arabcoders]

@MagnuM2K the new builds are out, please try and report again pull and restart and try to import, now it should ONLY ignore the malformed item instead of bailing out completely on all other items.

[MagnuM2K]

That seemed to have done the trick.
Now that I know which file was causing the issue, I've rename it as well.

Thx!

[arabcoders]

Np. glad it worked for you.