Fix JSONDecodeError in JournalStorage

[y0z]

Motivation

• Fix JSONDecodeError in JournalStorage JSONDecodeError in JournalStorage #5138

Description of the changes

• Maintain remaining_log_size to allow writing by another process while reading the log.

[codecov]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (45a9dc9) 89.41% compared to head (33fc7e9) 89.35%.
Report is 38 commits behind head on master.

Files	Patch %	Lines
optuna/storages/_journal/file.py	60.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5195      +/-   ##
==========================================
- Coverage   89.41%   89.35%   -0.07%     
==========================================
  Files         206      206              
  Lines       15175    15106      -69     
==========================================
- Hits        13569    13498      -71     
- Misses       1606     1608       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ernestum]

Hi thanks for fixing this. I just did a test run of this branch on the cluster with 100 parallel workers and observed no more crashes :-)

[c-bata]

Thank you for your pull request. I left two comments.

[c-bata]

The changes look good to me, but a second thorough review would be helpful.

[c-bata]

LGTM. Thanks for the update!

[y0z]

I applied all the comments. PTAL.

[not522]

Sorry, I realize that my previous comment (using os.linesep) doesn't work on Windows. It is because the line separator for what_to_write.encode("utf-8") is \n instead of \r\n. Could you revert the last commit?

[y0z]

I see. I've reverted the change.

[not522]

Thank you for the update. LGTM.