Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COMMON: Skip possible UTF-8 BOM when reading INI files #5435

merged 1 commit into from Nov 18, 2023


Copy link

@OMGPizzaGuy OMGPizzaGuy commented Nov 15, 2023

After briefly considering a more complex solution including detection additional encoding byte-order marks, I thought it would be best to simply read and skip UTF-8 BOM when reading INI files. This would at least prevent someone like me from accidentally wiping out their scummvm.ini files.

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of UTF-8 byte-order mark (BOM) in configuration files to ensure proper parsing and functionality.

Copy link

coderabbitai bot commented Nov 15, 2023


The updates across multiple files introduce a new feature to handle the UTF-8 byte-order mark (BOM) in configuration and INI file parsing. When loading from a stream, the system now checks for and removes the BOM to ensure the subsequent data is interpreted correctly. This change is consistent across the ConfigManager and INIFile classes, improving the robustness of file parsing without altering any external interfaces or function signatures.


File Path Change Summary
common/config-manager.cpp Introduced UTF-8 BOM detection and removal in loadFromStream function.
common/formats/ini-file.cpp Added check for UTF-8 BOM and its removal in loadFromStream function of INIFile class.

🐇🍂 In the code where bytes do stream,
A rabbit fixed the BOM unseen.
Now files parse clean, with nary a mark,
As autumn leaves dance in the park. 🍁✨


Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=

Copy link

sev- commented Nov 15, 2023

How do these characters appear in the file in the first place? After editing with an external editor?

Copy link
Member Author

Yes, it was added with an external editor.

Copy link

This is a good idea and I think the simple approach is appropriate. I think the comment should include that external editors can be the source, and maybe the consequences of not stripping it. Otherwise the code will keep raising those questions.

I expected coderabbit to suggest sizeof(UTF8_BOM) instead of 3! =)

Copy link
Member Author

To be fair, I was specifically testing UTF-8 with BOM when trying to determine why u8.ini wasn't being read from ultima8.dat. I just did not expect to be prompted to replace an invalid scummvm.ini or that exiting instead of specifically clicking cancel would replace it.

In a previous job where we implemented UTF-8 reading of INI configuration files, we relied on the BOM to indicate that it should not be read with the system codepage. My understanding is this has become uncommon as UTF-8 BOM is discouraged.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between ade01bd and 0f45263.
Files selected for processing (2)
  • common/config-manager.cpp (2 hunks)
  • common/formats/ini-file.cpp (2 hunks)
Files skipped from review due to trivial changes (2)
  • common/config-manager.cpp
  • common/formats/ini-file.cpp

Copy link

sev- commented Nov 18, 2023

Okay, thank you!

@sev- sev- merged commit 5f75054 into scummvm:master Nov 18, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet
3 participants