Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

解析Json时,是否添加对UTF8-Bom文件的支持 #40

Closed
XingYunSong opened this issue Jul 13, 2023 · 13 comments
Closed

解析Json时,是否添加对UTF8-Bom文件的支持 #40

XingYunSong opened this issue Jul 13, 2023 · 13 comments

Comments

@XingYunSong
Copy link

XingYunSong commented Jul 13, 2023

是否能添加对UTF8-Bom文件的支持 xpack\json_decoder.h

 JsonDecoder(const std::string& str, bool isfile=false):xdoc_type(NULL, ""),_doc(new rapidjson::Document),_val(_doc) {
    std::string err;
    std::string data;

    do {
        if (isfile) {
            std::ifstream fs(str.c_str(), std::ifstream::binary);
            if (!fs) {
                err = "Open file["+str+"] fail.";
                break;
            }
            std::string _tmp((std::istreambuf_iterator<char>(fs)), std::istreambuf_iterator<char>());


            if (_tmp[0] == '\xEF' && _tmp[1] == '\xBB' && _tmp[2] == '\xBF')//if this formate is utf8-bom
            {
                data = _tmp.substr(3, _tmp.length() - 3);
            }
            else
            {
                data.swap(_tmp);
            }


            _doc->Parse(data);
        } else  {
            _doc->Parse(str);
        }
@xyz347
Copy link
Owner

xyz347 commented Jul 13, 2023

现在就支持bom

@XingYunSong
Copy link
Author

在windows下解析UTF8带BOM的文件时就会报下面错误,就是没有支持BOM文件,去掉BOM的头就可以解析正确了
strError | "Parse json file "cfg/DataBrowserConfig.json" fail. err=Invalid value.. offset=锘縶\r\n\t"Description":"A Simulati" | std::basic_string<char,std::char_traits,std::allocator >

@xyz347
Copy link
Owner

xyz347 commented Jul 13, 2023

这个和bom没关系,应该是编码导致的,现在不支持gbk编码

@XingYunSong
Copy link
Author

图片
通过notpad++查看文件确定为UTF-8-BOM编码

@xyz347
Copy link
Owner

xyz347 commented Jul 13, 2023

贴一下json

@XingYunSong
Copy link
Author

{
"Description":"A Simulation Program",
"Version":"V1.0",
"DomainId":23
}

@XingYunSong
Copy link
Author

解析用的
xpack::json::decode_file("cfg/DataBrowserConfig.json", sConfigInfo);

@xyz347
Copy link
Owner

xyz347 commented Jul 13, 2023

我明天看看

@XingYunSong
Copy link
Author

XingYunSong commented Jul 13, 2023

像最开始那样在解析文件函数里面添加删除BOM头就可以成功解析,最开始里面有代码。好的,谢谢!

@xyz347
Copy link
Owner

xyz347 commented Jul 14, 2023

试了一下,Ubuntu(gcc)/Windows(msvc 2019)都是正常的,rapidjson处理了BOM了:https://github.com/xyz347/xpack/blob/master/rapidjson/encodedstream.h#L72

@XingYunSong
Copy link
Author

XingYunSong commented Jul 14, 2023

我用的VS2017
此处定义了一个strDeleteBom字符串,去掉BOM头,看到Json是正常的,作为对比
图片
通过两张图对比,当报错时,数据仍然是未去掉BOM头的内容,并且过程中没有调用到encodedstream.h#72这里
图片

@xyz347 xyz347 reopened this Jul 17, 2023
@xyz347
Copy link
Owner

xyz347 commented Jul 17, 2023

用最新的xpack的代码

@XingYunSong
Copy link
Author

最新版本是可以的,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants