Some utf8 characters may become "???', if polling is used

[HenryOrz]

Hi,
I found that there is something confusing at com.corundumstudio.socketio.protocol.PacketDecoder, it may convert some utf8 characters to "???"
I create an issue

// TODO optimize
public ByteBuf preprocessJson(Integer jsonIndex, ByteBuf content) throws IOException {
    String packet = URLDecoder.decode(content.toString(CharsetUtil.UTF_8), CharsetUtil.UTF_8.name());

    if (jsonIndex != null) {
        /**
        * double escaping is required for escaped new lines because unescaping of new lines can be done safely on server-side
        * (c) socket.io.js
        *
        * @see https://github.com/Automattic/socket.io-client/blob/1.3.3/socket.io.js#L2682
        */
        packet = packet.replace("\\\\n", "\\n");

        // skip "d="
        packet = packet.substring(2);
    }
    packet = new String(packet.getBytes(CharsetUtil.ISO_8859_1), CharsetUtil.UTF_8);

    return Unpooled.wrappedBuffer(packet.getBytes(CharsetUtil.UTF_8));
}

this line is very confusing:
packet = new String(packet.getBytes(CharsetUtil.ISO_8859_1), CharsetUtil.UTF_8);

some other detail I record in the issue515

[mrniko]

Hi! Thanks for your contribution. How about this line:

https://github.com/HenryOrz/netty-socketio/blob/9414a990af7e4b146e61c181c3a9261645bc9dc3/src/main/java/com/corundumstudio/socketio/protocol/PacketDecoder.java#L146

Does it affect too?

[HenryOrz]

Thanks! I'm not sure about this line.
In my test case, the message has been converted to "???" before calling this line
But I think it maybe affect in some cases.
I will try to check it

[HenryOrz]

Hi, this line affect too:
https://github.com/HenryOrz/netty-socketio/blob/9414a990af7e4b146e61c181c3a9261645bc9dc3/src/main/java/com/corundumstudio/socketio/protocol/PacketDecoder.java#L146
I removed it. And it worked in my test case

[mrniko]

Thank you!

[fcbflying]

@HenryOrz The polling is used, I removed the line, and it works when send message from client to server. But when sending utf8 characters from server to client, the client received '中���'. If you also have the same issue?

[fengsuxing]

@HenryOrz The polling is used, I removed the line, and it works when send message from client to server. But when sending utf8 characters from server to client, the client received '中���'. If you also have the same issue?

HAVE YOU SOLVE IT?

[zhaoxiangyu]

@HenryOrz The polling is used, I removed the line, and it works when send message from client to server. But when sending utf8 characters from server to client, the client received '中���'. If you also have the same issue?

HAVE YOU SOLVE IT?

HAVE YOU SOLVE IT?

[fcbflying]

@HenryOrz The polling is used, I removed the line, and it works when send message from client to server. But when sending utf8 characters from server to client, the client received '中���'. If you also have the same issue?

HAVE YOU SOLVE IT?

HAVE YOU SOLVE IT?

It works now, but I still rollback from v1.7.18 to v1.7.12. Because I got another bug in v1.7.18, sometimes when reconnect, it still use the previous SocketIOClient rather than create a new one.