Logging messages in its raw format

[suntong]

Following up on #31, if I want to collect different raw message strings received by openwechat and display them in terminal, what shall I do?

The purpose of this is that, when I got an attachment, say a .pdf file, I want to see if the file name is anywhere within the message itself. And same for other cases like getting pictures, or voices, etc.

You can use code in README or from #9 as the starting point. THX!

[eatmoreapple]

你可以通过访问msg.Content属性来获取当前文件消息相关的信息
下面是消息原始内容实例（图片消息）

{
"MsgId": "8733495435436186388",
"FromUserName": "@07023c3d24b6335fc03d2167bd99738cabe2f77f287c37cc1be180be278ee6d1",
"ToUserName": "@739ad0700033802d4834036bf53c5d151f9c5599a35acb2f16ce5ad1e94ed182",
"MsgType": 3,
"Content": "&lt;?xml version=\"1.0\"?&gt;<br/>&lt;msg&gt;<br/>\t&lt;img aeskey=\"0397f8bea20e2c408f6ef4532d45c72e\" encryver=\"0\" cdnthumbaeskey=\"0397f8bea20e2c408f6ef4532d45c72e\" cdnthumburl=\"30780201000471306f020100020430ff370f02033d120002040f403379020460f588db044a617570696d675f386531326461656238376637366335665f313632363730343039313331355f30306633633363372d366665362d343833372d393232642d613238373963396463663430020401152a010201000400\" cdnthumblength=\"6243\" cdnthumbheight=\"120\" cdnthumbwidth=\"90\" cdnmidheight=\"0\" cdnmidwidth=\"0\" cdnhdheight=\"0\" cdnhdwidth=\"0\" cdnmidimgurl=\"30780201000471306f020100020430ff370f02033d120002040f403379020460f588db044a617570696d675f386531326461656238376637366335665f313632363730343039313331355f30306633633363372d366665362d343833372d393232642d613238373963396463663430020401152a010201000400\" length=\"1\" cdnbigimgurl=\"30780201000471306f020100020430ff370f02033d120002040f403379020460f588db044a617570696d675f386531326461656238376637366335665f313632363730343039313331355f30306633633363372d366665362d343833372d393232642d613238373963396463663430020401152a010201000400\" hdlength=\"64848\" md5=\"76bd10b2da160f869e4cf07e9d236f55\" hevc_mid_size=\"15412\" /&gt;<br/>&lt;/msg&gt;<br/>",
"Status": 3,
"ImgStatus": 2,
"CreateTime": 1626704091,
"VoiceLength": 0,
"PlayLength": 0,
"FileName": "",
"FileSize": "",
"MediaId": "",
"Url": "",
"AppMsgType": 0,
"StatusNotifyCode": 0,
"StatusNotifyUserName": "",
"RecommendInfo": {
"UserName": "",
"NickName": "",
"QQNum": 0,
"Province": "",
"City": "",
"Content": "",
"Signature": "",
"Alias": "",
"Scene": 0,
"VerifyFlag": 0,
"AttrStatus": 0,
"Sex": 0,
"Ticket": "",
"OpCode": 0
}
,
"ForwardFlag": 0,
"AppInfo": {
"AppID": "",
"Type": 0
}
,
"HasProductId": 0,
"Ticket": "",
"ImgHeight": 120,
"ImgWidth": 90,
"SubMsgType": 0,
"NewMsgId": 8733495435436186388,
"OriContent": "",
"EncryFileName": ""
}

[suntong]

Thanks a lot!

[suntong]

Strange, why my msg.Content is in xml format (image and all else), while yours is in JSON?

Here is my msg.Content after receiving an image:

&lt;?xml version=\"1.0\"?&gt;<br/>&lt;msg&gt;<br/>\t&lt;img aeskey=\"93dc26e94e7764eb58621e7906321570\" encryver=\"0\" cdnthumbaeskey=\"93dc26e94e7764eb58621e7906321570\" cdnthumburl=\"30780201000471306f02010002048b8fde7b02033d096802045d8b2065020460f781ca044a617570696d675f623761333130636266336265663536365f313632363833333335333632395f62346139313765342d366162362d343463622d383432312d3730306362353532376665350204012d0a020201000400\" cdnthumblength=\"7486\" cdnthumbheight=\"107\" cdnthumbwidth=\"120\" cdnmidheight=\"0\" cdnmidwidth=\"0\" cdnhdheight=\"0\" cdnhdwidth=\"0\" cdnmidimgurl=\"30780201000471306f02010002048b8fde7b02033d096802045d8b2065020460f781ca044a617570696d675f623761333130636266336265663536365f313632363833333335333632395f62346139313765342d366162362d343463622d383432312d3730306362353532376665350204012d0a020201000400\" length=\"62022\" md5=\"2d585878ca83673d9852c61b790afd1b\" hevc_mid_size=\"62022\" /&gt;<br/>&lt;/msg&gt;<br/>

[suntong]

你可以通过访问msg.Content属性来获取当前文件消息相关的信息

Hmm, no, that's not the kind of raw format that I am talking about. For e.g., this is,

&lt;?xml version=\"1.0\"?&gt;<br/>&lt;msg&gt;<br/>\t&lt;appmsg appid=\"\" sdkver=\"0\"&gt;<br/>\t\t&lt;title&gt;经济学家调查：美国经济增速已见顶 将面临一系列独特风险&lt;/title&gt;<br/>\t\t&lt;des&gt;受调查的经济学家们表示，美国经济今年的整体增长速度将达到6.9%，而到2022年将降低到3.2%，到2023年则进一步降低到2.3%。&lt;/des&gt;<br/>\t\t&lt;username /&gt;<br/>\t\t&lt;action&gt;view&lt;/action&gt;<br/>\t\t&lt;type&gt;5&lt;/type&gt;

This is a type==5 message, and there are many more different types, like

• [Feature Request] type 51 & 54 appmsg please wechaty/puppet-padlocal#76, type 51 & 54
• [Feature Request] Handle the rest of appmsg please wechaty/puppet-padlocal#71, type 3
• Quoted text message type error: Unknown wechaty/puppet-padlocal#58, type 57

etc

[suntong]

Found the source,

https://github.com/eatMoreApple/openwechat/blob/b2178d32a28d61fb195adbc1575d574afbaed49f/parser.go#L108

Would it be possible that you cache this buffer.Bytes() as string inside msg as Payload or RawPayload please?

(or maybe before here):

https://github.com/eatMoreApple/openwechat/blob/b2178d32a28d61fb195adbc1575d574afbaed49f/caller.go#L208

[suntong]

The reason behind that is that,

I want to collect every possible raw messages openwechat can received, and use CI/CD automation to automatically generate Go date structure for them.

[eatmoreapple]

我觉得不用呀，因为它已经被转换了Message结构体了，再存储一份岂不是冗余的数据了吗，如果您想实现您的需求，再进行一次转换就好了呀

[suntong]

Hmm... the point is that I want to verify if the process of 转换成Message结构体 is solid.
E.g., There are many types, 3, 5, 51, 54 etc, are they all have been converted exactly as they should be?

Basically different types (3, 5, 51, 54 etc) have totally different date structures, as we can see from the above linked examples, I want to verify if they indeed have been converted properly. I.e., it is hard to believe that different types, different date structures, can all be converted to the same Message结构体 properly.

[suntong]

For e.g., this is straight from msg.Content, not quite being converted properly.

&lt;?xml version=\"1.0\"?&gt;<br/>&lt;msg&gt;<br/>\t&lt;appmsg appid=\"\" sdkver=\"0\"&gt;<br/>\t\t&lt;title&gt;经济学家调查：美国经济增速已见顶 将面临一系列独特风险&lt;/title&gt;<br/>\t\t&lt;des&gt;受调查的经济学家们表示，美国经济今年的整体增长速度将达到6.9%，而到2022年将降低到3.2%，到2023年则进一步降低到2.3%。&lt;/des&gt;<br/>\t\t&lt;username /&gt;<br/>\t\t&lt;action&gt;view&lt;/action&gt;<br/>\t\t&lt;type&gt;5&lt;/type&gt; . . .

[suntong]

Right after login, I'd receive at least two messages:

TS=0724T16:17:03 Msg=收到消息 type="51 (0,0)" content=
TS=0724T16:17:08 Msg=收到消息 type="51 (0,0)" content=

I wish to see the raw Payload of the above 2 messages as well.

I tried to make the following simple changes:

$ git diff caller.go 
diff --git a/caller.go b/caller.go

index f807d00..f43b3e0 100644
--- a/caller.go
+++ b/caller.go
@@ -3,6 +3,7 @@ package openwechat
 import (
        "bytes"
        "errors"
+       "fmt"
        "net/http"
        "net/url"
        "os"
@@ -195,6 +196,13 @@ func (c *Caller) WebWxSync(request *BaseRequest, response *WebInitResponse, info
                return nil, err
        }
        defer resp.Body.Close()
+
+       var buffer bytes.Buffer
+       if _, err := buffer.ReadFrom(resp.Body); err != nil {
+               return nil, err
+       }
+       fmt.Println("PL: ", string(buffer.Bytes()))
+
        var webWxSyncResponse WebWxSyncResponse
        if err := scanJson(resp, &webWxSyncResponse); err != nil {
                return nil, err

But my code would hang because of that. Verified many times by flipping them on & off.

Don't know why it is that, :(.

It'll be nice to see the raw Payload, but let me park the idea for a while...

[eatmoreapple]

会不会是你上面将Body读完了，然后下面再去读就读取不了了

[suntong]

Ah, it must be then. THX!

[suntong]

I still need such raw-format dumping function badly. @eatmoreapple, please think of a way so that I can get it.

So far, the best way I can think of is to define a global variable, and set it within scanJson after buffer.ReadFrom(resp.Body), then define a new API, GetLastResponseBodyRaw() or something so that I can read from this global variable.

@eatmoreapple, please implement anything so that I can get the raw message strings received from the WX server without any modification. This is for my #64 implementation. Thanks!

[suntong]

Hmm... hold on, let me investigate a fiddler-recording-alike solution first...