Lack of documentation on SPT(json) split into CSVs to be fed to trained GNN model for code classification experiment

[singlelionwith2cubs]

Hi,

Created SPT from https://github.com/IBM/Project_CodeNet/tree/main/tools/spt-generator for given sample C code.

Nitins-MacBook-Air:spt-generator nitinnanda$ ./scripts/run/spt-gen.sh -d=/Users/nitinnanda/Downloads/SPT ./examples/c/helloworld.c
/Users/nitinnanda/Downloads/SPT/helloworld.json is generated!
/Users/nitinnanda/Downloads/SPT/helloworld.csv is generated.
Nitins-MacBook-Air:spt-generator nitinnanda$ 

Created trained model from https://github.com/IBM/Project_CodeNet/tree/main/model-experiments/gnn-based-experiments

(pyg) Nitins-MacBook-Air:saved_models nitinnanda$ ls -ltr
total 11400
-rw-r--r--  1 nitinnanda  staff  5834983 Jun 21 18:07 gcn_lr1e-3_10_23.pt
(pyg) Nitins-MacBook-Air:saved_models nitinnanda$ 

Now the SPT here helloworld.json needs to be split into CSVs to feed to trained model gcn_lr1e-3_10_23.pt however, there is very little to no documentation given on this split.

Please guide.

[vthost]

We created .csv files that describe the graph structure (see also the files used in the Open Graph Benchmark). As mentioned in the README, the files are provided (https://developer.ibm.com/technologies/artificial-intelligence/data/project-codenet/) and do not need to be generated.
In detail, the files describe the following:

(1) edge.csv:
Each line is a pair of (source node ID, target node ID)
(2) node-feat.csv:
Each line is a feature vector of a node. Each node contains 4 features: (i) is it a token (ii) token type (iii) parsing rule type (iv) is it a reserved word
(3) node_dfs_order.csv:
Each line is the DFS ID of a node.
(4) num-edge-list.csv:
Each line is the number of edges in each SPT.
(5) graph-label.csv:
Each line is the class of a SPT.
(6) node_depth.csv:
Each line is the distance between the corresponding node and its SPT root.
(7) node_is_attributed.csv
Each line indicates if the node is a leaf node or an internal node.
(8) num-node-list.csv
Each line is the number of nodes in a SPT.

[singlelionwith2cubs]

But these are for the train set/SPT against entire 1k dataset (as given in the experiment). If I have to train using my own custom SPT created (in this case for ex. helloworld.c )using spt-generator I'll have to split it into these csvs. From that perspective the documentation for splitting of custom SPTs into csvs is not given.

[vthost]

Yes, you are right. This repository is focusing on experiments over our Project CodeNet datasets. It is beyond the scope of our work to cover arbitrary data. For the latter, please look at more general frameworks such as DLG, Pytorch Geometric, etc.

Nevertheless, the above description of the files should enable you to process your graphs.

[singlelionwith2cubs]

Thanks. Request if we could please consider a few scenarios being covered end to end ex. code classification, code-to-code translation and clone detection with flexibility to support BYOSC will actually make this entire preposition more real & acceptable on the ground for accelerating Hybrid cloud pitch.

[vthost]

Thank you for these suggestions!

Please note that the current model-experiments are intended to showcase example uses of our datasets. We lack the resources to provide extended support for custom data.

[weidotwisc]

Hi, The label is *not* a node-level label. The label is the Graph-level label (aka which problem this piece of code tries to solve). Please see Table 6 of the paper at https://arxiv.org/pdf/2105.12655.pdf: for the 4 benchmarks that we provided, there are 1000 classes for C++1000, 1400 classes for C++1400 , 800 classes for Python800 and 250 classes for Java250. Table 7 of the same paper reports classification results on these benchmarks. Thanks! Wei

…

On Fri, Mar 11, 2022 at 12:31 AM Han Peng ***@***.***> wrote: Hi, I have some questions about the node-feat.csv. As you say, this file has 4 fields, : (i) is it a token (ii) token type (iii) parsing rule type (iv) is it a reserved word So, here is my understanding: for a JSON file (for example: https://github.com/IBM/Project_CodeNet/blob/main/tools/spt-generator/examples/demos/c/helloworld.json ) the (i) is about the "node-type", for "Rule" vs."Token" the (iii) is about the "type-rule-name" or "type-rule-index", the (iv) is about the "reserved-word-flag", right? So the (ii) is about the "label"? I mean the token label idx in the token vocab? But puzzlingly, I count the max number for this item in the node-feat.csv file and only get about ~200. While the max vocab size should be 5000 in your paper (perhaps I missed the details in your paper). So why? — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA36LKHRTG5FRZXDF6S3DCTU7LLBDANCNFSM47DZO6XQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[geert56]

Dear Han, I cannot find your question on github. Did you file an issue? Or re-open an old one?Anyway, as designer of the JSON-Graph format, all I can say is that the "label"field for a node intends to give a human-readable description of it. So it should not be a number. The "id" field typically is a number counted from 0 for the identification of a node.A JSON file represents a source code file, and must contain all tokens. The vocabulary isthe grand total union over all tokens I guess. I am not familiar with uses of the JSON files. Any derived data formats should have been documentedseparately. Geert On Friday, March 11, 2022, 12:31:03 AM EST, Han Peng ***@***.***> wrote: Hi, I have some questions about the node-feat.csv. As you say, this file has 4 fields, : (i) is it a token (ii) token type (iii) parsing rule type (iv) is it a reserved word So, here is my understanding: for a JSON file (for example: https://github.com/IBM/Project_CodeNet/blob/main/tools/spt-generator/examples/demos/c/helloworld.json) the (i) is about the "node-type", for "Rule" vs."Token" the (iii) is about the "type-rule-name" or "type-rule-index", the (iv) is about the "reserved-word-flag", right? So the (ii) is about the "label"? I mean the token label idx in the token vocab? But puzzlingly, I count the max number for this item in the node-feat.csv file and only get about ~200. While the max vocab size should be 5000 in your paper (perhaps I missed the details in your paper). So why? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[weidotwisc]

Hi Han, Maybe I have misunderstood your question. There is no "label" in node-feat.csv. If by "label" you mean the token-type in node-feat.csv, then it is the type of tokens (e.g., identifier). We didn't create a vocabulary for the GNN experiments -- as a result all identifiers have the same token-type -- it is indeed an area one could improve on! Thanks! Wei On Fri, Mar 11, 2022 at 10:03 AM Geert Janssen ***@***.***> wrote:

…

Dear Han, I cannot find your question on github. Did you file an issue? Or re-open an old one?Anyway, as designer of the JSON-Graph format, all I can say is that the "label"field for a node intends to give a human-readable description of it. So it should not be a number. The "id" field typically is a number counted from 0 for the identification of a node.A JSON file represents a source code file, and must contain all tokens. The vocabulary isthe grand total union over all tokens I guess. I am not familiar with uses of the JSON files. Any derived data formats should have been documentedseparately. Geert On Friday, March 11, 2022, 12:31:03 AM EST, Han Peng ***@***.***> wrote: Hi, I have some questions about the node-feat.csv. As you say, this file has 4 fields, : (i) is it a token (ii) token type (iii) parsing rule type (iv) is it a reserved word So, here is my understanding: for a JSON file (for example: https://github.com/IBM/Project_CodeNet/blob/main/tools/spt-generator/examples/demos/c/helloworld.json ) the (i) is about the "node-type", for "Rule" vs."Token" the (iii) is about the "type-rule-name" or "type-rule-index", the (iv) is about the "reserved-word-flag", right? So the (ii) is about the "label"? I mean the token label idx in the token vocab? But puzzlingly, I count the max number for this item in the node-feat.csv file and only get about ~200. While the max vocab size should be 5000 in your paper (perhaps I missed the details in your paper). So why? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA36LKBSXTG6UKIHGNPKAJTU7NOC7ANCNFSM47DZO6XQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>