-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues on the performance #3
Comments
@VoidHaruhi what did you use for 'data/twitter/ent2id.txt' and 'twitter.emd' please? |
@Shreyas-Bhat "ent2id.txt" is an original data file, data preprocess use this file to convert entity number to id in the graph. The command line "--pretrained_embeddings='./embed/twitter/twitter.emd'" means that you save the pretrained embedding into which directory |
@VoidHaruhi Thanks! where can I get this "ent2id.txt" and the pre-trained embedding? |
@Shreyas-Bhat I download the dataset from https://github.com/THUDM/GATNE and put the data directory "./GATNE/data/" into "./SLiCE/data/". You can debug the code and check where the model take them as input. |
@VoidHaruhi thanks for your reply and sorry for asking again, but the code is not running without "ent2id.txt" and pre-trained embedding, which is absent in https://github.com/THUDM/GATNE. |
@Shreyas-Bhat Sorry for the wrong information. It has been a long time since I ran this code, so I cannot remember how I got "ent2id.txt". Please contact the author. |
@VoidHaruhi Sure. If I am not asking for too much, can you please share the scripts you used to run this model? As you have been able to produce the training procedure. |
@Shreyas-Bhat You need to install the python packages in requirements.txt and torch-geometric (maybe a bit annoying because of version conflicts). And check the input and output path. I'm not sure but it will be ok with careful debugging. |
@Shreyas-Bhat Hi, I',m also the guy who want to run this code, and really want to know if you get the ent2id.txt and the embeddings. I really need your kind help. |
@yasNing not yet :( |
@Shreyas-Bhat @yasNing ent2id.txt can be generated by taking |
Trying file upload again for example mappings, upload wasn't working |
@colby-ham Thanks very much! I guess I know how to buld my datasets! |
@colby-ham Thank you very much for clarifying. @yasNing can we connect outside this thread? I am working on a separate dataset too and could use your help in the steps. |
@yasNing can you please tell me how you generate train.txt, test.txt, valid.txt etc for your dataset? |
@Shreyas-Bhat The dataset format is the same as GATNE. https://github.com/THUDM/GATNE/tree/master/data/twitter |
@Shreyas-Bhat And you should use train.txt to generate ent2id.txt. Also, use node2Vec to generate .emd |
@yasNing are you using the Twitter dataset? I am using a dataset which is not there in the GitHub link you have shared. Any idea how I can transform that dataset into the .txt format provided? Thanks! |
You just create your own datasets seriously by the format. I guess it will be OK. |
@yasNing how to generate ent2id.txt? |
@yasNing, I mean to say, how to generate ent2id.txt on your own dataset? thanks |
@Shreyas-Bhat Take all of the node IDs from your dataset, enumerate over them starting from 0, and for each node add an item to a dictionary. The keys are the original entity ID strings, the values are the converted integer range value. The values should be from 0 to num_entities-1. After creating this dictionary, json dump to ent2id.txt |
@Shreyas-Bhat The author has given the data format. If you don't know how to make a dataset in this situation, then I don't think you need to reproduce this paper. |
I used the hyperparameters provided, but the performance is actually bad. Take DBLP and twitter as examples, each of the results is about 20% below your result in the paper. Here is the command lines I used and the results.
![result](https://user-images.githubusercontent.com/45940459/129528920-c768c8aa-b83a-4e27-a37e-9bb316062edc.PNG)
![image](https://user-images.githubusercontent.com/45940459/129529012-3976b56b-0b55-416c-9235-837c35015e29.png)
DBLP:
nohup python3 main.py --base_embedding_dim=128 --batch_size=128 --beam_width=4 --checkpoint=15 --d_ff=512 --d_k=64 --d_model=128 --d_v=64 --data_name='dblp' --false_edge_gen='double' --ft_batch_size=100 --ft_checkpoint=500 --ft_d_ff=512 --ft_drop_rate=0.1 --ft_input_option='last4_cat' --ft_layer='ffn' --ft_lr=5e-05 --ft_n_epochs=10 --gcn_option='no_gcn' --get_bert_encoder_embeddings=False --lr=0.0001 --max_length=12 --n_epochs=10 --n_heads=4 --n_layers=4 --node_edge_composition_func='mult' --num_gcn_layers=2 --num_walks_per_node=5 --outdir='./output/dblp/' --path_option='shortest' --pretrained_method='node2vec' --walk_type='dfs' 2>&1 &
Twitter:
nohup python3 main.py --base_embedding_dim=128 --batch_size=128 --beam_width=4 --checkpoint=15 --d_ff=512 --d_k=64 --d_model=128 --d_v=64 --data_name='twitter' --emb_dir=None --false_edge_gen='double' --ft_batch_size=100 --ft_checkpoint=500 --ft_d_ff=512 --ft_drop_rate=0.1 --ft_input_option='last4_cat' --ft_layer='ffn' --ft_lr=5e-05 --ft_n_epochs=10 --gcn_option='no_gcn' --get_bert_encoder_embeddings=False --lr=0.0001 --max_length=12 --n_epochs=10 --n_heads=4 --n_layers=4 --node_edge_composition_func='mult' --num_gcn_layers=2 --num_walks_per_node=10 --path_option='shortest' --pretrained_embeddings='./embed/twitter/twitter.emd' --outdir='./output/twitter/' --pretrained_method='node2vec' --walk_type='dfs' 2>&1 &
The text was updated successfully, but these errors were encountered: