Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨🤖 Update TransH to ER-Model #954

Merged
merged 38 commits into from
Jun 2, 2022
Merged

✨🤖 Update TransH to ER-Model #954

merged 38 commits into from
Jun 2, 2022

Conversation

mberr
Copy link
Member

@mberr mberr commented May 31, 2022

This PR updates TransH to new-style ERModel. Also updates the experiment configuration from JSON to YAML.

It also splits the TransH regularizer into its two components, and modernizes the tests.

Benchmark

FB15k

commit training evaluation
6e3eb43 (update-transh) 1.59s 14.78s (bs=512)
f5c6922 (master) 1.59s 14.71s (bs=1024)

WN18

commit training evaluation
6e3eb43 (update-transh) 0.52s 1.71s (bs=512)
f5c6922 (master) 0.51s 1.72s (bs=512)

FB15k

metadata:
  title: "Learn FB15K Dataset with TransH as described by Wang et al., 2014"
pipeline:
  dataset: "fb15k"
  model: "TransH"
  model_kwargs:
    predict_with_sigmoid: false
    embedding_dim: 100
    scoring_fct_norm: 2
    entity_initializer: "xavier_uniform"
    relation_initializer: "xavier_uniform"
  regularizer: "TransH"
  regularizer_kwargs:
    weight: 1.0
    epsilon: 1.0e-05
  optimizer: "SGD"
  optimizer_kwargs:
    lr: 0.005
  loss: "MarginRankingLoss"
  loss_kwargs:
    reduction: "mean"
    margin: 0.25
  training_loop: "sLCWA"
  negative_sampler: "bernoulli"
  negative_sampler_kwargs:
    num_negs_per_pos: 1
  training_kwargs:
    num_epochs: 5
    batch_size: 4800
  evaluator_kwargs:
    filtered: true

WN18

metadata:
  title: "Learn WN18 Dataset with TransH as described by Wang et al., 2014"
pipeline:
  dataset: "wn18"
  model: "TransH"
  model_kwargs:
    predict_with_sigmoid: false
    embedding_dim: 50
    scoring_fct_norm: 2
    entity_initializer: "xavier_uniform"
    relation_initializer: "xavier_uniform"
  regularizer: "TransH"
  regularizer_kwargs:
    weight: 0.25
    epsilon: 1.0e-05
  optimizer: "SGD"
  optimizer_kwargs:
    lr: 0.01
  loss: "MarginRankingLoss"
  loss_kwargs:
    reduction: "mean"
    margin: 1.0
  training_loop: "sLCWA"
  negative_sampler: "bernoulli"
  negative_sampler_kwargs:
    num_negs_per_pos: 1
  training_kwargs:
    num_epochs: 5
    batch_size: 4800
  evaluator_kwargs:
    filtered: true

Dependencies

@mberr mberr added the ✨ ERModel Upgrading old-style models to new-style label May 31, 2022
@mberr mberr added this to In Progress in ERModel Migration May 31, 2022
@mberr mberr marked this pull request as ready for review May 31, 2022 15:35
@mberr mberr requested a review from cthoyt May 31, 2022 15:35
@mberr mberr removed the request for review from cthoyt June 1, 2022 13:44
@@ -1702,6 +1703,41 @@ def prepare_filter_triples(
return mapped_triples


def normalize_with_default(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR this into class resolver?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

mberr added a commit to cthoyt/class-resolver that referenced this pull request Jun 1, 2022
cthoyt added a commit to cthoyt/class-resolver that referenced this pull request Jun 1, 2022
cf. pykeen/pykeen#954

Co-authored-by: Charles Tapley Hoyt <cthoyt@gmail.com>
@mberr mberr requested a review from cthoyt June 2, 2022 08:57
@mberr mberr marked this pull request as draft June 2, 2022 11:58
@mberr mberr marked this pull request as ready for review June 2, 2022 11:58
@mberr mberr mentioned this pull request Jun 2, 2022
8 tasks
@mberr mberr merged commit c84081c into master Jun 2, 2022
@mberr mberr deleted the update-transh branch June 2, 2022 12:34
@mberr mberr moved this from In Progress to Done in ERModel Migration Jun 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ ERModel Upgrading old-style models to new-style
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants