Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP][GSoC 2018] Similarity Learning #2050

Closed
wants to merge 115 commits into from
Closed

[WIP][GSoC 2018] Similarity Learning #2050

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
e249ed4
handle deprecation
aneesh-joshi Feb 8, 2018
62f6c82
Merge branch 'develop' of https://github.com/RaRe-Technologies/gensim…
aneesh-joshi Feb 14, 2018
1677e98
handle max_count
aneesh-joshi Feb 18, 2018
e8c08f8
change flag name
aneesh-joshi Feb 18, 2018
258d033
make flake8 compatible
aneesh-joshi Feb 18, 2018
875c65c
move max_vocab to prepare vocab
aneesh-joshi Feb 20, 2018
0aa8426
correct max_vocab semantics
aneesh-joshi Feb 20, 2018
390f333
remove unnecessary nextline
aneesh-joshi Feb 20, 2018
8c508c7
fix bug and make flake8 complaint
aneesh-joshi Feb 21, 2018
c826b19
refactor code and change sorting to key based
aneesh-joshi Feb 22, 2018
35dc681
add tests
aneesh-joshi Mar 5, 2018
67f6a14
introduce effective_min_count
aneesh-joshi Mar 5, 2018
7b1f612
make flake8 compliant
aneesh-joshi Mar 5, 2018
fafee70
remove clobbering of min_count
aneesh-joshi Mar 7, 2018
9d99660
remove min_count assertion
aneesh-joshi Mar 7, 2018
6c06fbc
.\gensim\models\word2vec.py
aneesh-joshi Mar 7, 2018
c5a0e6e
Revert ".\gensim\models\word2vec.py"
aneesh-joshi Mar 7, 2018
fdd2aab
rename max_vocab to max_final_vocab
aneesh-joshi Mar 7, 2018
974d587
update test to max_final_vocab
aneesh-joshi Mar 7, 2018
ddb3556
move and modify comment docs
aneesh-joshi Mar 7, 2018
c54d8a9
make flake8 compliant
aneesh-joshi Mar 7, 2018
f379616
refactor word2vec.py
aneesh-joshi Mar 8, 2018
46d3885
handle possible old model load errors
aneesh-joshi Mar 11, 2018
2cf5625
include effective_min_count tests
aneesh-joshi Mar 11, 2018
8578e3d
make flake compliant
aneesh-joshi Mar 11, 2018
a43fea3
remove check for max_final_vocab
aneesh-joshi Mar 13, 2018
340a8cf
include backward compat for 3.3 models
aneesh-joshi Mar 15, 2018
0b62407
remove unnecessary newline
aneesh-joshi Mar 15, 2018
5b7a6c2
add test case for max_final_vocab
aneesh-joshi Mar 19, 2018
48ad4dc
merge master
aneesh-joshi May 14, 2018
3cf1e0b
add dssm model and SL vocab class
aneesh-joshi May 15, 2018
218d133
add evaluation scripts
aneesh-joshi May 24, 2018
93b18f2
refactor code and add write up
aneesh-joshi May 26, 2018
90db732
Delete d2v_check.py
aneesh-joshi May 26, 2018
7a6e868
Delete d2v_eval.py
aneesh-joshi May 26, 2018
82eed4a
Delete eval_theirs.py
aneesh-joshi May 26, 2018
8baaae9
Delete predict.test.wikiqa.txtDRMM
aneesh-joshi May 26, 2018
dc5db12
Delete quora_qp_check.py
aneesh-joshi May 26, 2018
ba3053a
Delete w2v_avg_eval.py
aneesh-joshi May 26, 2018
2af2a83
Update WriteUp 25 May.md
aneesh-joshi May 26, 2018
5691116
Update WriteUp 25 May.md
aneesh-joshi May 26, 2018
28004ac
add sample mz output
aneesh-joshi May 26, 2018
3639619
fix dssm.py, add doctrings and refactor
aneesh-joshi May 28, 2018
43829b3
get_data.py
aneesh-joshi May 28, 2018
0f58bdf
add get_data.py
aneesh-joshi May 28, 2018
f6a6175
add unified script for evaluation
aneesh-joshi May 28, 2018
9285d76
add evaluation_script
aneesh-joshi May 28, 2018
8814a5a
make partially flake8 compliant
aneesh-joshi May 28, 2018
da4d00b
finish pep8 compliance
aneesh-joshi May 29, 2018
be348b7
fix line endings and some tox flake issues
aneesh-joshi May 29, 2018
bf7d0eb
add docstrings and unzipping in get_data
aneesh-joshi May 29, 2018
2e4a5a8
add support for saving evaluaton results in a .csv
aneesh-joshi May 29, 2018
28fa12f
fix flake8
aneesh-joshi May 29, 2018
7600dc4
add support for savinf as csv
aneesh-joshi May 30, 2018
ff1903e
add drmm_tks
aneesh-joshi Jun 2, 2018
984a025
add writeups, fix hanging indents, add glove download
aneesh-joshi Jun 4, 2018
ef73071
Fix minor bug/typo
aneesh-joshi Jun 5, 2018
d7c931a
sl_vocab working
aneesh-joshi Jun 6, 2018
c4f1818
everythin working
aneesh-joshi Jun 8, 2018
651f532
add list generator, add drmm_tks to evaluation script
aneesh-joshi Jun 8, 2018
e96913e
make flake8 compliant
aneesh-joshi Jun 8, 2018
fe64ee0
Update Docs
aneesh-joshi Jun 8, 2018
cff7fb8
before removing pandas
aneesh-joshi Jun 16, 2018
617f4a4
remove pandas dependency
aneesh-joshi Jun 16, 2018
473030c
add random seed
aneesh-joshi Jun 16, 2018
ae50259
remove grabage files
aneesh-joshi Jun 16, 2018
216dc01
add ipynb
aneesh-joshi Jun 19, 2018
cab3ac1
clean up
aneesh-joshi Jun 20, 2018
1443ac3
clean further
aneesh-joshi Jun 20, 2018
c65beb0
clean further
aneesh-joshi Jun 20, 2018
554833a
remove ipynb checkpoints
aneesh-joshi Jun 20, 2018
5972471
Update setup.py
aneesh-joshi Jun 20, 2018
13a396b
put old ipynb back
aneesh-joshi Jun 20, 2018
55958a4
fix flake and remove unnecessary files
aneesh-joshi Jun 20, 2018
d8a1409
remove ipynb checkpoints
aneesh-joshi Jun 20, 2018
9e4554e
move everything to models.experimental
aneesh-joshi Jun 21, 2018
4bbd365
move code further
aneesh-joshi Jun 21, 2018
1b534cc
fix ipynb calls
aneesh-joshi Jun 21, 2018
2e18938
add KeyedVectors, fix vocab building and build from vocab
aneesh-joshi Jun 24, 2018
cc440cb
add lazy warning for keras and tf
aneesh-joshi Jun 24, 2018
3e8b8e5
remove lambda layers and add save functionality
aneesh-joshi Jun 25, 2018
07da82b
add custom layer for topk
aneesh-joshi Jun 25, 2018
afab56a
add load function
aneesh-joshi Jun 26, 2018
ee58169
make requested changes
aneesh-joshi Jun 26, 2018
c719939
merge keras and non keras class
aneesh-joshi Jun 26, 2018
d56bbc1
remove keep_full_embedding
aneesh-joshi Jun 27, 2018
2de9b55
add params to train for multiple trains
aneesh-joshi Jun 27, 2018
08acdc5
make training iterable
aneesh-joshi Jun 28, 2018
13a51fb
add KeyedVector support as input
aneesh-joshi Jun 28, 2018
d556a00
add seeding in general and for each unk word
aneesh-joshi Jun 28, 2018
5f83741
fix docs
aneesh-joshi Jun 28, 2018
2ac7f31
update notebook
aneesh-joshi Jun 28, 2018
d6818ee
make some requested changes
aneesh-joshi Jun 28, 2018
256f319
Merge branch 'develop' of https://github.com/RaRe-Technologies/gensim…
aneesh-joshi Jun 28, 2018
5c72137
fix load bug
aneesh-joshi Jun 29, 2018
350f4aa
fix doctests
aneesh-joshi Jun 29, 2018
2e68051
fix module imports
aneesh-joshi Jun 29, 2018
376b28f
add all doctests
aneesh-joshi Jul 1, 2018
0651f44
add some ignored files
aneesh-joshi Jul 1, 2018
5864db0
add drmm_tks test
aneesh-joshi Jul 1, 2018
20fbbfc
fix imports
aneesh-joshi Jul 1, 2018
50386af
fix docs
aneesh-joshi Jul 1, 2018
aab50fa
finalize docs
aneesh-joshi Jul 1, 2018
236e4b7
fix typo
aneesh-joshi Jul 1, 2018
66e2385
fix doctests
aneesh-joshi Jul 1, 2018
d662bac
fix file name typo
aneesh-joshi Jul 1, 2018
127b441
fix flake
aneesh-joshi Jul 1, 2018
4190b99
add w2v eval in notebook
aneesh-joshi Jul 2, 2018
1299bb7
add evaluate model and complete ipynb
aneesh-joshi Jul 2, 2018
157b7d7
remove models
aneesh-joshi Jul 2, 2018
451e3b1
- adds non model files to docs
aneesh-joshi Jul 3, 2018
fd575ea
fix TopK Layer bug
aneesh-joshi Jul 4, 2018
5280853
add drmm
aneesh-joshi Jul 4, 2018
7dec231
get well tuned model
aneesh-joshi Jul 5, 2018
5219d9e
* Tune paramaters to get MAP : 0.60 and
aneesh-joshi Jul 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/src/apiref.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ Modules:
models/deprecated/keyedvectors
models/deprecated/fasttext_wrapper
models/base_any2vec
models/experimental/drmm_tks
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need also include other files to documentation building (like callbacks, layers, etc)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@menshikh-iv

Please refer to the link below which shows the diff of the requested changes

451e3b1?utf8=%E2%9C%93&diff=unified

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note

tox -e docs will throw errors. Not on my files but on some keras files since I am inheriting from the Keras Layer class which has some unformatted docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aneesh-joshi that's shouldn't happen (because you include only your files, not Keras), can you show me log of tox -e docs that mention the error in some Keras file (not your)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

/home/aneeshj/Projects/gensim/.tox/docs/local/lib/python2.7/site-packages/gensim/models/experimental/custom_layers.py:docstring of gensim.models.experimental.custom_layers.TopKLayer.add_weight:10: WARNING: Unexpected indentation.
/home/aneeshj/Projects/gensim/.tox/docs/local/lib/python2.7/site-packages/gensim/models/experimental/custom_layers.py:docstring of gensim.models.experimental.custom_layers.TopKLayer.add_weight:12: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/aneeshj/Projects/gensim/.tox/docs/local/lib/python2.7/site-packages/gensim/models/experimental/custom_layers.py:docstring of gensim.models.experimental.custom_layers.TopKLayer.call:4: WARNING: Inline strong start-string without end-string.

I haven't implemented any of the above functions. Just inherited the Layer class.

Copy link
Contributor

@menshikh-iv menshikh-iv Jul 4, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, looks like you are right (issue with docstring of the parent class that we can't control).
Simple workaround - define these methods yourself and call super (but don't worry much about it now), you have more critical tasks now.

models/experimental/custom_callbacks
models/experimental/custom_layers
models/experimental/custom_losses
models/experimental/evaluation_metrics
similarities/docsim
similarities/index
sklearn_api/atmodel
Expand Down
9 changes: 9 additions & 0 deletions docs/src/models/experimental/custom_callbacks.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:mod:`models.experimental.custom_callbacks` -- Custom Callbacks for Similarity Learning
=======================================================================================

.. automodule:: gensim.models.experimental.custom_callbacks
:synopsis: Custom Callbacks for Similarity Learning
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/src/models/experimental/custom_layers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:mod:`models.experimental.custom_layers` -- Custom Layers for Similarity Learning
=================================================================================

.. automodule:: gensim.models.experimental.custom_layers
:synopsis: Custom Layers for Similarity Learning
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/src/models/experimental/custom_losses.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:mod:`models.experimental.custom_losses` -- Loss for Similarity Learning
========================================================================

.. automodule:: gensim.models.experimental.custom_losses
:synopsis: Loss functions for Similarity Learning
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/src/models/experimental/drmm_tks.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:mod:`models.experimental.drmm_tks` -- Neural Nets for Similarity Learning
==========================================================================

.. automodule:: gensim.models.experimental.drmm_tks
:synopsis: Neural Network Similarity Learning
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/src/models/experimental/evaluation_metrics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:mod:`models.experimental.evaluation_metrics` -- Evaluation Metrics for Similarity Learning
===========================================================================================

.. automodule:: gensim.models.experimental.evaluation_metrics
:synopsis: Evaluation Metrics for Similarity Learning
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
1 change: 1 addition & 0 deletions gensim/models/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

from . import wrappers # noqa:F401
from . import deprecated # noqa:F401
from . import experimental # noqa:F401

from gensim import interfaces, utils

Expand Down
Loading