[WIP] Refactor documentation API Reference for gensim.summarization #1709

yurkai · 2017-11-12T12:52:54Z

menshikh-iv

Great start! Thanks @yurkai, let's continue your work.

menshikh-iv · 2017-11-13T09:44:57Z

gensim/summarization/textcleaner.py

+"""Text Cleaner
+
+This module contains functions and processors used for processing text, 
+extracting sentences from text, working with acronyms and abbreviations.


Need to add examples/highlights/motivation here (after you finish with docstrings in this file).

menshikh-iv · 2017-11-13T09:45:38Z

gensim/summarization/textcleaner.py

-RE_SENTENCE = re.compile(r'(\S.+?[.!?])(?=\s+|$)|(\S.+?)(?=[\n]|$)', re.UNICODE)
+"""str: special separator used in abbreviations."""
+RE_SENTENCE = re.compile(r'(\S.+?[.!?])(?=\s+|$)|(\S.+?)(?=[\n]|$)', re.UNICODE)  # backup (\S.+?[.!?])(?=\s+|$)|(\S.+?)(?=[\n]|$)
+"""SRE_Pattern: pattern to split text to sentences."""


Problem with building here

@yurkai good example, how to document it: https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/parsing/preprocessing.py#L21

menshikh-iv · 2017-11-13T09:46:12Z

gensim/summarization/textcleaner.py



 def split_sentences(text):
+    """Splits and returns list of sentences from given text. It preserves 


Examples section should be nice (here and everywhere).

menshikh-iv · 2017-11-13T09:46:51Z

gensim/summarization/textcleaner.py

+
+    Returns
+    -------
+    str:


Doesn't match with return type

menshikh-iv · 2017-11-13T09:48:02Z

gensim/summarization/textcleaner.py

+        Input text.
+    separator : str
+        The separator between words to be replaced.
+    regexs : str


doesn't match

menshikh-iv · 2017-11-13T09:52:13Z

gensim/summarization/textcleaner.py

+    ----------
+    words : list
+        List of words.
+    separator : str


str -> str, optional

menshikh-iv · 2017-11-13T09:52:39Z

gensim/summarization/textcleaner.py

+    words : list
+        List of words.
+    separator : str
+        The separator bertween elements. Blank set as default.


Blank? I see " ", not ""

menshikh-iv · 2017-11-13T09:53:03Z

gensim/summarization/textcleaner.py

+    ----------
+    text : list
+        Input text.
+    deacc : bool


bool, option - here and everywhere

menshikh-iv · 2017-11-13T09:53:29Z

gensim/summarization/textcleaner.py

+
+    Parameters
+    ----------
+    text : list


type doesn't match

menshikh-iv · 2017-11-13T09:53:53Z

gensim/summarization/textcleaner.py

+
+    Parameters
+    ----------
+    text : list


type doesn't match

menshikh-iv

Review for the small part (I'll review fully later)

menshikh-iv · 2017-11-17T07:27:36Z

gensim/summarization/commons.py

+
+    Parameters
+    ----------
+    sequence : list


list of ?

menshikh-iv · 2017-11-17T07:30:21Z

gensim/summarization/commons.py

+    Returns
+    -------
+    Graph
+        Created graph.


Graph produced by sequence

menshikh-iv · 2017-11-17T07:31:03Z

gensim/summarization/commons.py

+
+    Returns
+    -------
+    Graph


Concrete link to type like

:class:`~gensim. ... ...

here and everywhere (for "gensim-defined" types).

menshikh-iv · 2017-11-17T07:32:34Z

gensim/summarization/commons.py

 from gensim.summarization.graph import Graph


 def build_graph(sequence):
+    """Creates and returns graph with given sequence of values.


What's "type" of graph (oriented, etc)?

menshikh-iv · 2017-11-17T07:33:02Z

gensim/summarization/commons.py

@@ -15,6 +51,15 @@ def build_graph(sequence):


 def remove_unreachable_nodes(graph):
+    """Removes unreachable nodes (nodes with no edges). Works inplace.


. Works inplace. -> , inplace.

menshikh-iv · 2017-11-17T07:34:31Z

gensim/summarization/graph.py

-        @param node: Node identifier
+        Parameters
+        ----------
+        node : str or float


menshikh-iv · 2017-11-17T07:35:12Z

gensim/summarization/graph.py

-        @attention: While nodes can be of any type, it's strongly recommended
+        """Adds given node to the graph.
+
+        Note


Note -> Warning

menshikh-iv · 2017-11-21T10:24:43Z

gensim/summarization/keywords.py

+    ----------
+    text : str
+        Sequence of values.
+    ratio : float


this is optional too

menshikh-iv · 2017-11-21T10:24:59Z

gensim/summarization/keywords.py

+        If no "words" option is selected, the number of sentences is
+        reduced by the provided ratio, else, the ratio is ignored.
+    words : list
+        .


Need to add descriptions to parameters

menshikh-iv · 2017-11-21T10:26:04Z

gensim/summarization/bm25.py

@@ -3,20 +3,72 @@
 #
 # Licensed under the GNU LGPL v2.1 - http://www.gnu.org/licenses/lgpl.html

+"""This module contains function of computing BM25 scores for documents in 
+corpus and helper class `BM25` used in calculations.


Missed link to BM25 algorithm (wiki for example)

menshikh-iv · 2017-12-12T10:23:57Z

Nice work @yurkai 👍, pay attention to my commits and continue your work 🥇

yurkai and others added 2 commits November 12, 2017 15:49

Added docstrings in textcleaner.py

1c6009c

Merge branch 'develop' into fix-1668

851b02c

menshikh-iv suggested changes Nov 13, 2017

View reviewed changes

menshikh-iv changed the title ~~Added docstrings in textcleaner.py~~ [WIP] Refactor documentation API Reference for gensim.summarization Nov 13, 2017

menshikh-iv added this to In Progress in Documentation Nov 13, 2017

menshikh-iv moved this from In Progress to Yury (in progress) in Documentation Nov 13, 2017

yurkai added 3 commits November 14, 2017 01:38

Added docstrings to bm25.py

5cbb184

syntactic_unit.py docstrings and typo

31be095

added doctrings for graph modules

c6c608b

menshikh-iv suggested changes Nov 17, 2017

View reviewed changes

yurkai added 2 commits November 17, 2017 17:11

keywords draft

d5247c1

keywords draft updated

3031cd0

menshikh-iv suggested changes Nov 21, 2017

View reviewed changes

yurkai and others added 17 commits November 21, 2017 19:49

keywords draft updated again

4d7b0a9

keywords edited

2c8ef28

pagerank started

254dce7

pagerank summarizer docstring added

a2c2102

fixed types in docstrings in commons, bm25, graph and keywords

1a87934

fixed types, examples and types in docstrings

0ca8332

Merge branch 'develop' into fix-1668

ed188ae

fix pep8

20b19d6

fix doc build

6ec29bf

fix bm25

e2a2e60

fix graph

d7056e4

fix graph[2]

400966c

fix commons

44f617c

fix keywords

d2fed6c

fix keywords[2]

84b0f3a

fix mz_entropy

ba8b1b6

fix pagerank_weighted

2a283d7

menshikh-iv added 5 commits December 12, 2017 13:01

fix graph rst

6bd1584

fix summarizer

7ec89fa

fix syntactic_unit

fa5efce

fix textcleaner

0014d88

fix

1a0166a

menshikh-iv merged commit f09b7db into piskvorky:develop Dec 12, 2017

Documentation automation moved this from Yuri (in progress) to Done Dec 12, 2017

PeteBleackley mentioned this pull request Jan 5, 2018

Documentation for Montemurro and Zanette algorithm #1826

Closed

pyup-bot mentioned this pull request Feb 5, 2018

Scheduled weekly dependency update for week 05 workforce-data-initiative/skills-ml#120

Closed

piskvorky mentioned this pull request Apr 30, 2018

Documentation fixes #2037

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Refactor documentation API Reference for gensim.summarization #1709

[WIP] Refactor documentation API Reference for gensim.summarization #1709

yurkai commented Nov 12, 2017 •

edited by menshikh-iv

menshikh-iv left a comment

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv Nov 13, 2017

menshikh-iv left a comment

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 17, 2017

menshikh-iv Nov 21, 2017

menshikh-iv Nov 21, 2017

menshikh-iv Nov 21, 2017

menshikh-iv commented Dec 12, 2017



		def split_sentences(text):
		"""Splits and returns list of sentences from given text. It preserves

		@@ -15,6 +51,15 @@ def build_graph(sequence):


		def remove_unreachable_nodes(graph):
		"""Removes unreachable nodes (nodes with no edges). Works inplace.

[WIP] Refactor documentation API Reference for gensim.summarization #1709

[WIP] Refactor documentation API Reference for gensim.summarization #1709

Conversation

yurkai commented Nov 12, 2017 • edited by menshikh-iv

menshikh-iv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

menshikh-iv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

menshikh-iv commented Dec 12, 2017

yurkai commented Nov 12, 2017 •

edited by menshikh-iv