Reformat docstrings in espnet/asr #914

Masao-Someki · 2019-06-22T09:30:30Z

Reformat docstrings of the following files.

./espnet/asr/asr_mix_utils.py
./espnet/asr/asr_utils.py
./espnet/asr/chainer_backend/asr.py
./espnet/asr/pytorch_backend/asr.py
./espnet/asr/pytorch_backend/asr_mix.py

Fixed typo

* reformat docstrings of the following files. [x] ./espnet/asr/asr_mix_utils.py [x] ./espnet/asr/asr_utils.py [x] ./espnet/asr/chainer_backend/asr.py [ ] ./espnet/asr/pytorch_backend/asr.py [ ] ./espnet/asr/pytorch_backend/asr_mix.py fixed typo

fixed typo

kan-bayashi · 2019-06-22T10:58:03Z

Thank you so much!

I have some requests and comments.

Capitalize the first word in each sentence
Add period the end of sentence
No need to add docstring to private method
No need to use variable name in Returns:

e.g.

# now
Returns:
    outputs (Tensor): batch of output tensors (B, T, D)

# after
Returns:
    Tensor: Batch of output tensors (B, T, D).

Masao-Someki · 2019-06-22T11:11:57Z

Okay!

Masao-Someki · 2019-06-22T12:34:40Z

I couldn't figure out the type of score at line 482 in ./asr_utils.py
Could you tell me where I can find out the type?

* fixed pytorch_backend * fixed some typos in `./asr_mix_utils.py` , `asr_utils.py` , `./chainer_backend/asr.py` . * fixed typos

kan-bayashi · 2019-06-22T12:41:24Z

espnet/espnet/asr/asr_utils.py

Line 475 in 6c9d976

score = float(hyp['score'])

Maybe float.

Masao-Someki · 2019-06-22T12:43:56Z

Oh...I don't know why I miss that sentence.
Thank you!

Masao-Someki · 2019-06-22T12:47:20Z

I'll add details to data-type in args and results tomorrow.

ex) Dict → Dict[String, Dict[String. List[String]]]

codecov · 2019-06-22T13:18:07Z

Codecov Report

Merging #914 into v.0.5.0 will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           v.0.5.0     #914   +/-   ##
========================================
  Coverage    49.32%   49.32%           
========================================
  Files          100      100           
  Lines        10724    10724           
========================================
  Hits          5290     5290           
  Misses        5434     5434

Impacted Files	Coverage Δ
espnet/asr/asr_mix_utils.py	`0% <ø> (ø)`	⬆️
espnet/asr/pytorch_backend/asr_mix.py	`0% <ø> (ø)`	⬆️
espnet/asr/chainer_backend/asr.py	`0% <ø> (ø)`	⬆️
espnet/asr/asr_utils.py	`26.1% <ø> (ø)`	⬆️
espnet/asr/pytorch_backend/asr.py	`0% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a966c3...d345e43. Read the comment docs.

kan-bayashi · 2019-06-23T01:33:46Z

It is better to unify the type format e.g. str or String, Dict or dict.
I like lower-case e.g. str, dict, list

* update data_type

Masao-Someki · 2019-06-23T03:48:30Z

I coudn't find out the following, so please collect or teach me where to watch...

espnet/asr/chainer_backend/asr.py: L100: type of device
espnet/asr/asr_mix_utils.py:L103: type of device
espnet/asr/asr_utils.py:L426:type of trainer
espnet/asr/chainer_backend/asr.py: L103, L162: explanation of accum_grad.

kan-bayashi · 2019-06-24T01:44:10Z

espnet/asr/chainer_backend/asr.py: L100: type of device

device (int or dict): The destination device info to send variables. In the case of cpu or single gpu, `device=-1 or 0`, respectively.
    In the case of multi-gpu, `device={"main":0, "sub_1": 1, ...}`.

espnet/asr/asr_mix_utils.py:L103: type of device

device (torch.device): The destination device to send tensor.

espnet/asr/asr_utils.py:L426:type of trainer

trainer (chainer.training.Trainer): Chainer's trainer instance.

espnet/asr/chainer_backend/asr.py: L103, L162: explanation of accum_grad.

accum_grad (int): The number of gradient accumulation. if set to 2, the network parameters will be updated once in twice, i.e. actual batchsize will be doubled.

Could you remove following highlighted warning?
https://travis-ci.org/espnet/espnet/jobs/549262746#L1954-L1956

Masao-Someki · 2019-06-24T04:30:36Z

I fixed the highlighted warnings

fixed typo

kan-bayashi · 2019-06-24T10:47:19Z

If ready to merge, please let me know.
I will review.

Masao-Someki · 2019-06-24T12:38:20Z

It's ready. Could you review it?

kan-bayashi · 2019-06-25T02:42:48Z

I will check today or tomorrow.

ShigekiKarita · 2019-06-26T09:12:22Z

espnet/asr/asr_mix_utils.py

+    """Make batch set from json dictionary.
+
+    Args:
+        data (dict[str, list[Any]): Dictionary loaded from data.json.


typo: dict[str, list[Any]]

ShigekiKarita · 2019-06-26T09:14:08Z

espnet/asr/asr_mix_utils.py

+        batch_size (int): Batch size.
+        max_length_in (int): Maximum length of input to decide adaptive batch size.
+        max_length_out (int): Maximum length of output to decide adaptive batch size.
+        num_batches (int): # Number of batches to use (for debug).


ShigekiKarita · 2019-06-26T09:16:39Z

espnet/asr/asr_mix_utils.py

+        min_batch_size (int): Mininum batch size (for multi-gpu).
+
+    Returns:
+        list[tuple[str, dict[str, list[dict[str, Any]]]]: List of batches.


typo: list[tuple[str, dict[str, list[dict[str, Any]]]]] list (1) of tuple (2) of dict (3) of list (4) of dict(5), you need 5 ]. LOL

ShigekiKarita · 2019-06-26T09:21:28Z

@kan-bayashi

It is better to unify the type format e.g. str or String, Dict or dict.
I like lower-case e.g. str, dict, list

I have different opinion. Because there are PEPs for type annotations, we can use them as clear rules. For example, they should be str, Dict, List

https://www.python.org/dev/peps/pep-0484/
https://www.python.org/dev/peps/pep-0526/

espnet/asr/asr_mix_utils.py

ShigekiKarita · 2019-06-26T09:31:06Z

espnet/asr/asr_utils.py

-    :param eta float {0.01,0.3,1.0}
-    """
+    Args:
+        model (Torch model): model.


model (torch.nn.Module): model.?

ShigekiKarita · 2019-06-26T09:32:06Z

espnet/asr/asr_utils.py

@@ -169,8 +196,12 @@ def _restore_snapshot(model, snapshot, load_fn=chainer.serializers.load_npz):


 def adadelta_eps_decay(eps_decay):
-    """Extension to perform adadelta eps decay"""
+    """Extension to perform adadelta eps decay.


Args: eps_decay (float): decay rate of eps

kan-bayashi · 2019-06-26T14:04:00Z

Thank you for your comments, Shigeki.
OK. We should follow these rules.

Fixed typos fixed typo this is a commit for typos. I tried a few times, so I rebase some commits.

kan-bayashi

Thank you so much.
Could you resolve conflict?

kan-bayashi · 2019-06-27T04:30:21Z

espnet/asr/asr_mix_utils.py

+        """Visualize attention weights matrix.
+
+        Args:
+            att_w(Tensor): attention weight matrix


Suggested change

att_w(Tensor): attention weight matrix

att_w(Tensor): Attention weight matrix.

kan-bayashi · 2019-06-27T04:31:40Z

espnet/asr/asr_mix_utils.py

+
+    Args:
+        js (dict[str, Any]): Groundtruth utterance dict.
+        nbest_hyps_sd (list[dict[str, Any]]): List of hypothesis for multi_speakers: nutts x nspkrs.


Suggested change

nbest_hyps_sd (list[dict[str, Any]]): List of hypothesis for multi_speakers: nutts x nspkrs.

nbest_hyps_sd (list[dict[str, Any]]): List of hypothesis for multi_speakers (# Utts x # Spkrs).

kan-bayashi · 2019-06-27T04:32:30Z

espnet/asr/asr_utils.py


+    Args:
+        eps_decay (float): decay rate of eps


Suggested change

eps_decay (float): decay rate of eps

eps_decay (float): Decay rate of eps.

kan-bayashi · 2019-06-27T04:32:55Z

espnet/asr/asr_utils.py

-    :param eta float {0.01,0.3,1.0}
-    """
+    Args:
+        model (torch.nn.model): model.


Suggested change

model (torch.nn.model): model.

model (torch.nn.model): Model.

kan-bayashi · 2019-06-27T04:33:40Z

espnet/asr/asr_utils.py

-    """
+    Args:
+        model (torch.nn.model): model.
+        iteration (int): number of iteration.


Suggested change

iteration (int): number of iteration.

epoch (int): Number of epochs.

kan-bayashi · 2019-06-27T04:41:28Z

espnet/asr/pytorch_backend/asr_mix.py

+            device (torch.device): The device to send to.
+
+        Returns:
+            tuple(torch.Tensor, torch.Tensor, torch.Tensor)


Suggested change

tuple(torch.Tensor, torch.Tensor, torch.Tensor)

tuple(torch.Tensor, torch.Tensor, torch.Tensor): Transformed batch.

kan-bayashi · 2019-06-27T04:42:55Z

espnet/asr/asr_mix_utils.py

@@ -12,12 +12,12 @@
 import matplotlib
 import numpy as np

-'''
+"""
 reuse modules in asr_utils:


Suggested change

reuse modules in asr_utils:

Utility functions for ASR mix recipes. Reuse following modules in asr_utils:

kan-bayashi · 2019-06-27T04:43:24Z

espnet/asr/asr_mix_utils.py

@@ -12,12 +12,12 @@
 import matplotlib
 import numpy as np

-'''
+"""
 reuse modules in asr_utils:
    CompareValueTrigger, restore_snapshot, _restore_snapshot, adadelta_eps_decay,


Suggested change

CompareValueTrigger, restore_snapshot, _restore_snapshot, adadelta_eps_decay,

CompareValueTrigger, restore_snapshot, adadelta_eps_decay,

kan-bayashi · 2019-06-27T04:45:11Z

espnet/asr/asr_mix_utils.py

@@ -12,12 +12,12 @@
 import matplotlib
 import numpy as np

-'''
+"""
 reuse modules in asr_utils:
    CompareValueTrigger, restore_snapshot, _restore_snapshot, adadelta_eps_decay,
    _adadelta_eps_decay, torch_snapshot, _torch_snapshot_object, AttributeDict,


Suggested change

_adadelta_eps_decay, torch_snapshot, _torch_snapshot_object, AttributeDict,

chainer_load, torch_snapshot, torch_save, torch_load, torch_resume

kan-bayashi · 2019-06-27T04:45:36Z

espnet/asr/asr_mix_utils.py

@@ -12,12 +12,12 @@
 import matplotlib
 import numpy as np

-'''
+"""
 reuse modules in asr_utils:
    CompareValueTrigger, restore_snapshot, _restore_snapshot, adadelta_eps_decay,
    _adadelta_eps_decay, torch_snapshot, _torch_snapshot_object, AttributeDict,
    get_model_conf, chainer_load, torch_save, torch_load, torch_resume


Suggested change

get_model_conf, chainer_load, torch_save, torch_load, torch_resume

AttributeDict, get_model_conf

* update docstring for review.

Masao-Someki · 2019-06-27T05:47:19Z

I wonder why L86 in asr_utils.py in branch v.0.5.0 says MT iaxi="1".

espnet/espnet/asr/asr_utils.py

Line 86 in 5a966c3

    
               :param int iaxis: dimension to access input (for ASR iaxis=0, for MT iaxis="1".)

The parametor is set as integer value, so I think it should be MT iaxis=1. Should I set "1" as a string in docstring?

* update kan-bayashi's review * resolve conflicts

kan-bayashi · 2019-06-27T08:13:40Z

espnet/espnet/asr/asr_utils.py

Line 132 in 5a966c3

dec_len = int(self.data[idx][1][self.ikey][self.iaxis]['shape'][0])

It seems iaxis should be int.

Masao-Someki · 2019-06-29T07:37:12Z

I think all the docstrings are updated, with no typos..
I don't know why codedec has an error. Do you have any idea to solve it?

kan-bayashi · 2019-07-01T21:14:45Z

Sorry for late reply.
Could you solve conflict?
Then we will merge.

Masao-Someki · 2019-07-02T04:10:18Z

I fixed the conflicts.
I clicked "commit merge" button by mistake. Sorry about that if it confuses you..

Co-Authored-By: Tomoki Hayashi <hayashi.tomoki@g.sp.m.is.nagoya-u.ac.jp>

sw005320 · 2019-07-02T10:26:54Z

thanks a lot!

reformat docstrings in ./espnet/asr

b0a14db

* reformat docstrings of the following files. [x] ./espnet/asr/asr_mix_utils.py [x] ./espnet/asr/asr_utils.py [x] ./espnet/asr/chainer_backend/asr.py [ ] ./espnet/asr/pytorch_backend/asr.py [ ] ./espnet/asr/pytorch_backend/asr_mix.py fixed typo

Masao-Someki changed the base branch from master to v.0.5.0 June 22, 2019 09:30

Masao-Someki added 2 commits June 22, 2019 18:32

Merge remote-tracking branch 'upstream/v.0.5.0' into docstring

ac5fb92

fixed typo

7297661

fixed typo

Masao-Someki force-pushed the docstring branch from 4c8919a to 7297661 Compare June 22, 2019 10:23

sw005320 added the Documentation label Jun 22, 2019

sw005320 added this to the v.0.5.0 milestone Jun 22, 2019

sw005320 requested a review from kan-bayashi June 22, 2019 12:28

update docstring

6c9d976

* fixed pytorch_backend * fixed some typos in `./asr_mix_utils.py` , `asr_utils.py` , `./chainer_backend/asr.py` . * fixed typos

Masao-Someki force-pushed the docstring branch from 20fa147 to 6c9d976 Compare June 22, 2019 12:38

update docstring

4db69e6

* update data_type

fixed typo

86fe257

fixed typos

4211635

fixed typos

0b446ae

fixed typo

Masao-Someki force-pushed the docstring branch from 4b12251 to 0b446ae Compare June 24, 2019 09:24

ShigekiKarita reviewed Jun 26, 2019

View reviewed changes

espnet/asr/asr_mix_utils.py Show resolved Hide resolved

ShigekiKarita reviewed Jun 26, 2019

View reviewed changes

fixed typos

23e6d12

Fixed typos fixed typo this is a commit for typos. I tried a few times, so I rebase some commits.

Masao-Someki force-pushed the docstring branch from 146023a to 23e6d12 Compare June 27, 2019 03:17

kan-bayashi reviewed Jun 27, 2019

View reviewed changes

update docstrings

acb8517

* update docstring for review.

update docstrings

ada144f

* update kan-bayashi's review * resolve conflicts

fixed typo

8956753

Merge branch 'v.0.5.0' into docstring

1b8eccf

kan-bayashi changed the title ~~[WIP] reformat docstrings in ./espnet/asr~~ Reformat docstrings in ./espnet/asr Jul 2, 2019

kan-bayashi changed the title ~~Reformat docstrings in ./espnet/asr~~ Reformat docstrings in espnet/asr Jul 2, 2019

Masao-Someki and others added 2 commits July 2, 2019 18:29

Update espnet/asr/asr_utils.py

0e2f440

Co-Authored-By: Tomoki Hayashi <hayashi.tomoki@g.sp.m.is.nagoya-u.ac.jp>

Update espnet/asr/asr_utils.py

d345e43

Co-Authored-By: Tomoki Hayashi <hayashi.tomoki@g.sp.m.is.nagoya-u.ac.jp>

sw005320 merged commit 76d0492 into espnet:v.0.5.0 Jul 2, 2019

Masao-Someki deleted the docstring branch July 19, 2019 03:36

	att_w(Tensor): attention weight matrix
	att_w(Tensor): Attention weight matrix.

	nbest_hyps_sd (list[dict[str, Any]]): List of hypothesis for multi_speakers: nutts x nspkrs.
	nbest_hyps_sd (list[dict[str, Any]]): List of hypothesis for multi_speakers (# Utts x # Spkrs).

	eps_decay (float): decay rate of eps
	eps_decay (float): Decay rate of eps.

	model (torch.nn.model): model.
	model (torch.nn.model): Model.

	iteration (int): number of iteration.
	epoch (int): Number of epochs.

	tuple(torch.Tensor, torch.Tensor, torch.Tensor)
	tuple(torch.Tensor, torch.Tensor, torch.Tensor): Transformed batch.

	reuse modules in asr_utils:
	Utility functions for ASR mix recipes. Reuse following modules in asr_utils:

	CompareValueTrigger, restore_snapshot, _restore_snapshot, adadelta_eps_decay,
	CompareValueTrigger, restore_snapshot, adadelta_eps_decay,

	_adadelta_eps_decay, torch_snapshot, _torch_snapshot_object, AttributeDict,
	chainer_load, torch_snapshot, torch_save, torch_load, torch_resume

	get_model_conf, chainer_load, torch_save, torch_load, torch_resume
	AttributeDict, get_model_conf

Reformat docstrings in espnet/asr #914

Reformat docstrings in espnet/asr #914

Conversation

Masao-Someki commented Jun 22, 2019 • edited Loading

kan-bayashi commented Jun 22, 2019 • edited Loading

Masao-Someki commented Jun 22, 2019

Masao-Someki commented Jun 22, 2019

kan-bayashi commented Jun 22, 2019

Masao-Someki commented Jun 22, 2019 • edited Loading

Masao-Someki commented Jun 22, 2019

codecov bot commented Jun 22, 2019 • edited Loading

Codecov Report

kan-bayashi commented Jun 23, 2019

Masao-Someki commented Jun 23, 2019

kan-bayashi commented Jun 24, 2019 • edited Loading

Masao-Someki commented Jun 24, 2019

kan-bayashi commented Jun 24, 2019

Masao-Someki commented Jun 24, 2019

kan-bayashi commented Jun 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShigekiKarita commented Jun 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kan-bayashi commented Jun 26, 2019

kan-bayashi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Masao-Someki commented Jun 27, 2019 • edited Loading

kan-bayashi commented Jun 27, 2019

Masao-Someki commented Jun 29, 2019

kan-bayashi commented Jul 1, 2019

Masao-Someki commented Jul 2, 2019 • edited Loading

sw005320 commented Jul 2, 2019

Masao-Someki commented Jun 22, 2019 •

edited

Loading

kan-bayashi commented Jun 22, 2019 •

edited

Loading

Masao-Someki commented Jun 22, 2019 •

edited

Loading

codecov bot commented Jun 22, 2019 •

edited

Loading

kan-bayashi commented Jun 24, 2019 •

edited

Loading

Masao-Someki commented Jun 27, 2019 •

edited

Loading

Masao-Someki commented Jul 2, 2019 •

edited

Loading