Add levenshtein alignment #828

pkufool · 2021-09-15T12:29:26Z

k2.levenshtein_graph

pkufool · 2021-09-15T13:08:04Z

ins_del_score_offset as an fsa attribute.

csukuangfj · 2021-09-15T13:15:29Z

k2/python/k2/fsa_algo.py

+
+def levenshtein_graph(symbols: Union[k2.RaggedTensor, List[List[int]]],
+        self_loop_weight: float = -1,
+        need_weight_bias: bool = False,


Maybe we need a better name for the argument need_weight_bias?

Also, I would recommend converting the returned weight_bias into an attribute of the resulting FSA.

pkufool · 2021-09-15T13:17:34Z

k2/python/k2/fsa_algo.py

@@ -1069,3 +1072,58 @@ def ctc_topo(max_token: int,
    ragged_arc, aux_labels = _k2.ctc_topo(max_token, device, modified)
    fsa = Fsa(ragged_arc, aux_labels=aux_labels)
    return fsa
+
+def levenshtein_graph(symbols: Union[k2.RaggedTensor, List[List[int]]],
+        self_loop_weight: float = -1,


@danpovey What would you suggest to name this argument, self_loop_weight may not be a good one.

It represents insertion and deletion cost. How about calling it "ins_del_cost"?

How about naming it penalty, and the fsa's attribute named penalty_bias.

csukuangfj · 2021-09-15T14:49:44Z

k2/csrc/fsa_algo.cu

+                                       // the next state.
+        if (state_idx01 == final_state - 1)
+          current_num_arcs = 2;
+        if (state_idx01 == final_state)


Suggested change

if (state_idx01 == final_state)

else if (state_idx01 == final_state)

csukuangfj · 2021-09-15T14:55:48Z

k2/python/csrc/torch/fsa_algo.cu

@@ -717,47 +717,6 @@ static void PybindCtcGraph(py::module &m) {
      },
      py::arg("symbols"), py::arg("device") = py::none(),
      py::arg("modified") = false, py::arg("need_arc_map") = true);
-


Please also remove device at line 706.

csukuangfj · 2021-09-15T14:56:09Z

k2/python/csrc/torch/fsa_algo.cu

+static void PybindLevenshteinGraph(py::module &m) {
+  m.def(
+      "levenshtein_graph",
+      [](RaggedAny &symbols, torch::optional<torch::Device> = {},


Suggested change

[](RaggedAny &symbols, torch::optional<torch::Device> = {},

[](RaggedAny &symbols

csukuangfj · 2021-09-15T15:00:51Z

k2/python/k2/fsa_algo.py

@@ -1022,6 +1022,9 @@ def ctc_graph(symbols: Union[List[List[int]], k2.RaggedTensor],
            [it for symbol in symbols for it in symbol],
            dtype=torch.int32,
            device=device)
+        symbols = k2.RaggedTensor(symbols)


This if..else can be simplified to

if isinstance(symbols, k2.RaggedTensor): assert device is None else: symbols = k2.RaggedTensor(symbols, device=device) symbol_values = symbols.values

csukuangfj · 2021-09-15T15:01:54Z

k2/python/k2/fsa_algo.py

+        If `need_weight_bias` set to True, a tensor with weight bias on each arc
+        will be returned as well.
+    '''
+    if isinstance(symbols, k2.RaggedTensor):


Suggested change

if isinstance(symbols, k2.RaggedTensor):

if isinstance(symbols, k2.RaggedTensor):

assert device is None

else:

symbols = k2.RaggedTensor(symbols, device=device)

csukuangfj · 2021-09-15T15:03:44Z

k2/python/k2/fsa_algo.py

+        weight is to set insertion and deletion penalty, which will affect the
+        shortest path searching produre.
+      need_weight_bias:
+        If set to True, a tensor with the bias to -1 on each arc will be


The documentation for need_weight_bias is not easy to understand.

danpovey · 2021-09-16T03:18:51Z

k2/csrc/fsa_algo.cu

@@ -453,9 +453,130 @@ FsaVec LinearFsas(const Ragged<int32_t> &symbols) {
      arcs);
 }

+FsaVec LevenshteinGraphs(const Ragged<int32_t> &symbols,
+                         float penalty /* = -1 */,


I think ins_del_score is a good name, since it explains what it is.
I'm surprised the default is -1 though, instead of -0.5. I thought we were going to multiply all scores by 0.5, so that the total cost equals the Levenshtein alignment score.

pkufool · 2021-09-17T03:57:25Z

k2/python/csrc/torch/v2/ragged_any.cu

@@ -200,6 +203,7 @@ RaggedAny::RaggedAny(const std::string &s, py::object dtype /*=py::none()*/,
  }

  ContextPtr context = GetContext(device);
+  DeviceGuard guard(context);


@csukuangfj I think we need a device guard here.

pkufool · 2021-09-17T04:13:29Z

Intersect_device like interface to get levenshtein alignments.

csukuangfj · 2021-09-17T04:20:25Z

k2/python/k2/fsa_algo.py

+
+            - A list of list-of-integers, e..g, `[ [1, 2], [1, 2, 3] ]`
+            - An instance of :class:`k2.RaggedTensor`.
+              Must have `num_axes == 2`.


Also with dtype torch.int32?

csukuangfj · 2021-09-17T04:22:14Z

k2/python/k2/fsa_algo.py

+        symbols, ins_del_score, True)
+    fsa = Fsa(ragged_arc, aux_labels=aux_labels)
+    if ins_del_score_offset_attr is not None:
+        fsa.__setattr__(name=ins_del_score_offset_attr, value=score_offsets)


Suggested change

fsa.__setattr__(name=ins_del_score_offset_attr, value=score_offsets)

setattr(fsa, ins_del_score_offset_attr, score_offsets)

csukuangfj · 2021-09-17T04:25:37Z

k2/python/k2/fsa_algo.py

+
+
+def levenshtein_distance(
+        a_fsas: Fsa,


Shall we rename the two inputs to refs and hyps?

csukuangfj · 2021-09-17T04:27:04Z

k2/python/k2/fsa_algo.py

+        that requires this.
+
+    Returns:
+      Returns a tuple contains an FsaVec and a torch.Tensor.


contains->containing

csukuangfj · 2021-09-17T04:31:20Z

k2/python/k2/fsa_algo.py

+    alignment = k2.shortest_path(lattice, use_double_scores=True).invert_()
+    alignment.rename_tensor_attribute_("aux_labels_levenshtein", "aux_labels")
+
+    alignment.scores -= alignment.ins_del_score_offset


Shall we pass the atrribute name as an optional argument?

csukuangfj · 2021-09-17T05:01:23Z

k2/python/k2/fsa_algo.py

+    distance = -alignment.get_tot_scores(
+        use_double_scores=False, log_semiring=False)
+
+    return alignment, distance


distance is already contained in alignment.
Can we rename this function to levenshtein_alignment and just
return the alignment?
Users can get the actual alignment and distance from the return value.

Also, levenshtein_distance returns both the alignment and distance. I
was expecting that it returns only the distance.

csukuangfj · 2021-09-18T05:39:52Z

k2/csrc/fsa_algo.cu

+  ExclusiveSum(num_arcs_for, &num_arcs_for);
+  Array1<int32_t> &states_to_arcs_row_splits = num_arcs_for;
+  RaggedShape states_to_arcs =
+      RaggedShape2(&states_to_arcs_row_splits, nullptr, -1);


Suggested change

RaggedShape2(&states_to_arcs_row_splits, nullptr, -1);

int32_t num_arcs = symbols.NumElements() * 3 + symbols.Dim0() * 2;

RaggedShape2(&states_to_arcs_row_splits, nullptr, num_arcs);

Pre-computing cached_tot_size can reduce one extra kernel call.

csukuangfj · 2021-09-18T05:50:20Z

k2/python/k2/fsa_algo.py

+        symbols = k2.RaggedTensor(symbols, device=device)
+
+    ragged_arc, aux_labels, score_offsets = _k2.levenshtein_graph(
+        symbols, ins_del_score, True)


If ins_del_score_offset_attr is None, then we don't need to compute score_offset
and need_score_offset can be set to False.

csukuangfj · 2021-09-18T11:51:33Z

+2

* Make k2 ragged tensor more PyTorch-y like. * Refactoring: Start to add the wrapper class AnyTensor. * Refactoring. * initial attempt to support autograd. * First working version with autograd for Sum(). * Fix comments. * Support __getitem__ and pickling. * Add more docs for k2.ragged.Tensor * Put documentation in header files. * Minor fixes. * Fix a typo. * Fix an error. * Add more doc. * Wrap RaggedShape. * [Not for Merge]: Move k2.Fsa related code to C++. * Remove extra files. * Update doc URL. (#821) * Support manipulating attributes of k2.ragged.Fsa. * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Add Fsa.get_forward_scores. * Implement backprop for Fsa.get_forward_scores() * Construct RaggedArc from unary function tensor (#30) * Construct RaggedArc from unary function tensor * Move fsa_from_unary_ragged and fsa_from_binary_tensor to C++ * add unit test to from unary function; add more functions to fsa * Remove some rabbish code * Add more unit tests and docs * Remove the unused code * Fix review comments, propagate attributes in To() * Change the argument type from RaggedAny to Ragged<int32_t> in autograd function * Delete declaration for template function * Apply suggestions from code review Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> * Fix documentation errors Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Wei Kang <wkang@pku.org.cn>

* Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Revert "Construct RaggedArc from unary function tensor (#30)" (#31) This reverts commit cca7a54. * Remove unused code. * Fix github actions. Avoid downloading all git LFS files. * Enable github actions for v2.0-pre branch. Co-authored-by: Wei Kang <wkang@pku.org.cn> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com>

* Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org>

* Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> * Add Hash64 (#895) * Add hash64 * Fix tests * Resize hash64 * Fix comments * fix typo * Modified rnnt (#902) * Add modified mutual_information_recursion * Add modified rnnt loss * Using more efficient way to fix boundaries * Fix modified pruned rnnt loss * Fix the s_begin constrains of pruned loss for modified version transducer * Fix Stack (#925) * return the correct layer * unskip the test * Fix 'TypeError' of rnnt_loss_pruned function. (#924) * Fix 'TypeError' of rnnt_loss_simple function. Fix 'TypeError' exception when calling rnnt_loss_simple(..., return_grad=False) at validation steps. * Fix 'MutualInformationRecursionFunction.forward()' return type check error for pytorch < 1.10.x * Modify return type. * Add documents about class MutualInformationRecursionFunction. * Formated code style. * Fix rnnt_loss_smoothed return type. Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Support torch 1.11.0 and CUDA 11.5 (#931) * Support torch 1.11.0 and CUDA 11.5 * Implement Rnnt decoding (#926) * first working draft of rnnt decoding * FormatOutput works... * Different num frames for FormatOutput works * Update docs * Fix comments, break advance into several stages, add more docs * Add python wrapper * Add more docs * Minor fixes * Fix comments * fix building docs (#933) * Release v1.14 * Remove unused DiscountedCumSum. (#936) * Fix compiler warnings. (#937) * Fix compiler warnings. * Minor fixes for RNN-T decoding. (#938) * Minor fixes for RNN-T decoding. * Removes arcs with label 0 from the TrivialGraph. (#939) * Implement linear_fsa_with_self_loops. (#940) * Implement linear_fsa_with_self_loops. * Fix the pruning with max-states (#941) Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> Co-authored-by: Wang, Guanbo <wgb14@outlook.com>

* Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> * Add Hash64 (#895) * Add hash64 * Fix tests * Resize hash64 * Fix comments * fix typo * Modified rnnt (#902) * Add modified mutual_information_recursion * Add modified rnnt loss * Using more efficient way to fix boundaries * Fix modified pruned rnnt loss * Fix the s_begin constrains of pruned loss for modified version transducer * Fix Stack (#925) * return the correct layer * unskip the test * Fix 'TypeError' of rnnt_loss_pruned function. (#924) * Fix 'TypeError' of rnnt_loss_simple function. Fix 'TypeError' exception when calling rnnt_loss_simple(..., return_grad=False) at validation steps. * Fix 'MutualInformationRecursionFunction.forward()' return type check error for pytorch < 1.10.x * Modify return type. * Add documents about class MutualInformationRecursionFunction. * Formated code style. * Fix rnnt_loss_smoothed return type. Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Support torch 1.11.0 and CUDA 11.5 (#931) * Support torch 1.11.0 and CUDA 11.5 * Implement Rnnt decoding (#926) * first working draft of rnnt decoding * FormatOutput works... * Different num frames for FormatOutput works * Update docs * Fix comments, break advance into several stages, add more docs * Add python wrapper * Add more docs * Minor fixes * Fix comments * fix building docs (#933) * Release v1.14 * Remove unused DiscountedCumSum. (#936) * Fix compiler warnings. (#937) * Fix compiler warnings. * Minor fixes for RNN-T decoding. (#938) * Minor fixes for RNN-T decoding. * Removes arcs with label 0 from the TrivialGraph. (#939) * Implement linear_fsa_with_self_loops. (#940) * Implement linear_fsa_with_self_loops. * Fix the pruning with max-states (#941) * Rnnt allow different encoder/decoder dims (#945) * Allow different encoder and decoder dim in rnnt_pruning * Bug fixes * Supporting building k2 on Windows (#946) * Fix nightly windows CPU build (#948) * Fix nightly building k2 for windows. * Run nightly build only if there are new commits. * Check the versions of PyTorch and CUDA at the import time. (#949) * Check the versions of PyTorch and CUDA at the import time. * More straightforward message when CUDA support is missing (#950) * Implement ArrayOfRagged (#927) * Implement ArrayOfRagged * Fix issues and pass tests * fix style * change few statements of functions and move the definiation of template Array1OfRagged to header file * add offsets test code * Fix precision (#951) * Fix precision * Using different pow version for windows and *nix * Use int64_t pow * Minor fixes Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> Co-authored-by: Wang, Guanbo <wgb14@outlook.com> Co-authored-by: Nickolay V. Shmyrev <nshmyrev@gmail.com> Co-authored-by: LvHang <hanglyu1991@gmail.com>

* [WIP]: Move k2.Fsa to C++ (#814) * Make k2 ragged tensor more PyTorch-y like. * Refactoring: Start to add the wrapper class AnyTensor. * Refactoring. * initial attempt to support autograd. * First working version with autograd for Sum(). * Fix comments. * Support __getitem__ and pickling. * Add more docs for k2.ragged.Tensor * Put documentation in header files. * Minor fixes. * Fix a typo. * Fix an error. * Add more doc. * Wrap RaggedShape. * [Not for Merge]: Move k2.Fsa related code to C++. * Remove extra files. * Update doc URL. (#821) * Support manipulating attributes of k2.ragged.Fsa. * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Add Fsa.get_forward_scores. * Implement backprop for Fsa.get_forward_scores() * Construct RaggedArc from unary function tensor (#30) * Construct RaggedArc from unary function tensor * Move fsa_from_unary_ragged and fsa_from_binary_tensor to C++ * add unit test to from unary function; add more functions to fsa * Remove some rabbish code * Add more unit tests and docs * Remove the unused code * Fix review comments, propagate attributes in To() * Change the argument type from RaggedAny to Ragged<int32_t> in autograd function * Delete declaration for template function * Apply suggestions from code review Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> * Fix documentation errors Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Wei Kang <wkang@pku.org.cn> * Remove pybind dependencies from RaggedArc. (#842) * Convert py::object and torch::IValue to each other * Remove py::object from RaggedAny * Remove py::object from RaggedArc * Move files to torch directory * remove unused files * Add unit tests * Remove v2 folder * Remove unused code * Remove unused files * Fix review comments & fix github actions * Check Ivalue contains RaggedAny * Minor fixes * Add attributes related unit test for FsaClass * Fix mutable_grad in older pytorch version * Fix github actions * Fix github action PYTHONPATH * Fix github action PYTHONPATH * Link pybind11::embed * import torch first (to fix macos github actions) * try to fix macos ci * Revert "Remove pybind dependencies from RaggedArc. (#842)" (#855) This reverts commit daa98e7. * Support torchscript. (#839) * WIP: Support torchscript. * Test jit module with faked data. I have compared the output from C++ with that from Python. The sums of the tensors are equal. * Use precomputed features to test the correctness. * Build DenseFsaVec from a torch tensor. * Get lattice for CTC decoding. * Support CTC decoding. * Link sentencepiece statically. Link sentencepiece dynamically causes segmentation fault at the end of the process. * Support loading HLG.pt * Refactoring. * Implement HLG decoding. * Add WaveReader to read wave sound files. * Take soundfiles as inputs. * Refactoring. * Support GPU. * Minor fixes. * Fix typos. * Use kaldifeat v1.7 * Add copyright info. * Fix compilation for torch >= 1.9.0 * Minor fixes. * Fix comments. * Fix style issues. * Fix compiler warnings. * Use `torch::class_` to register custom classes. (#856) * Remove unused code (#857) * Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Revert "Construct RaggedArc from unary function tensor (#30)" (#31) This reverts commit cca7a54. * Remove unused code. * Fix github actions. Avoid downloading all git LFS files. * Enable github actions for v2.0-pre branch. Co-authored-by: Wei Kang <wkang@pku.org.cn> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> * Implements Cpp version FsaClass (#858) * Add C++ version FsaClass * Propagates attributes for CreateFsaVec * Add more docs * Remove the code that unnecessary needed currently * Remove the code unnecessary for ctc decoding & HLG decoding * Update k2/torch/csrc/deserialization.h Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> * Fix Comments * Fix code style Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> * Using FsaClass for ctc decoding & HLG decoding (#862) * Using FsaClass for ctc decoding & HLG decoding * Update docs * fix evaluating kFsaPropertiesValid (#866) * Refactor deserialization code (#863) * Fix compiler warnings about the usage of `tmpnam`. * Refactor deserialization code. * Minor fixes. * Support rescoring with an n-gram LM during decoding (#867) * Fix compiler warnings about the usage of `tmpnam`. * Refactor deserialization code. * Minor fixes. * Add n-gram LM rescoring. * Minor fixes. * Clear cached FSA properties when its labels are changed. * Fix typos. * Refactor FsaClass. (#868) Since FSAs in decoding contain only one or two attributes, we don't need to use an IValue to add one more indirection. Just check the type of the attribute and process it correspondingly. * Refactor bin/decode.cu (#869) * Add CTC decode. * Add HLG decoding. * Add n-gram LM rescoring. * Remove unused files. * Fix style issues. * Add missing files. * Add attention rescoring. (#870) * WIP: Add attention rescoring. * Finish attention rescoring. * Fix style issues. * Resolve comments. (#871) * Resolve comments. * Minor fixes. * update v2.0-pre (#922) * Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> * Online decoding (#876) * Add OnlineIntersectDensePruned * Fix get partial results * Support online decoding on intersect_dense_pruned * Update documents * Update v2.0-pre (#942) * Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> * Add Hash64 (#895) * Add hash64 * Fix tests * Resize hash64 * Fix comments * fix typo * Modified rnnt (#902) * Add modified mutual_information_recursion * Add modified rnnt loss * Using more efficient way to fix boundaries * Fix modified pruned rnnt loss * Fix the s_begin constrains of pruned loss for modified version transducer * Fix Stack (#925) * return the correct layer * unskip the test * Fix 'TypeError' of rnnt_loss_pruned function. (#924) * Fix 'TypeError' of rnnt_loss_simple function. Fix 'TypeError' exception when calling rnnt_loss_simple(..., return_grad=False) at validation steps. * Fix 'MutualInformationRecursionFunction.forward()' return type check error for pytorch < 1.10.x * Modify return type. * Add documents about class MutualInformationRecursionFunction. * Formated code style. * Fix rnnt_loss_smoothed return type. Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Support torch 1.11.0 and CUDA 11.5 (#931) * Support torch 1.11.0 and CUDA 11.5 * Implement Rnnt decoding (#926) * first working draft of rnnt decoding * FormatOutput works... * Different num frames for FormatOutput works * Update docs * Fix comments, break advance into several stages, add more docs * Add python wrapper * Add more docs * Minor fixes * Fix comments * fix building docs (#933) * Release v1.14 * Remove unused DiscountedCumSum. (#936) * Fix compiler warnings. (#937) * Fix compiler warnings. * Minor fixes for RNN-T decoding. (#938) * Minor fixes for RNN-T decoding. * Removes arcs with label 0 from the TrivialGraph. (#939) * Implement linear_fsa_with_self_loops. (#940) * Implement linear_fsa_with_self_loops. * Fix the pruning with max-states (#941) Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> Co-authored-by: Wang, Guanbo <wgb14@outlook.com> * update v2.0-pre (#953) * Update doc URL. (#821) * Support indexing 2-axes RaggedTensor, Support slicing for RaggedTensor (#825) * Support index 2-axes RaggedTensor, Support slicing for RaggedTensor * Fix compiling errors * Fix unit test * Change RaggedTensor.data to RaggedTensor.values * Fix style * Add docs * Run nightly-cpu when pushing code to nightly-cpu branch * Prune with max_arcs in IntersectDense (#820) * Add checking for array constructor * Prune with max arcs * Minor fix * Fix typo * Fix review comments * Fix typo * Release v1.8 * Create a ragged tensor from a regular tensor. (#827) * Create a ragged tensor from a regular tensor. * Add tests for creating ragged tensors from regular tensors. * Add more tests. * Print ragged tensors in a way like what PyTorch is doing. * Fix test cases. * Trigger GitHub actions manually. (#829) * Run GitHub actions on merging. (#830) * Support printing ragged tensors in a more compact way. (#831) * Support printing ragged tensors in a more compact way. * Disable support for torch 1.3.1 * Fix test failures. * Add levenshtein alignment (#828) * Add levenshtein graph * Contruct k2.RaggedTensor in python part * Fix review comments, return aux_labels in ctc_graph * Fix tests * Fix bug of accessing symbols * Fix bug of accessing symbols * Change argument name, add levenshtein_distance interface * Fix test error, add tests for levenshtein_distance * Fix review comments and add unit test for c++ side * update the interface of levenshtein alignment * Fix review comments * Release v1.9 * Support a[b[i]] where both a and b are ragged tensors. (#833) * Display import error solution message on MacOS (#837) * Fix installation doc. (#841) * Fix installation doc. Remove Windows support. Will fix it later. * Fix style issues. * fix typos in the install instructions (#844) * make cmake adhere to the modernized way of finding packages outside default dirs (#845) * import torch first in the smoke tests to preven SEGFAULT (#846) * Add doc about how to install a CPU version of k2. (#850) * Add doc about how to install a CPU version of k2. * Remove property setter of Fsa.labels * Update Ubuntu version in GitHub CI since 16.04 reaches end-of-life. * Support PyTorch 1.10. (#851) * Fix test cases for k2.union() (#853) * Fix out-of-boundary access (read). (#859) * Update all the example codes in the docs (#861) * Update all the example codes in the docs I have run all the modified codes with the newest version k2. * do some changes * Fix compilation errors with CUB 1.15. (#865) * Update README. (#873) * Update README. * Fix typos. * Fix ctc graph (make aux_labels of final arcs -1) (#877) * Fix LICENSE location to k2 folder (#880) * Release v1.11. (#881) It contains bugfixes. * Update documentation for hash.h (#887) * Update documentation for hash.h * Typo fix * Wrap MonotonicLowerBound (#883) * Wrap MonotonicLowerBound * Add unit tests * Support int64; update documents * Remove extra commas after 'TOPSORTED' properity and fix RaggedTensor constructer parameter 'byte_offset' out-of-range bug. (#892) Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Fix small typos (#896) * Fix k2.ragged.create_ragged_shape2 (#901) Before the fix, we have to specify both `row_splits` and `row_ids` while calling `k2.create_ragged_shape2` even if one of them is `None`. After this fix, we only need to specify one of them. * Add rnnt loss (#891) * Add cpp code of mutual information * mutual information working * Add rnnt loss * Add pruned rnnt loss * Minor Fixes * Minor fixes & fix code style * Fix cpp style * Fix code style * Fix s_begin values in padding positions * Fix bugs related to boundary; Fix s_begin padding value; Add more tests * Minor fixes * Fix comments * Add boundary to pruned loss tests * Use more efficient way to fix boundaries (#906) * Release v1.12 (#907) * Change the sign of the rnnt_loss and add reduction argument (#911) * Add right boundary constrains for s_begin * Minor fixes to the interface of rnnt_loss to make it return positive value * Fix comments * Release a new version * Minor fixes * Minor fixes to the docs * Fix building doc. (#908) * Fix building doc. * Minor fixes. * Minor fixes. * Fix building doc (#912) * Fix building doc * Fix flake8 * Support torch 1.10.x (#914) * Support torch 1.10.x * Fix installing PyTorch. * Update INSTALL.rst (#915) * Update INSTALL.rst Setting a few additional env variables to enable compilation from source *with CUDA GPU computation support enabled* * Fix torch/cuda/python versions in the doc. (#918) * Fix torch/cuda/python versions in the doc. * Minor fixes. * Fix building for CUDA 11.6 (#917) * Fix building for CUDA 11.6 * Minor fixes. * Implement Unstack (#920) * Implement unstack * Remove code does not relate to this PR * Remove for loop on output dim; add Unstack ragged * Add more docs * Fix comments * Fix docs & unit tests * SubsetRagged & PruneRagged (#919) * Extend interface of SubsampleRagged. * Add interface for pruning ragged tensor. * Draft of new RNN-T decoding method * Implements SubsampleRaggedShape * Implements PruneRagged * Rename subsample-> subset * Minor fixes * Fix comments Co-authored-by: Daniel Povey <dpovey@gmail.com> * Add Hash64 (#895) * Add hash64 * Fix tests * Resize hash64 * Fix comments * fix typo * Modified rnnt (#902) * Add modified mutual_information_recursion * Add modified rnnt loss * Using more efficient way to fix boundaries * Fix modified pruned rnnt loss * Fix the s_begin constrains of pruned loss for modified version transducer * Fix Stack (#925) * return the correct layer * unskip the test * Fix 'TypeError' of rnnt_loss_pruned function. (#924) * Fix 'TypeError' of rnnt_loss_simple function. Fix 'TypeError' exception when calling rnnt_loss_simple(..., return_grad=False) at validation steps. * Fix 'MutualInformationRecursionFunction.forward()' return type check error for pytorch < 1.10.x * Modify return type. * Add documents about class MutualInformationRecursionFunction. * Formated code style. * Fix rnnt_loss_smoothed return type. Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> * Support torch 1.11.0 and CUDA 11.5 (#931) * Support torch 1.11.0 and CUDA 11.5 * Implement Rnnt decoding (#926) * first working draft of rnnt decoding * FormatOutput works... * Different num frames for FormatOutput works * Update docs * Fix comments, break advance into several stages, add more docs * Add python wrapper * Add more docs * Minor fixes * Fix comments * fix building docs (#933) * Release v1.14 * Remove unused DiscountedCumSum. (#936) * Fix compiler warnings. (#937) * Fix compiler warnings. * Minor fixes for RNN-T decoding. (#938) * Minor fixes for RNN-T decoding. * Removes arcs with label 0 from the TrivialGraph. (#939) * Implement linear_fsa_with_self_loops. (#940) * Implement linear_fsa_with_self_loops. * Fix the pruning with max-states (#941) * Rnnt allow different encoder/decoder dims (#945) * Allow different encoder and decoder dim in rnnt_pruning * Bug fixes * Supporting building k2 on Windows (#946) * Fix nightly windows CPU build (#948) * Fix nightly building k2 for windows. * Run nightly build only if there are new commits. * Check the versions of PyTorch and CUDA at the import time. (#949) * Check the versions of PyTorch and CUDA at the import time. * More straightforward message when CUDA support is missing (#950) * Implement ArrayOfRagged (#927) * Implement ArrayOfRagged * Fix issues and pass tests * fix style * change few statements of functions and move the definiation of template Array1OfRagged to header file * add offsets test code * Fix precision (#951) * Fix precision * Using different pow version for windows and *nix * Use int64_t pow * Minor fixes Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> Co-authored-by: Wang, Guanbo <wgb14@outlook.com> Co-authored-by: Nickolay V. Shmyrev <nshmyrev@gmail.com> Co-authored-by: LvHang <hanglyu1991@gmail.com> * Add C++ Rnnt demo (#947) * rnnt_demo compiles * Change graph in RnntDecodingStream from shared_ptr to const reference * Change out_map from Array1 to Ragged * Add rnnt demo * Minor fixes * Add more docs * Support log_add when getting best path * Port kaldi::ParseOptions for parsing commandline options. (#974) * Port kaldi::ParseOptions for parsing commandline options. * Add more tests. * More tests. * Greedy search and modified beam search for pruned stateless RNN-T. (#975) * First version of greedy search. * WIP: Implement modified beam search and greedy search for pruned RNN-T. * Implement modified beam search. * Fix compiler warnings * Fix style issues * Update torch_api.h to include APIs for CTC decoding Co-authored-by: Wei Kang <wkang@pku.org.cn> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Jan "yenda" Trmal <jtrmal@gmail.com> Co-authored-by: pingfengluo <pingfengluo@gmail.com> Co-authored-by: Mingshuang Luo <37799481+luomingshuang@users.noreply.github.com> Co-authored-by: Ludwig Kürzinger <lumaku@users.noreply.github.com> Co-authored-by: Daniel Povey <dpovey@gmail.com> Co-authored-by: drawfish <duisheng.chen@gmail.com> Co-authored-by: gzchenduisheng <gzchenduisheng@corp.netease.com> Co-authored-by: alexei-v-ivanov <alexei_v_ivanov@ieee.org> Co-authored-by: Wang, Guanbo <wgb14@outlook.com> Co-authored-by: Nickolay V. Shmyrev <nshmyrev@gmail.com> Co-authored-by: LvHang <hanglyu1991@gmail.com>

pkufool added 2 commits September 15, 2021 20:25

Add levenshtein graph

6b0349c

Contruct k2.RaggedTensor in python part

03fa978

csukuangfj reviewed Sep 15, 2021

View reviewed changes

pkufool commented Sep 15, 2021

View reviewed changes

csukuangfj reviewed Sep 15, 2021

View reviewed changes

pkufool added 5 commits September 15, 2021 23:09

Merge branch 'master' into levenshtein

75f8ae9

Fix review comments, return aux_labels in ctc_graph

3d977ce

Fix tests

fd2cfcb

Fix bug of accessing symbols

02bfb65

Fix bug of accessing symbols

1df027a

danpovey reviewed Sep 16, 2021

View reviewed changes

pkufool added 3 commits September 16, 2021 13:58

Change argument name, add levenshtein_distance interface

96c9fee

Merge branch 'master' into levenshtein

50a763d

Fix test error, add tests for levenshtein_distance

116ddb5

pkufool commented Sep 17, 2021

View reviewed changes

csukuangfj reviewed Sep 17, 2021

View reviewed changes

Fix review comments and add unit test for c++ side

d5cb765

csukuangfj reviewed Sep 17, 2021

View reviewed changes

pkufool added 2 commits September 17, 2021 16:21

update the interface of levenshtein alignment

c27b320

Fix conflicts

c8afee8

pkufool added the ready Ready for review and trigger GitHub actions to run label Sep 17, 2021

pkufool changed the title ~~Add levenshtein graph~~ Add levenshtein alignment Sep 17, 2021

csukuangfj reviewed Sep 18, 2021

View reviewed changes

csukuangfj mentioned this pull request Sep 18, 2021

Refactor decode.py to make it more readable and more modular. k2-fsa/icefall#44

Merged

Fix review comments

6b10609

pkufool added ready Ready for review and trigger GitHub actions to run and removed ready Ready for review and trigger GitHub actions to run labels Sep 18, 2021

pkufool merged commit d73a5b5 into k2-fsa:master Sep 19, 2021

pkufool deleted the levenshtein branch October 1, 2021 08:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add levenshtein alignment #828

Add levenshtein alignment #828

pkufool commented Sep 15, 2021 •

edited

Loading

pkufool commented Sep 15, 2021 •

edited

Loading

csukuangfj Sep 15, 2021

pkufool Sep 15, 2021

csukuangfj Sep 15, 2021

pkufool Sep 16, 2021

csukuangfj Sep 15, 2021

csukuangfj Sep 15, 2021

csukuangfj Sep 15, 2021

csukuangfj Sep 15, 2021

csukuangfj Sep 15, 2021

csukuangfj Sep 15, 2021

danpovey Sep 16, 2021

pkufool Sep 17, 2021

pkufool commented Sep 17, 2021 •

edited

Loading

csukuangfj Sep 17, 2021

csukuangfj Sep 17, 2021

csukuangfj Sep 17, 2021

csukuangfj Sep 17, 2021

csukuangfj Sep 17, 2021

csukuangfj Sep 17, 2021

csukuangfj Sep 18, 2021

csukuangfj Sep 18, 2021

csukuangfj commented Sep 18, 2021

	if (state_idx01 == final_state)
	else if (state_idx01 == final_state)

	[](RaggedAny &symbols, torch::optional<torch::Device> = {},
	[](RaggedAny &symbols

	fsa.__setattr__(name=ins_del_score_offset_attr, value=score_offsets)
	setattr(fsa, ins_del_score_offset_attr, score_offsets)

	RaggedShape2(&states_to_arcs_row_splits, nullptr, -1);
	int32_t num_arcs = symbols.NumElements() * 3 + symbols.Dim0() * 2;
	RaggedShape2(&states_to_arcs_row_splits, nullptr, num_arcs);

Add levenshtein alignment #828

Add levenshtein alignment #828

Conversation

pkufool commented Sep 15, 2021 • edited Loading

pkufool commented Sep 15, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkufool commented Sep 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csukuangfj commented Sep 18, 2021

pkufool commented Sep 15, 2021 •

edited

Loading

pkufool commented Sep 15, 2021 •

edited

Loading

pkufool commented Sep 17, 2021 •

edited

Loading