[runtime] add server runtime support #29
Conversation
Change-Id: Iaeac0b6819bde5c7d63eada284c83835af8424e7
Change-Id: Ice7102e96400ea150460e511e6c1f179de045016
Change-Id: If7f111517e57c1fce81611d1ae638885596ccc42
Change-Id: Ifa561e2d91011de976bc702b5b0ff35e9915fc79
Change-Id: Id8ac62fe1ba9bf852b2bae13d4e699ab01983542
Change-Id: Iee7c09a50a9725a75da78b694889ff63749c9bf2
Change-Id: I40560a6ca01643dfbd2d6dce24cb1abbad433ed6
Change-Id: I5d090782407e97df34603db7f49fdfb249bf19c4
Change-Id: I326a03bd290d37b6fdb29ad8b99b243aabfcc103
Change-Id: I8f91439dad1b325f8d3efcf3aa7e4630be05853b
Change-Id: I1a69780c1880cd1b85796d4597fe368ea14049f5
Change-Id: Id3c4921c4969984c72bb224141a9c53337c32a5e
Change-Id: Icb2d66939397af393e2ec700d7f94ffe2cf027fc
Change-Id: I8d31748fd2656cb78baf058b616dfb7f6b8c446f
Change-Id: I8a2dac65729e766cec796d01e05a036c26f7810f
Change-Id: I8e6fc9af247d8bc55369b6a127320e6a956dd4fd
robin1001
requested review from
jschenxiaoyu,
placebokkk,
glynpu,
whiteshirt0429 and
XiangmingWang
Change-Id: I233c2ea043ac2dfc50403c2613f29d9e1fa1de00
runtime/server/x86/decoder/BUILD
Outdated
| srcs = ['ctc_prefix_beam_search.cc'], | ||
| hdrs = ['ctc_prefix_beam_search.h'], | ||
| deps = [ | ||
| "@com_github_google_glog//:glog", |
| cur_hyps_.clear(); | ||
| PrefixScore prefix_score(0.0, -std::numeric_limits<float>::max()); | ||
| std::vector<int> empty; | ||
| cur_hyps_[empty] = prefix_score; |
There was a problem hiding this comment.
Add BeginDecode() function to process cur_hyps_[vector()] = prefix_score
| @@ -0,0 +1,108 @@ | |||
| // Copyright (c) 2016 HR | |||
| cc_library( | ||
| name = 'utils', | ||
| hdrs = ['blocking_queue.h'], | ||
| deps = [ |
| while (queue_.empty()) { | ||
| not_empty_condition_.wait(lock); | ||
| } | ||
| T t(std::move(queue_.front())); |
There was a problem hiding this comment.
T t = queue_.front();
q.front() returns reference
There was a problem hiding this comment.
T t(std::move(queue_.front())); is fine.
| not_empty_condition_.notify_one(); | ||
| } | ||
|
|
||
| T Pop() { |
There was a problem hiding this comment.
generally, Pop doesn't return element.
There was a problem hiding this comment.
STL doesn't, but most of the Queue design does
| mutable std::mutex mutex_; | ||
| std::condition_variable not_full_condition_; | ||
| std::condition_variable not_empty_condition_; | ||
| std::queue<T> queue_; |
There was a problem hiding this comment.
Actually, it is not needed here as std::mutex is not copyable or assignable.
Change-Id: I6ef0d112d395c8d1ba8efb6fc00eca28e47dbc3a
| PreEmphasis(0.97, &data); | ||
| Hamming(&data); | ||
| // copy data to fft_real | ||
| memset(fft_real.data(), 0, sizeof(float) * fft_points_); |
There was a problem hiding this comment.
memset(fft_real.data() + frame_length_, 0, sizeof(float) * (fft_points_ - frame_length_));There was a problem hiding this comment.
why should we subscribe frame_length here?
There was a problem hiding this comment.
Because it is fft_real.data() + frame_length_. Otherwise, you may get segfault.
| memset(fft_real.data(), 0, sizeof(float) * fft_points_); | ||
| memset(fft_img.data(), 0, sizeof(float) * fft_points_); | ||
| memcpy(fft_real.data(), data.data(), sizeof(float) * frame_length_); | ||
| fft(fft_real.data(), fft_img.data(), fft_points_); |
There was a problem hiding this comment.
There should be a function rfft for real numbers, which should be more efficient.
There was a problem hiding this comment.
yes, i didn't go through the details of fft or rfft, so just use one I have used.
| wave.data() + i * frame_shift_ + frame_length_); | ||
| // optional add noise | ||
| if (dither_ != 0.0) { | ||
| for (size_t j = 0; j < data.size(); j++) |
There was a problem hiding this comment.
Use torch::rand to generate random numbers.
Use operations for torch::Tensor to do BLAS operations.
There was a problem hiding this comment.
In our design, torch Tensor is only used for inference part, so we didn't use it here.
There was a problem hiding this comment.
You're depending libtorch anyway. Can't the design be changed or is your approach faster?
There was a problem hiding this comment.
Okay, I will refine if I know the performance details of torch tensor
| @@ -0,0 +1,108 @@ | |||
| // Copyright (c) 2016 HR | |||
There was a problem hiding this comment.
Here just keep it, will use a more efficient fft later.
| CHECK(str != NULL); | ||
| CHECK_GE(id, 0); | ||
| std::string symbol = str; | ||
| symbol_tabel_[id] = symbol; |
There was a problem hiding this comment.
Check that there are no repeated ids or symbols.
| float mel_high_freq = MelScale(high_freq); | ||
| float mel_freq_delta = (mel_high_freq - mel_low_freq) / (num_bins+1); | ||
| bins_.resize(num_bins_); | ||
| center_freqs_.resize(num_bins_); |
There was a problem hiding this comment.
Do you need to mention that the code is copied/modified from kaldi?
| mutable std::mutex mutex_; | ||
| std::condition_variable not_full_condition_; | ||
| std::condition_variable not_empty_condition_; | ||
| std::queue<T> queue_; |
There was a problem hiding this comment.
Actually, it is not needed here as std::mutex is not copyable or assignable.
| // copy data to fft_real | ||
| memset(fft_real.data(), 0, sizeof(float) * fft_points_); | ||
| memset(fft_img.data(), 0, sizeof(float) * fft_points_); | ||
| memcpy(fft_real.data(), data.data(), sizeof(float) * frame_length_); |
There was a problem hiding this comment.
You are going to assign fft_real.data()[0] = data.data[0]; Is there any point
to do fft_real.data()[0] = 0 ?
There was a problem hiding this comment.
this is required in the fft function
There was a problem hiding this comment.
I know that. But you are doing
fft_real.data()[0] = 0; // this one can be removed
fft_real.data()[0] = data.data[0];
There was a problem hiding this comment.
Okay I see, will fix it.
* [runtime] x86 compile okay Change-Id: Iaeac0b6819bde5c7d63eada284c83835af8424e7 * add gtest and ctc beam search works okay Change-Id: Ice7102e96400ea150460e511e6c1f179de045016 * add frontend feature pipeline Change-Id: If7f111517e57c1fce81611d1ae638885596ccc42 * solve glog conflicts, add torch asr model Change-Id: Ifa561e2d91011de976bc702b5b0ff35e9915fc79 * refine asr model, use shared_ptr Change-Id: Id8ac62fe1ba9bf852b2bae13d4e699ab01983542 * refine feature pipeline Change-Id: Iee7c09a50a9725a75da78b694889ff63749c9bf2 * add blocking queue Change-Id: I40560a6ca01643dfbd2d6dce24cb1abbad433ed6 * ctc prefix beam search works Change-Id: I5d090782407e97df34603db7f49fdfb249bf19c4 * add rescoring support Change-Id: I326a03bd290d37b6fdb29ad8b99b243aabfcc103 * add cpplint ci Change-Id: I8f91439dad1b325f8d3efcf3aa7e4630be05853b * add cpplint annotations Change-Id: I1a69780c1880cd1b85796d4597fe368ea14049f5 * refine cpplint annotations path Change-Id: Id3c4921c4969984c72bb224141a9c53337c32a5e * refine cpplint annotation format Change-Id: Icb2d66939397af393e2ec700d7f94ffe2cf027fc * add CPPLINT.cfg Change-Id: I8d31748fd2656cb78baf058b616dfb7f6b8c446f * refine style Change-Id: I8a2dac65729e766cec796d01e05a036c26f7810f * add eval mode Change-Id: I8e6fc9af247d8bc55369b6a127320e6a956dd4fd * remove extra space in blocking_queue.h Change-Id: I233c2ea043ac2dfc50403c2613f29d9e1fa1de00 * add disallow copy and assign Change-Id: I6ef0d112d395c8d1ba8efb6fc00eca28e47dbc3a
No description provided.