Skip to content

Commit

Permalink
We accept TranslationRequest now, and vector<text>
Browse files Browse the repository at this point in the history
Towards #77.

- What was earlier translateWithOptions is now renamed to translate,
  making it a method overload based on arguments.
- A new translateMultiple is provided, which is blocking. This allows
  the browser to sent separate vector<text> which is queued as a whole,
  but each text maps to a Request each for which there is a Response. This
  is closer to what exists in TranslationModel as of now, except there are
  speed gains from properly batching. translateMultiple accepts
  TranslationRequest as well, although it does nothing with it.
  • Loading branch information
Jerin Philip committed Apr 7, 2021
1 parent c7400f6 commit 5bf29c1
Show file tree
Hide file tree
Showing 3 changed files with 66 additions and 8 deletions.
2 changes: 1 addition & 1 deletion app/service-cli.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ int main(int argc, char *argv[]) {

// Wait on future until Response is complete
std::future<Response> responseFuture =
service.translateWithOptions(std::move(input), responseOptions);
service.translate(std::move(input), responseOptions);
responseFuture.wait();
Response response = responseFuture.get();

Expand Down
45 changes: 40 additions & 5 deletions src/translator/service.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,36 @@ std::future<Response> Service::translate(std::string &&input) {
New<Options>(); // TODO(jerinphilip): Take this in as argument

// Hardcode responseOptions for now
return translateWithOptions(std::move(input), responseOptions);
return translate(std::move(input), responseOptions);
}

std::future<Response>
Service::translateWithOptions(std::string &&input,
ResponseOptions responseOptions) {
std::vector<Response>
Service::translateMultiple(std::vector<std::string> &&inputs,
TranslationRequest translationRequest) {
ResponseOptions responseOptions = New<Options>();

// TODO(jerinphilip) Set options based on TranslationRequest, if and when it
// becomes non-dummy.

std::vector<std::future<Response>> responseFutures;
for (auto &input : inputs) {
std::future<Response> inputResponse =
queueRequest(std::move(input), responseOptions);
responseFutures.push_back(std::move(inputResponse));
}

dispatchTranslate();

std::vector<Response> responses;
for (auto &future : responseFutures) {
future.wait();
responses.push_back(std::move(future.get()));
}
return responses;
}

std::future<Response> Service::queueRequest(std::string &&input,
ResponseOptions responseOptions) {
Segments segments;
AnnotatedText source(std::move(input));
text_processor_.process(source, segments);
Expand All @@ -138,12 +162,23 @@ Service::translateWithOptions(std::string &&input,
std::move(responseBuilder));

batcher_.addWholeRequest(request);
return future;
}

std::future<Response> Service::translate(std::string &&input,
ResponseOptions responseOptions) {
std::future<Response> future =
queueRequest(std::move(input), responseOptions);
dispatchTranslate();
return future;
}

void Service::dispatchTranslate() {
if (numWorkers_ == 0) {
blocking_translate();
} else {
async_translate();
}
return future;
}

Service::~Service() {
Expand Down
27 changes: 25 additions & 2 deletions src/translator/service.h
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#ifndef SRC_BERGAMOT_SERVICE_H_
#define SRC_BERGAMOT_SERVICE_H_

#include "TranslationRequest.h"
#include "batch_translator.h"
#include "batcher.h"
#include "data/types.h"
Expand Down Expand Up @@ -104,10 +105,32 @@ class Service {
/// @param [in] responseOptions: Options indicating whether or not to include
/// some member in the Response, also specify any additional configurable
/// parameters.
std::future<Response> translateWithOptions(std::string &&source,
ResponseOptions options);
std::future<Response> translate(std::string &&source,
ResponseOptions options);

/// Translate an input, providing TranslationRequest across all texts to
/// construct Response. Provides the browser with the ability to break texts
/// into multiple Request keeping gains from efficiently batching internally.
/// Also useful when one has to set/unset alignments or quality in the
/// Response to save compute spent in constructing these objects.

/// @param [in] source: rvalue reference of the string to be translated /
/// @param [in] translationRequest: TranslationRequest (Unified API)
/// indicating whether or not to include some member in the Response, also
/// specify any / additional configurable parameters.

std::vector<Response>
translateMultiple(std::vector<std::string> &&source,
TranslationRequest translationRequest);

private:
/// Queue an input for translation.
std::future<Response> queueRequest(std::string &&input,
ResponseOptions responseOptions);

/// Dispatch call to translate after inserting in queue
void dispatchTranslate();

/// Build numTranslators number of translators with options from options
void build_translators(Ptr<Options> options, size_t numTranslators);
/// Initializes a blocking translator without using std::thread
Expand Down

0 comments on commit 5bf29c1

Please sign in to comment.