Callback on Request completion to construct Response

[jerinphilip]

The changes proposed in this issue is to be implemented if approved following Alignments PR (#46), and can help #56, #53.

tl;dr: f(x; \theta) = ResponseBuilder(histories; vocabs, std::promise<Response>)

It's a bit unnatural to have std::promise<Response> and vocabs in Request, which should only hold request information.

bergamot-translator/src/translator/request.h

Lines 38 to 40 in 317433a

	Request(size_t Id, size_t lineNumberBegin,
	std::vector<Ptr<Vocab const>> &vocabs_, AnnotatedBlob &&source,
	Segments &&segments, std::promise<Response> responsePromise);

Similarly Response is constructed from Histories and Vocabs (both of which needn't be there, #53). This can be simplified / neatened by having a ResponseBuilder outside Response, breaking the following messy constructor into meaningful units.

bergamot-translator/src/translator/response.cpp

Lines 11 to 103 in 317433a

	Response::Response(AnnotatedBlob &&source, Histories &&histories,
	std::vector<Ptr<Vocab const>> &vocabs)
	: source(std::move(source)), histories_(std::move(histories)) {
	// Reserving length at least as much as source_ seems like a reasonable thing
	// to do to avoid reallocations.
	target.blob.reserve(source.blob.size());
	// In a first step, the decoded units (individual senteneces) are compiled
	// into a huge string. This is done by computing indices first and appending
	// to the string as each sentences are decoded.
	std::vector<std::pair<size_t, size_t>> translationRanges;
	std::vector<size_t> sentenceBegins;
	size_t offset{0};
	bool first{true};
	for (auto &history : histories_) {
	// TODO(jerin): Change hardcode of nBest = 1
	NBestList onebest = history->nBest(1);
	Result result = onebest[0]; // Expecting only one result;
	Words words = std::get<0>(result);
	auto targetVocab = vocabs.back();
	std::string decoded;
	std::vector<string_view> targetMappings;
	targetVocab->decodeWithByteRanges(words, decoded, targetMappings);
	if (first) {
	first = false;
	} else {
	target.blob += " ";
	++offset;
	}
	sentenceBegins.push_back(translationRanges.size());
	target.blob += decoded;
	auto decodedStringBeginMarker = targetMappings.front().begin();
	for (auto &sview : targetMappings) {
	size_t startIdx = offset + sview.begin() - decodedStringBeginMarker;
	translationRanges.emplace_back(startIdx, startIdx + sview.size());
	}
	offset += decoded.size();
	// Alignments
	// TODO(jerinphilip): The following double conversion might not be
	// necessary. Hard alignment can directly be exported, but this would mean
	// WASM bindings for a structure deep within marian source.
	auto hyp = std::get<1>(result);
	auto softAlignment = hyp->tracebackAlignment();
	auto hardAlignment = data::ConvertSoftAlignToHardAlign(
	softAlignment, /*threshold=*/0.2f); // TODO(jerinphilip): Make this a
	// configurable parameter.
	Alignment unified_alignment;
	for (auto &p : hardAlignment) {
	unified_alignment.emplace_back((Point){p.srcPos, p.tgtPos, p.prob});
	}
	alignments.push_back(std::move(unified_alignment));
	// Quality scores: Sequence level is obtained as normalized path scores.
	// Word level using hypothesis traceback. These are most-likely logprobs.
	auto normalizedPathScore = std::get<2>(result);
	auto wordQualities = hyp->tracebackWordScores();
	wordQualities.pop_back();
	qualityScores.push_back((Quality){normalizedPathScore, wordQualities});
	}
	// Once we have the indices in translation (which might be resized a few
	// times) ready, we can prepare and store the string_view as annotations
	// instead. This is accomplished by iterating over available sentences using
	// sentenceBegin and using addSentence(...) API from Annotation.
	for (size_t i = 1; i <= sentenceBegins.size(); i++) {
	std::vector<string_view> targetMappings;
	size_t begin = sentenceBegins[i - 1];
	size_t safe_end = (i == sentenceBegins.size()) ? translationRanges.size()
	: sentenceBegins[i];
	for (size_t idx = begin; idx < safe_end; idx++) {
	auto &p = translationRanges[idx];
	size_t begin_idx = p.first;
	size_t end_idx = p.second;
	const char *data = &target.blob[begin_idx];
	size_t size = end_idx - begin_idx;
	targetMappings.emplace_back(data, size);
	}
	target.addSentence(targetMappings);
	}

ResponseBuilder will handle taking in histories and be initialized with vocabs and the promise. Using histories and vocab, moving the present constructor of Response into ResponseBuilder can enable the construction of a Response there, following which the std::promise<Response> can be set with the newly constructed instance of Response.

1. Response will thus end up carrying only data members (AnnotatedBlobs of source, target. QualityScores, Alignments).
2. Consequently, Response should become very thin, and in theory ready for WASM export (conditioned on Annotation being exportable). No string_view offending, alignments and stub QualityScores ready.
3. An instance of ResponseBuilder initialized with vocabs and promise and accepting histories additionally can be registered as a callback to Request instead of the existing spread mechanism (to be fired once translation of request is completed), consolidating the transition from processed Request -> Response logic into this callback.
4. QualityEstimation people can be pointed towards just ResponseBuilder, where they will have additional access to histories to just operate and vocabs etc. Feels like a saner API.
5. When amend/cancel capable futures are required, the std::promise can be replaced with std::promise equivalent of the enhanced future.