[BIT-484] Add TextCausalLMNext #848
Conversation
Specifies messaging of topk server token phrases with probabilities. Server last position token predictions are retokenized to token phrases with the bittensor tokenizer. Allows for zero translation loss CausalLM next generation between different tokenizers. Also adds comment specifying proto compile command, which is useful to see for manual compilation instruction.
Specifies messaging of topk server token phrases with probabilities. Server last position token predictions are retokenized to token phrases with the bittensor tokenizer. Allows for zero translation loss CausalLM next generation between different tokenizers.
Specifies messaging of topk server token phrases with probabilities. Server last position token predictions are retokenized to token phrases with the bittensor tokenizer. Allows for zero translation loss CausalLM next generation between different tokenizers.
Specifies messaging of topk server token phrases with probabilities. Server last position token predictions are retokenized to token phrases with the bittensor tokenizer. Allows for zero translation loss CausalLM next generation between different tokenizers.
Tokenizer utilities adds functions to compact and unravel topk server token phrases (standard tokenized), to be used with TextCausalLMNext synapse.
Unit test new tokenizer utility functions that compact and unravel topk server token phrases (standard tokenized), to be used with TextCausalLMNext synapse.
Calculates the cross entropy of a phrase prediction against a target phrase, so that this is a multi-token extension of typical cross entropy calculated for next token prediction, to be used with TextCausalLMNext synapse.
Adds unit test for calculating the cross entropy of a phrase prediction against a target phrase, so that this is a multi-token extension of typical cross entropy calculated for next token prediction, to be used with TextCausalLMNext synapse.
Pull Request Test Coverage Report for Build 9de67db5-bc8e-4bd7-a246-c05490ce7857
💛 - Coveralls |
| @@ -436,7 +436,80 @@ def text_causal_lm ( | |||
| self.update_stats( formatted_endpoints, synapses, formatted_inputs, outputs, codes, times ) | |||
| return outputs[0], codes[0], times[0] | |||
|
|
|||
| def text_last_hidden_state( | |||
| def text_causal_lm_next( | |||
There was a problem hiding this comment.
Should we expect this synapse to be called on its own or always in conjunction with CausalLM?
There was a problem hiding this comment.
TextCausalLMNext can be called independently of TextCausalLM, but for initial tests they'll both run so comparisons can be performed.
| """ Factory function which returns a TextCausalLMNext synapse adapter given arguments. | ||
| Args: | ||
| topk (:obj:`int`): | ||
| Specifies the number of topk server token phrases to return. |
There was a problem hiding this comment.
A little bit confused about the topk variable. Are they the topk logits of the server's tokenizer?
There was a problem hiding this comment.
Yes, topk logits/phrases, under the assumption that each logit represents a text phrase (the server represents this with a single token with associated probability, but it could be multiple std tokens (a token phrase) at the validator).
| return forward_request_tensor | ||
|
|
||
| def decode_forward_request_tensor(self, forward_request_tensor: torch.Tensor) -> torch.Tensor: | ||
| return forward_request_tensor |
There was a problem hiding this comment.
Do we need a encoding or decoding function?
There was a problem hiding this comment.
Currently, encoding and decoding is best performed at the origin, that is server and validator forward() functions respectively. The reason being that variable length phrases need to otherwise be converted to padded tensors, simply for the purpose of passing it to the synapse encode_forward_request_tensor(), where it would need to be decoded and then re-encoded and compacted, thus duplicating work best done immediately at the origin.
|
|
||
| def check_forward_response_tensor(self, forward_request_tensor, forward_response_tensor): | ||
| # forward_request_tensor: [batch_size, sequence_len] | ||
| # forward_response_tensor: [ >= batch_size * (2 * topk + 1)] |
There was a problem hiding this comment.
The response tensor is a 1-D tensor?
There was a problem hiding this comment.
Yes, the response is 1-D to efficiently compact variable length token phrases by omitting padding that would otherwise be required to make it n-D.
Adds the TextCausalLMNext Synapse, as well as the tokenizer_utils needed to encode/decode the synapse. Both core_server and core_validator are running successfully on Nobunaga with this branch.
TextCausalLMNext sends a user-defined topk of server token phrases for input context continuation, which validator/client uses for validated generation where server phrase probabilities are evaluated against a groundtruth continuation. Supports backward gradients from validator/client to server.
Check detailed commit explanations.