Sourcery refactored master branch#1
Conversation
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
| for line in reader: | ||
| if sys.version_info[0] == 2: | ||
| line = list(unicode(cell, 'utf-8') for cell in line) | ||
| line = [unicode(cell, 'utf-8') for cell in line] |
There was a problem hiding this comment.
Function DataProcessor._read_tsv refactored with the following changes:
- Replace list(), dict() or set() with comprehension (
collection-builtin-to-comprehension)
| """See base class.""" | ||
| logger.info("LOOKING AT {}".format( | ||
| os.path.join(data_dir, "train.tsv"))) | ||
| logger.info(f'LOOKING AT {os.path.join(data_dir, "train.tsv")}') |
There was a problem hiding this comment.
Function MrpcProcessor.get_train_examples refactored with the following changes:
- Replace call to format with f-string (
use-fstring-for-formatting)
| guid = "%s-%s" % (set_type, i) | ||
| guid = f"{set_type}-{i}" |
There was a problem hiding this comment.
Function MrpcProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function MnliProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, i) | ||
| guid = f"{set_type}-{i}" |
There was a problem hiding this comment.
Function ColaProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function MnliProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, i) | ||
| guid = f"{set_type}-{i}" |
There was a problem hiding this comment.
Function ColaProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, i) | ||
| guid = f"{set_type}-{i}" |
There was a problem hiding this comment.
Function Sst2Processor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function StsbProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function QqpProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function QnliProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function RteProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| guid = "%s-%s" % (set_type, line[0]) | ||
| guid = f"{set_type}-{line[0]}" |
There was a problem hiding this comment.
Function WnliProcessor._create_examples refactored with the following changes:
- Replace interpolated string formatting with f-string (
replace-interpolation-with-fstring)
| else: | ||
| # Account for [CLS] and [SEP] with "- 2" | ||
| if len(tokens_a) > max_seq_length - 2: | ||
| tokens_a = tokens_a[:(max_seq_length - 2)] | ||
| elif len(tokens_a) > max_seq_length - 2: | ||
| tokens_a = tokens_a[:(max_seq_length - 2)] |
There was a problem hiding this comment.
Function convert_examples_to_features refactored with the following changes:
- Merge else clause's nested if statement into elif (
merge-else-if-into-elif) - Replace interpolated string formatting with f-string [×4] (
replace-interpolation-with-fstring)
This removes the following comments ( why? ):
# Account for [CLS] and [SEP] with "- 2"
| print("Converting TensorFlow checkpoint from {}".format(tf_path)) | ||
| print(f"Converting TensorFlow checkpoint from {tf_path}") | ||
| # Load weights from TF model | ||
| init_vars = tf.train.list_variables(tf_path) | ||
| names = [] | ||
| arrays = [] | ||
| for name, shape in init_vars: | ||
| print("Loading TF weight {} with shape {}".format(name, shape)) | ||
| print(f"Loading TF weight {name} with shape {shape}") |
There was a problem hiding this comment.
Function load_tf_weights_in_bert refactored with the following changes:
- Replace call to format with f-string [×4] (
use-fstring-for-formatting) - Merge duplicate blocks in conditional (
merge-duplicate-blocks) - Replace multiple comparisons of same variable with
inoperator [×2] (merge-comparisons) - Remove redundant conditional [×2] (
remove-redundant-if)
| if masked_lm_labels is not None and next_sentence_label is not None: | ||
| loss_fct = CrossEntropyLoss(ignore_index=-1) | ||
| masked_lm_loss = loss_fct( | ||
| prediction_scores.view(-1, self.config.vocab_size), | ||
| masked_lm_labels.view(-1)) | ||
| next_sentence_loss = loss_fct(seq_relationship_score.view(-1, 2), | ||
| next_sentence_label.view(-1)) | ||
| #print("loss is {} {}".format(masked_lm_loss, next_sentence_loss)) | ||
| total_loss = masked_lm_loss + next_sentence_loss | ||
| # if log: | ||
| # self.log_summary_writer(logs={'train_loss': total_loss.item()}) | ||
| return total_loss | ||
| else: | ||
| if masked_lm_labels is None or next_sentence_label is None: | ||
| return prediction_scores, seq_relationship_score | ||
| loss_fct = CrossEntropyLoss(ignore_index=-1) | ||
| masked_lm_loss = loss_fct( | ||
| prediction_scores.view(-1, self.config.vocab_size), | ||
| masked_lm_labels.view(-1)) | ||
| next_sentence_loss = loss_fct(seq_relationship_score.view(-1, 2), | ||
| next_sentence_label.view(-1)) | ||
| return masked_lm_loss + next_sentence_loss |
There was a problem hiding this comment.
Function BertForPreTraining.forward refactored with the following changes:
- Swap if/else branches (
swap-if-else-branches) - Remove unnecessary else after guard condition (
remove-unnecessary-else) - Inline variable that is immediately returned (
inline-immediately-returned-variable)
This removes the following comments ( why? ):
# self.log_summary_writer(logs={'train_loss': total_loss.item()})
# if log:
#print("loss is {} {}".format(masked_lm_loss, next_sentence_loss))
| masked_lm_loss = loss_fct( | ||
| return loss_fct( | ||
| prediction_scores.view(-1, self.config.vocab_size), | ||
| masked_lm_labels.view(-1)) | ||
| return masked_lm_loss | ||
| masked_lm_labels.view(-1), | ||
| ) |
There was a problem hiding this comment.
Function BertForMaskedLM.forward refactored with the following changes:
- Inline variable that is immediately returned (
inline-immediately-returned-variable)
| if next_sentence_label is not None: | ||
| loss_fct = CrossEntropyLoss(ignore_index=-1) | ||
| next_sentence_loss = loss_fct(seq_relationship_score.view(-1, 2), | ||
| next_sentence_label.view(-1)) | ||
| return next_sentence_loss | ||
| else: | ||
| if next_sentence_label is None: | ||
| return seq_relationship_score | ||
| loss_fct = CrossEntropyLoss(ignore_index=-1) | ||
| return loss_fct( | ||
| seq_relationship_score.view(-1, 2), next_sentence_label.view(-1) | ||
| ) |
There was a problem hiding this comment.
Function BertForNextSentencePrediction.forward refactored with the following changes:
- Swap if/else branches (
swap-if-else-branches) - Remove unnecessary else after guard condition (
remove-unnecessary-else) - Inline variable that is immediately returned (
inline-immediately-returned-variable)
| loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1)) | ||
| return loss | ||
| return loss_fct(logits.view(-1, self.num_labels), labels.view(-1)) |
There was a problem hiding this comment.
Function BertForSequenceClassification.forward refactored with the following changes:
- Inline variable that is immediately returned (
inline-immediately-returned-variable)
| if labels is not None: | ||
| loss_fct = CrossEntropyLoss() | ||
| loss = loss_fct(reshaped_logits, labels) | ||
| return loss | ||
| else: | ||
| if labels is None: | ||
| return reshaped_logits | ||
| loss_fct = CrossEntropyLoss() | ||
| return loss_fct(reshaped_logits, labels) |
There was a problem hiding this comment.
Function BertForMultipleChoice.forward refactored with the following changes:
- Swap if/else branches (
swap-if-else-branches) - Remove unnecessary else after guard condition (
remove-unnecessary-else) - Inline variable that is immediately returned (
inline-immediately-returned-variable)
Branch
masterrefactored by Sourcery.If you're happy with these changes, merge this Pull Request using the Squash and merge strategy.
See our documentation here.
Run Sourcery locally
Reduce the feedback loop during development by using the Sourcery editor plugin:
Review changes via command line
To manually merge these changes, make sure you're on the
masterbranch, then run:Help us improve this pull request!