Sourcery refactored main branch by sourcery-ai[bot] · Pull Request #1 · Deadstarjay62/the-algorithm

sourcery-ai · 2023-05-30T00:50:18Z

Branch main refactored by Sourcery.

If you're happy with these changes, merge this Pull Request using the Squash and merge strategy.

See our documentation here.

Run Sourcery locally

Reduce the feedback loop during development by using the Sourcery editor plugin:

Review changes via command line

To manually merge these changes, make sure you're on the main branch, then run:

git fetch origin sourcery/main
git merge --ff-only FETCH_HEAD
git reset HEAD^

Help us improve this pull request!

sourcery-ai

Due to GitHub API limits, only the first 60 comments can be shown.

sourcery-ai · 2023-05-30T00:50:20Z

    logging.info(f"Uploading to GCS with path {self.gcs_output_path}")
    assert os.path.isdir(local_path)
-    for local_file in glob.glob(local_path + "/*"):
+    for local_file in glob.glob(f"{local_path}/*"):


Function MergeAndBuildIndex.extract_output refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:21Z

-  learn = trainer.learn
-  if args.distributed or args.num_workers is not None:
-    learn = trainer.train_and_evaluate
-
  if not args.directly_export_best:
    logging.info("Starting training")
    start = datetime.now()
+    learn = (trainer.train_and_evaluate if args.distributed
+             or args.num_workers is not None else trainer.learn)


Function main refactored with the following changes:

Move setting of default value for variable into else branch (introduce-default-else)

Replace if statement with if expression (assign-if-exp)

Move assignments closer to their usage (move-assign)

sourcery-ai · 2023-05-30T00:50:21Z

-    return parser.parse_args()
-  else:
-    return parser.parse_args(args)
+  return parser.parse_args() if args is None else parser.parse_args(args)


Function get_params refactored with the following changes:

Replace if statement with if expression (assign-if-exp)

sourcery-ai · 2023-05-30T00:50:21Z

-    if opt.output_checkpoint_dir == "none" or opt.output_checkpoint_dir == opt.warm_start_base_dir:
-      _warm_start_base_dir = os.path.normpath(opt.warm_start_base_dir) + "_backup_warm_start"
+    if opt.output_checkpoint_dir in ["none", opt.warm_start_base_dir]:
+      _warm_start_base_dir = (
+          f"{os.path.normpath(opt.warm_start_base_dir)}_backup_warm_start")


Function _main refactored with the following changes:

Split conditional into multiple branches (split-or-ifs)

Merge duplicate blocks in conditional (merge-duplicate-blocks)

Replace multiple comparisons of same variable with in operator [×2] (merge-comparisons)

Use f-string instead of string concatenation (use-fstring-for-concatenation)

Remove redundant conditional (remove-redundant-if)

sourcery-ai · 2023-05-30T00:50:22Z

-    output = tf.transpose(transposed_residual, perm=[0, 2, 1])
-
-    return output
+    return tf.transpose(transposed_residual, perm=[0, 2, 1])


Function ChannelWiseDense.call refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:24Z

-    new_user_metric_ops = {name + "_new_users": ops for name, ops in new_user_metric_ops.items()}
+    new_user_metric_ops = {
+        f"{name}_new_users": ops
+        for name, ops in new_user_metric_ops.items()
+    }


Function add_new_user_metrics refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:24Z

-    classnames_unweighted = ["unweighted_" + classname for classname in classnames]
+    classnames_unweighted = [f"unweighted_{classname}" for classname in classnames]


Function get_meta_learn_single_binary_task_metric_fn refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:24Z

-    classnames_unweighted = ["unweighted_" + classname for classname in classnames]
+    classnames_unweighted = [f"unweighted_{classname}" for classname in classnames]


Function get_meta_learn_dual_binary_tasks_metric_fn refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:24Z

-    return parser.parse_args()
-  else:
-    return parser.parse_args(args)
+  return parser.parse_args() if args is None else parser.parse_args(args)


Function get_params refactored with the following changes:

Replace if statement with if expression (assign-if-exp)

sourcery-ai · 2023-05-30T00:50:25Z

  sparse_shape = tf.stack([features.dense_shape[0], sparse_feature_dim])
-  sparse_tf = tf.SparseTensor(features.indices, features.values, sparse_shape)
-  return sparse_tf
+  return tf.SparseTensor(features.indices, features.values, sparse_shape)


Function _sparse_feature_fixup refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:28Z

-    if not base:
-      return base
-    return f"{base}:{suffix}"
+    return base if not base else f"{base}:{suffix}"


Function self_atten_dense refactored with the following changes:

Lift code into else after jump in control flow (reintroduce-else)

Replace if statement with if expression (assign-if-exp)

sourcery-ai · 2023-05-30T00:50:29Z

-  bn_gw_normalized_dense = tf.layers.batch_normalization(
-    gw_normalized_dense,
-    training=is_training,
-    renorm_momentum=0.9999,
-    momentum=0.9999,
-    renorm=is_training,
-    trainable=True,
+  return tf.layers.batch_normalization(
+      gw_normalized_dense,
+      training=is_training,
+      renorm_momentum=0.9999,
+      momentum=0.9999,
+      renorm=is_training,
+      trainable=True,
  )

-  return bn_gw_normalized_dense


Function get_input_trans_func refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:29Z

-    if is_training:
-      with tf.variable_scope("sparse_dropout"):
-        values = input_tensor.values
-        keep_mask = tf.keras.backend.random_binomial(
-          tf.shape(values), p=1 - rate, dtype=tf.float32, seed=None
-        )
-        keep_mask.set_shape([None])
-        keep_mask = tf.cast(keep_mask, tf.bool)
-
-        keep_indices = tf.boolean_mask(input_tensor.indices, keep_mask, axis=0)
-        keep_values = tf.boolean_mask(values, keep_mask, axis=0)
-
-        dropped_tensor = tf.SparseTensor(keep_indices, keep_values, input_tensor.dense_shape)
-        return dropped_tensor
-    else:
+    if not is_training:
      return input_tensor
+    with tf.variable_scope("sparse_dropout"):
+      values = input_tensor.values
+      keep_mask = tf.keras.backend.random_binomial(
+        tf.shape(values), p=1 - rate, dtype=tf.float32, seed=None
+      )
+      keep_mask.set_shape([None])
+      keep_mask = tf.cast(keep_mask, tf.bool)
+
+      keep_indices = tf.boolean_mask(input_tensor.indices, keep_mask, axis=0)
+      keep_values = tf.boolean_mask(values, keep_mask, axis=0)
+
+      return tf.SparseTensor(keep_indices, keep_values, input_tensor.dense_shape)


Function tensor_dropout refactored with the following changes:

Swap if/else branches (swap-if-else-branches)

Remove unnecessary else after guard condition (remove-unnecessary-else)

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:29Z

-  bn_gw_normalized_dense = tf.layers.batch_normalization(
-    gw_normalized_dense,
-    training=is_training,
-    renorm_momentum=0.9999,
-    momentum=0.9999,
-    renorm=is_training,
-    trainable=True,
+  return tf.layers.batch_normalization(
+      gw_normalized_dense,
+      training=is_training,
+      renorm_momentum=0.9999,
+      momentum=0.9999,
+      renorm=is_training,
+      trainable=True,
  )

-  return bn_gw_normalized_dense


Function adaptive_transformation refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:30Z

-      name + "_group_weight",
-      [1, group_num, input_dim, out_dim],
-      initializer=customized_glorot_uniform(
-        fan_in=input_dim * init_multiplier, fan_out=out_dim * init_multiplier
-      ),
-      trainable=True,
+        f"{name}_group_weight",
+        [1, group_num, input_dim, out_dim],
+        initializer=customized_glorot_uniform(fan_in=input_dim * init_multiplier,
+                                              fan_out=out_dim * init_multiplier),
+        trainable=True,
    )
    self.b = tf.get_variable(
-      name + "_group_bias",
-      [1, group_num, out_dim],
-      initializer=tf.constant_initializer(0.0),
-      trainable=True,
+        f"{name}_group_bias",
+        [1, group_num, out_dim],
+        initializer=tf.constant_initializer(0.0),
+        trainable=True,


Function FastGroupWiseTrans.__init__ refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:31Z


  end = datetime.now()
-  logging.info("Evaluating time: " + str(end - start))
+  logging.info(f"Evaluating time: {str(end - start)}")


Lines 89-89 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:31Z


-  output_dict = {"output": logits}
-  return output_dict
+  return {"output": logits}


Function deepnorm_light_ranking refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:32Z

-PREDICTED_CLASSES = \
-  ["tf_target"] + ["tf_" + label_name for label_name in LABEL_NAMES] + ["tf_timelines.earlybird_score"] + \
-  ["lolly_target"] + ["lolly_" + label_name for label_name in LABEL_NAMES] + ["lolly_timelines.earlybird_score"]
+PREDICTED_CLASSES = (
+    (
+        ["tf_target"]
+        + [f"tf_{label_name}" for label_name in LABEL_NAMES]
+        + ["tf_timelines.earlybird_score"]
+        + ["lolly_target"]
+    )
+    + [f"lolly_{label_name}" for label_name in LABEL_NAMES]
+) + ["lolly_timelines.earlybird_score"]


Lines 19-21 refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:32Z

-        class_metric_name = metric_name + "_" + (classes[i] if classes is not None else str(i))
+        class_metric_name = f"{metric_name}_" + (classes[i] if classes
+                                                 is not None else str(i))


Function get_multi_binary_class_metric_fn refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:32Z

-  export_outputs = {
-    tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
-      tf.estimator.export.PredictOutput(
-        {"prediction": tf.identity(graph_output["output"], name="output_scores")}
-      )
+  return {
+      tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
+      tf.estimator.export.PredictOutput({
+          "prediction":
+          tf.identity(graph_output["output"], name="output_scores")
+      })
  }
-  return export_outputs


Function earlybird_output_fn refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:36Z

-  logging.info("Training and Evaluation time: " + str(trainingEndTime - trainingStartTime))
+  logging.info(
+      f"Training and Evaluation time: {str(trainingEndTime - trainingStartTime)}"
+  )


Lines 186-212 refactored with the following changes:

Swap positions of nested conditionals [×2] (swap-nested-ifs)

Hoist nested repeated code outside conditional statements [×2] (hoist-similar-statement-from-if)

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:36Z

-  lolly_activations = tf.math.subtract(tf.math.log(eb_lolly_scores), tf.math.log(inverse_eb_lolly_scores))
-  return lolly_activations
+  return tf.math.subtract(tf.math.log(eb_lolly_scores),
+                          tf.math.log(inverse_eb_lolly_scores))


Function get_lolly_logits refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:36Z

  logged_eb_lolly_scores = tf.reshape(labels[:, EB_SCORE_IDX], (-1, 1))
-  eb_lolly_scores = tf.truediv(logged_eb_lolly_scores, 100.0)
-  return eb_lolly_scores
+  return tf.truediv(logged_eb_lolly_scores, 100.0)


Function get_lolly_scores refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:36Z

  def parse(self, line):
-    match = re.search(self.pattern(), line)
-    if match:
+    if match := re.search(self.pattern(), line):


Function Parser.parse refactored with the following changes:

Use named expression to simplify assignment and conditional (use-named-expression)

sourcery-ai · 2023-05-30T00:50:36Z

-        print("Missing feature with id: " + str(feature_id))
+        print(f"Missing feature with id: {str(feature_id)}")


Function DBv2DataExampleParser._parse_match refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:38Z

  print('Smart bias init to ', smart_bias_value)
-  output_bias = tf.keras.initializers.Constant(smart_bias_value)
-  return output_bias
+  return tf.keras.initializers.Constant(smart_bias_value)


Function _get_bias refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2023-05-30T00:50:38Z

-    last_layer = tf.keras.layers.Dense(
-      kwargs["num_classes"], activation="softmax", kernel_initializer=glorot,
-      bias_initializer=output_bias, name=layer_name
+    return tf.keras.layers.Dense(
+        kwargs["num_classes"],
+        activation="softmax",
+        kernel_initializer=glorot,
+        bias_initializer=output_bias,
+        name=layer_name,
    )

  elif kwargs.get('num_raters', 1) > 1:
    if kwargs.get('multitask', False):
      raise NotImplementedError
-    last_layer = tf.keras.layers.Dense(
-      kwargs['num_raters'], activation="sigmoid", kernel_initializer=glorot,
-      bias_initializer=output_bias, name='probs')
+    return tf.keras.layers.Dense(
+        kwargs['num_raters'],
+        activation="sigmoid",
+        kernel_initializer=glorot,
+        bias_initializer=output_bias,
+        name='probs',
+    )

  else:
-    last_layer = tf.keras.layers.Dense(
-      1, activation="sigmoid", kernel_initializer=glorot,
-      bias_initializer=output_bias, name=layer_name
+    return tf.keras.layers.Dense(
+        1,
+        activation="sigmoid",
+        kernel_initializer=glorot,
+        bias_initializer=output_bias,
+        name=layer_name,
    )

-  return last_layer


Function get_last_layer refactored with the following changes:

Lift return into if (lift-return-into-if)

sourcery-ai · 2023-05-30T00:50:39Z

-      if self.test
-      else f"..."
-    )
+    self.logdir = "..."


Function Trainer._init_dirnames refactored with the following changes:

Remove an unnecessary condition used during variable assignment (remove-redundant-condition)

Replace f-string with no interpolated values with string (remove-redundant-fstring)

sourcery-ai · 2023-05-30T00:50:39Z

-    fold_logdir = self.logdir + f"_fold{fold}"
-    fold_checkpoint_path = self.checkpoint_path + f"_fold{fold}/{{epoch:02d}}"
+    fold_logdir = f"{self.logdir}_fold{fold}"
+    fold_checkpoint_path = f"{self.checkpoint_path}_fold{fold}/{{epoch:02d}}"


Function Trainer.get_callbacks refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

sourcery-ai · 2023-05-30T00:50:39Z

-      warm_up_schedule = WarmUp(
-        initial_learning_rate=self.learning_rate,
-        decay_schedule_fn=learning_rate_fn,
-        warmup_steps=warm_up_steps,
+      return WarmUp(
+          initial_learning_rate=self.learning_rate,
+          decay_schedule_fn=learning_rate_fn,
+          warmup_steps=warm_up_steps,
      )
-      return warm_up_schedule


Function Trainer.get_lr_schedule refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

'Refactored by Sourcery'

cd64fa3

sourcery-ai Bot requested a review from Deadstarjay62 May 30, 2023 00:50

sourcery-ai Bot commented May 30, 2023

View reviewed changes

		classnames_unweighted = ["unweighted_" + classname for classname in classnames]
		classnames_unweighted = [f"unweighted_{classname}" for classname in classnames]

		print("Missing feature with id: " + str(feature_id))
		print(f"Missing feature with id: {str(feature_id)}")

Conversation

sourcery-ai Bot commented May 30, 2023

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot May 30, 2023

Choose a reason for hiding this comment