Sourcery refactored master branch by sourcery-ai[bot] · Pull Request #1 · sycomix/visualbert

sourcery-ai · 2024-01-06T01:28:57Z

Branch master refactored by Sourcery.

If you're happy with these changes, merge this Pull Request using the Squash and merge strategy.

See our documentation here.

Run Sourcery locally

Reduce the feedback loop during development by using the Sourcery editor plugin:

Review changes via command line

To manually merge these changes, make sure you're on the master branch, then run:

git fetch origin sourcery/master
git merge --ff-only FETCH_HEAD
git reset HEAD^

Help us improve this pull request!

sourcery-ai

Due to GitHub API limits, only the first 60 comments can be shown.

sourcery-ai · 2024-01-06T01:28:59Z

 GQA_ROOT = '../'

-path = Path(GQA_ROOT + 'data')
+path = Path(f'{GQA_ROOT}data')


Lines 6-29 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

Replace interpolated string formatting with f-string [×2] (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:28:59Z

 GQA_ROOT = '../'

-path = Path(GQA_ROOT + 'data')
+path = Path(f'{GQA_ROOT}data')


Lines 6-35 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

Replace interpolated string formatting with f-string [×2] (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:29:00Z

-path = Path(GQA_ROOT + 'data')
+path = Path(f'{GQA_ROOT}data')
 split2name = {
    'submit': 'submission_all_questions.json'
    }

 for split, name in split2name.items():
-    with open(path / ("%s" % name)) as f:
+    with open(path / f"{name}") as f:


Lines 6-25 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

Replace interpolated string formatting with f-string [×2] (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:29:00Z


 import os, sys
-sys.path.insert(0, BUTD_ROOT + "/tools")
+sys.path.insert(0, f"{BUTD_ROOT}/tools")


Lines 14-14 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2024-01-06T01:29:00Z

    # First check if file exists, and if it is complete
    # never use set, it loses the order!!! F***
-    wanted_ids = set([image_id[1] for image_id in image_ids])
+    wanted_ids = {image_id[1] for image_id in image_ids}


Function generate_tsv refactored with the following changes:

Replace list(), dict() or set() with comprehension (collection-builtin-to-comprehension)

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai · 2024-01-06T01:29:01Z

    # First check if file exists, and if it is complete
    # never use set, it loses the order!!! F***
-    wanted_ids = set([image_id[1] for image_id in image_ids])
+    wanted_ids = {image_id[1] for image_id in image_ids}


Function generate_tsv refactored with the following changes:

Replace list(), dict() or set() with comprehension (collection-builtin-to-comprehension)

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai · 2024-01-06T01:29:01Z


-    args = parser.parse_args()
-    return args
+    return parser.parse_args()


Function parse_args refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2024-01-06T01:29:01Z

-    args.cfg_file = BUTD_ROOT + "experiments/cfgs/faster_rcnn_end2end_resnet.yml" # s = 500
-    args.prototxt = BUTD_ROOT + "models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt"
-    args.outfile = "%s_obj36.tsv" % args.split
-    
+    args.cfg_file = f"{BUTD_ROOT}experiments/cfgs/faster_rcnn_end2end_resnet.yml"
+    args.prototxt = f"{BUTD_ROOT}models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt"
+    args.outfile = f"{args.split}_obj36.tsv"
+


Lines 183-199 refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

Replace interpolated string formatting with f-string (replace-interpolation-with-fstring)

This removes the following comments ( why? ):

# s = 500

sourcery-ai · 2024-01-06T01:29:01Z


 import os, sys
-sys.path.insert(0, BUTD_ROOT + "/tools")
+sys.path.insert(0, f"{BUTD_ROOT}/tools")


Lines 7-7 refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2024-01-06T01:29:01Z

    # First check if file exists, and if it is complete
    # never use set, it loses the order!!! F***
-    wanted_ids = set([image_id[1] for image_id in image_ids])
+    wanted_ids = {image_id[1] for image_id in image_ids}


Function generate_tsv refactored with the following changes:

Replace list(), dict() or set() with comprehension (collection-builtin-to-comprehension)

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai · 2024-01-06T01:29:04Z


-    args = parser.parse_args()
-    return args
+    return parser.parse_args()


Function parse_args refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2024-01-06T01:29:04Z

-    args.cfg_file = BUTD_ROOT + "experiments/cfgs/faster_rcnn_end2end_resnet.yml" # s = 500
-    args.prototxt = BUTD_ROOT + "models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt"
-    args.outfile = "%s_obj36.tsv" % "vg_gqa"
-    
+    args.cfg_file = f"{BUTD_ROOT}experiments/cfgs/faster_rcnn_end2end_resnet.yml"
+    args.prototxt = f"{BUTD_ROOT}models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt"
+    args.outfile = 'vg_gqa_obj36.tsv'
+


Lines 159-175 refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

Replace interpolated string formatting with f-string (replace-interpolation-with-fstring)

Simplify unnecessary nesting, casting and constant values in f-strings (simplify-fstring-formatting)

Replace f-string with no interpolated values with string (remove-redundant-fstring)

This removes the following comments ( why? ):

# s = 500

sourcery-ai · 2024-01-06T01:29:04Z

        optimizer = 'bert'      # The bert optimizer will be bind later.
    else:
-        assert False, "Please add your optimizer %s in the list." % optim
+        assert False, f"Please add your optimizer {optim} in the list."


Function get_optimizer refactored with the following changes:

Replace interpolated string formatting with f-string (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:29:04Z

    parser.add_argument("--algorithm", dest="algorithm", default=None, type=str)
    parser.add_argument("--save_path", dest="save_path", default=None, type=str)
-    
+


Function parse_args refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

Replace call to format with f-string [×2] (use-fstring-for-formatting)

This removes the following comments ( why? ):

# File where you need to keep the logs

sourcery-ai · 2024-01-06T01:29:04Z

-    print("Start to load Faster-RCNN detected objects from %s" % fname)
+    print(f"Start to load Faster-RCNN detected objects from {fname}")
    with open(fname) as f:
        reader = csv.DictReader(f, FIELDNAMES, delimiter="\t")
        for i, item in enumerate(tqdm(reader)):

            for key in ['img_h', 'img_w', 'num_boxes']:
                item[key] = int(item[key])
-            
+


Function load_obj_tsv refactored with the following changes:

Replace interpolated string formatting with f-string (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:29:05Z

-        print("Load LXMERT pre-trained model from %s" % path)
-        state_dict = torch.load("%s_LXRT.pth" % path)
+        print(f"Load LXMERT pre-trained model from {path}")
+        state_dict = torch.load(f"{path}_LXRT.pth")


Function LXRTEncoder.load refactored with the following changes:

Replace interpolated string formatting with f-string [×2] (replace-interpolation-with-fstring)

sourcery-ai · 2024-01-06T01:29:05Z

        etag_bytes = etag.encode('utf-8')
        etag_hash = sha256(etag_bytes)
-        filename += '.' + etag_hash.hexdigest()
+        filename += f'.{etag_hash.hexdigest()}'


Function url_to_filename refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2024-01-06T01:29:05Z

-        raise EnvironmentError("file {} not found".format(cache_path))
+        raise EnvironmentError(f"file {cache_path} not found")

-    meta_path = cache_path + '.json'
+    meta_path = f'{cache_path}.json'
    if not os.path.exists(meta_path):
-        raise EnvironmentError("file {} not found".format(meta_path))
+        raise EnvironmentError(f"file {meta_path} not found")


Function filename_to_url refactored with the following changes:

Replace call to format with f-string [×2] (use-fstring-for-formatting)

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2024-01-06T01:29:05Z

-        raise EnvironmentError("file {} not found".format(url_or_filename))
+        raise EnvironmentError(f"file {url_or_filename} not found")
    else:
        # Something unknown
-        raise ValueError("unable to parse {} as a URL or as a local path".format(url_or_filename))
+        raise ValueError(
+            f"unable to parse {url_or_filename} as a URL or as a local path"
+        )


Function cached_path refactored with the following changes:

Replace call to format with f-string [×2] (use-fstring-for-formatting)

sourcery-ai · 2024-01-06T01:29:06Z

-        raise ValueError("bad s3 path {}".format(url))
+        raise ValueError(f"bad s3 path {url}")


Function split_s3_path refactored with the following changes:

Replace call to format with f-string (use-fstring-for-formatting)

sourcery-ai · 2024-01-06T01:29:08Z

-            raise IOError("HEAD request failed for url {} with status code {}"
-                          .format(url, response.status_code))
+            raise IOError(
+                f"HEAD request failed for url {url} with status code {response.status_code}"
+            )


Function get_from_cache refactored with the following changes:

Replace call to format with f-string (use-fstring-for-formatting)

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai · 2024-01-06T01:29:08Z

-            if i < 1:
-                _flag = True
-
+        _flag = any(i < 1 for i in self.upsample_ratios)


Function CustomBatchSampler.__init__ refactored with the following changes:

Use any() instead of for loop (use-any)

sourcery-ai · 2024-01-06T01:29:08Z

-                
+
                self.rotate_index[index] = self.rotate_index[index] + 1 #% sample_num
                if self.rotate_index[index] == sample_num:
                    self.all_indexes[index] = torch.randperm(i).tolist()
                    self.rotate_index[index] = 0 # Reset rotate index 

                random.shuffle(random_indexes)
-                random_indexes = [j + current_index for j in random_indexes]
-                random_indexes = chunks(random_indexes, self.batch_size)
-                #self.all_batched_indexes.extend(random_indexes)
+                        #self.all_batched_indexes.extend(random_indexes)
            else:
                random_indexes = torch.randperm(i).tolist()
-                random_indexes = [j + current_index for j in random_indexes]
-                random_indexes = chunks(random_indexes, self.batch_size)
-                #self.all_batched_indexes.extend(random_indexes)
-
+                        #self.all_batched_indexes.extend(random_indexes)
+
+            random_indexes = [j + current_index for j in random_indexes]
+            random_indexes = chunks(random_indexes, self.batch_size)
            random.shuffle(random_indexes)
            self.all_batched_indexes.append(random_indexes)

            if self.upsample_ratios[index] > 1:
-                for k in range(self.upsample_ratios[index] - 1):
+                for _ in range(self.upsample_ratios[index] - 1):


Function CustomBatchSampler.prepare_indexes refactored with the following changes:

Hoist repeated code outside conditional statement [×2] (hoist-statement-from-if)

Replace unused for index with underscore (for-index-underscore)

Replace a for append loop with list extend (for-append-to-extend)

sourcery-ai · 2024-01-06T01:29:08Z

-        return sum([len(i) for i in self.datasets])
+        return sum(len(i) for i in self.datasets)


Function ConcateDataset.__len__ refactored with the following changes:

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai · 2024-01-06T01:29:08Z

-        if self.h5_num_boxes is not None:
-            obj_num = self.h5_num_boxes[image_index]
-        else:
-            obj_num = 36
+        obj_num = 36 if self.h5_num_boxes is None else self.h5_num_boxes[image_index]
        feats = self.h5_features[image_index]
        boxes = self.h5_boxes[image_index]
        img_h = self.h5_wh[image_index][1]
        img_w = self.h5_wh[image_index][0]
-        
+


Function ImageFeatureDataset.__getitem__ refactored with the following changes:

Swap if/else branches (swap-if-else-branches)

Replace if statement with if expression (assign-if-exp)

sourcery-ai · 2024-01-06T01:29:09Z

-        if isinstance(visn_input, tuple) or isinstance(visn_input, list):
+        if isinstance(visn_input, (tuple, list)):


Function VisualFeatEncoder.forward refactored with the following changes:

Merge isinstance calls (merge-isinstance)

sourcery-ai · 2024-01-06T01:29:09Z

-    if feat_2 is None:
-        return feat_1
-    return torch.cat((feat_1, feat_2), dim=dim)
+    return feat_1 if feat_2 is None else torch.cat((feat_1, feat_2), dim=dim)


Function _cat_with_none refactored with the following changes:

Lift code into else after jump in control flow (reintroduce-else)

Replace if statement with if expression (assign-if-exp)

sourcery-ai · 2024-01-06T01:29:09Z

-            
-                print("\n\n!! Has {} layers".format(len(self.layer) + 1))
+
+                print(f"\n\n!! Has {len(self.layer) + 1} layers")
            else:
-                print("\n\n!! Has {} layers".format(len(self.layer)))
+                print(f"\n\n!! Has {len(self.layer)} layers")


Function LXRTEncoder.__init__ refactored with the following changes:

Replace call to format with f-string [×3] (use-fstring-for-formatting)

sourcery-ai · 2024-01-06T01:29:09Z

-        output = {}
-        for key in self.visual_losses:
-            output[key] = self.decoder_dict[key](hidden_states)
-        return output
+        return {
+            key: self.decoder_dict[key](hidden_states)
+            for key in self.visual_losses
+        }


Function BertVisualObjHead.forward refactored with the following changes:

Convert for loop into dictionary comprehension (dict-comprehension)

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai · 2024-01-06T01:29:09Z

-                "Parameter config in `{}(config)` should be an instance of class `BertConfig`. "
-                "To create a model from a Google pretrained model use "
-                "`model = {}.from_pretrained(PRETRAINED_MODEL_NAME)`".format(
-                    self.__class__.__name__, self.__class__.__name__
-                ))
+                f"Parameter config in `{self.__class__.__name__}(config)` should be an instance of class `BertConfig`. To create a model from a Google pretrained model use `model = {self.__class__.__name__}.from_pretrained(PRETRAINED_MODEL_NAME)`"
+            )


Function BertPreTrainedModel.__init__ refactored with the following changes:

Replace call to format with f-string (use-fstring-for-formatting)

'Refactored by Sourcery'

18d66c2

sourcery-ai Bot requested a review from sycomix January 6, 2024 01:28

sourcery-ai Bot commented Jan 6, 2024

View reviewed changes

		parser.add_argument("--algorithm", dest="algorithm", default=None, type=str)
		parser.add_argument("--save_path", dest="save_path", default=None, type=str)

		raise ValueError("bad s3 path {}".format(url))
		raise ValueError(f"bad s3 path {url}")

		return sum([len(i) for i in self.datasets])
		return sum(len(i) for i in self.datasets)

		if isinstance(visn_input, tuple) or isinstance(visn_input, list):
		if isinstance(visn_input, (tuple, list)):

Conversation

sourcery-ai Bot commented Jan 6, 2024

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Jan 6, 2024

Choose a reason for hiding this comment