Object Version Replication - add the ability to replicate object versions.

[liranmauda]

Explain the changes

Config.js:

• Create config.BUCKET_DIFF_FOR_REPLICATION to determine if BucketDiff is in replication mode.

Replication api:

• copy now get an object instead of keys and return an object instead of an array

Replication Server:

• copy_objects_mixed_types:
  • Will get objects instead of keys
  • Will return an object of num_of_objects and size_of_objects
  • if there is only one metadata per key, copy it as none version (even if there is a version)
  • If there is more than one metadata, copy all from the older to the newest, considering the versionID

Replication Utils:

• copy_objects will get objects instead of keys
• Cleaned unused get_object_md and check_data_or_md_changed functions

Replication Scanner:

• Use BucketDiff for replication instead of the functions in the scanner, and remove the unused functions
• Support Object versions replication
• Fix the metrics to consume the new replay format.

Gap

1. will not copy the last version if the latest is a delete marker, in order to do so we will need to be were if there is versioning enabled (maybe add to the replication_api sync_version flag)
2. need to add tests for replication of object versions
3. This will break:
   

   noobaa-core/src/server/bg_services/log_replication_scanner.js

   Lines 209 to 220 in 0db21bb

   	async copy_objects(src_bucket_name, dst_bucket_name, keys) {
   	const copy_type = replication_utils.get_copy_type();
   	await replication_utils.copy_objects(
   	this._scanner_sem,
   	this.client,
   	copy_type,
   	src_bucket_name.unwrap(),
   	dst_bucket_name.unwrap(),
   	keys,
   	);
   	dbg.log2('log_replication_scanner: scan copy_objects: ', keys);
   	}

Testing Instructions:

1. Run npx jest --testPathPattern="test_bucket_diff.test.js"
2. Run unit tests test_bucket_replication.js

• Tests added

[jackyalbo]

in general It looks ok to me, under the assumptions and the gaps mentioned in the PR. I still think @romayalon should also take a look as she knows much more in this area than me :)

[Neon-White]

I did not realize that #7427 was dependent on this PR, so I left my review comments there

[liranmauda]

I did not realize that #7427 was dependent on this PR, so I left my review comments there

@Neon-White Thanks for reviewing.
Can you please redo the review here?
the other PR is not getting updated and some comments are not relevant.

[Neon-White]

No major comments from my side