Introducing migrationProjectId and decoupling Spanner projectId

[darshan-sj]

Introduced migration project id. Utilized the new migration project id to create migration resources instead of spanner's project id.

Tested:

• CLI
  • schema, data, schema-and-data, cleanup commands for Sharded Live migration
  • schema, data, schema-and-data, cleanup commands for Non Sharded Live migration
  • Both the above cases with different migration project id and spanner project id
• UI
  • schema, data, schema-and-data, cleanup commands for Sharded Live migration
  • schema, data, schema-and-data, cleanup commands for Non Sharded Live migration

Pending tasks:

• Feature request - UI support for separate migration shard id #807

• Feature request - Correct cleanup command to store and use migration project id from metadata db #808

• Feature request - Document permissions required to run migration with different migration project and spanner project #809

• Documentation - Document usage of Cleanup command in the github documentation page #810

• Tests pass

• Appropriate changes to README are included in PR

[codecov]

Codecov Report

Merging #802 (f52849c) into master (c5e5b2e) will decrease coverage by 0.04%.
The diff coverage is 21.01%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #802      +/-   ##
==========================================
- Coverage   45.96%   45.92%   -0.04%     
==========================================
  Files         166      166              
  Lines       20013    20032      +19     
  Branches      471      471              
==========================================
+ Hits         9199     9200       +1     
- Misses      10300    10318      +18     
  Partials      514      514              

Components	Coverage Δ
backend-apis	44.38% <23.07%> (-0.09%)	⬇️
backend-library	50.56% <20.61%> (-0.04%)	⬇️
cli	0.00% <ø> (ø)
frontend	32.08% <ø> (ø)

Files	Coverage Δ
common/utils/utils.go	0.00% <ø> (ø)
conversion/conversion.go	29.62% <100.00%> (ø)
conversion/mocks.go	91.66% <100.00%> (ø)
conversion/resource_generation.go	94.98% <100.00%> (ø)
sources/common/infoschema.go	0.00% <ø> (ø)
webv2/api/schema.go	57.86% <100.00%> (ø)
sources/dynamodb/schema.go	76.34% <0.00%> (ø)
sources/spanner/infoschema.go	5.72% <0.00%> (ø)
sources/sqlserver/infoschema.go	56.96% <0.00%> (ø)
webv2/session/session_handler.go	1.68% <0.00%> (ø)
... and 10 more

[manitgupta]

Overall looks good. Added some comments to check if there is an opportunity to simplify by using context instead of adding a new parameter wherever possible.

This is not a small amount to work to figure out quickly! Thanks a lot!

[krishnamoorthy-r]

Thanks for the changes, looks much bigger than what I originally thought!

[manitgupta]

Please update the PR description with -

1. What is to be done as follow up work. Create bugs and linked them here.
2. What testing has been done. I would recommend smoke testing the common pathways.

[manitgupta]

1. Please also update the Github documentation with these changes
2. Please update the PR description with what features break under what scenarios. For example - cleanup CLI will not work for cases when spanner project is different from migration resources project (and so on and so forth)

[darshan-sj]

Updated the description with manual testing done and listed down the pending tasks.

[darshan-sj]

I have updated the github documentation as well