Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CA-390277: Reduce record usage on CLI cross-pool migrations #5773

Merged
merged 4 commits into from
Jul 16, 2024

Conversation

psafont
Copy link
Member

@psafont psafont commented Jul 3, 2024

Using records in cross-pool migration code is dangerous, as the code interacts with potentially newer hosts. This means that fields in the record might be different from what's expected. In particular, adding an enum field can break the deserialization, and removing a field as well.

Instead, introduce and use a new call get_all_where: this calls allows the host to return a filtered list of object references.

With these changes, only SR records are used. This is done to minimize the number of calls done.

@psafont
Copy link
Member Author

psafont commented Jul 3, 2024

I've verified manually that it works with and without remote-host, and with and without a remote-network, especially using the name label. For the network, the bridge parameter is strange since it needs the reference, not its name (e.g. xenbr0).

# xe vm-migrate live=true remote-username=root remote-password=xxx \
   remote-master=uk-14-05.internal host=uk-14-05 \
   remote-network="Pool-wide network associated with eth0" \
   vm=a8d49c18-69af-fbd4-b4ab-51916073ef15 \
   vif:78502fd1-9b80-cad7-6eb3-cbea04bd1eee=296f1a07-bfc7-8038-7108-f8ddb9cbd322
Performing a storage live migration. Your VM's VDIs will be migrated with the VM.
Will migrate to remote host: uk-14-05, using remote network: Pool-wide network associated with eth0. Here is the VDI mapping:
VDI 807f74a0-be10-48e7-b992-df423e6ff71a -> SR bb7073ab-1e0d-40be-72c8-4653539ce2e7

ocaml/xapi-cli-server/cli_operations.ml Show resolved Hide resolved
ocaml/xapi-cli-server/cli_operations.ml Show resolved Hide resolved
ocaml/xapi-cli-server/cli_operations.ml Show resolved Hide resolved
ocaml/idl/datamodel_utils.ml Show resolved Hide resolved
ocaml/idl/ocaml_backend/gen_db_actions.ml Show resolved Hide resolved
ocaml/xapi-cli-server/cli_operations.ml Show resolved Hide resolved
remote Client.PBD.get_all_where ~expr
|> List.map (fun pbd ->
let sr = remote Client.PBD.get_SR ~self:pbd in
(sr, remote Client.SR.get_record ~self:sr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it is safe to use the SR record, but not the host and network record?

Copy link
Member Author

@psafont psafont Jul 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not safe, but not using them means risking a large amount of roundtrips between both servers to order the SRs.

The internal tests should now try to find this issue continually and catch this issue in time

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So using SR record is unsafe, but there hasn't been problems using them. And we can fix them once there are changes causing them to become real problems, is that right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we can fix them once there are changes causing them to become real problems, is that right?

We have 3 options here:

  • Roll back breaking changes when the are found
  • Create queries for sorting (mimicking SORT BY, similar to the current where) and use that. This seems like reinventing SQL queries.
  • Make the client robust to adding fields (ignore them), and removing them (allowing client to fail if they needed it)

@coveralls
Copy link

Coverage Status

coverage: 44.887%. remained the same
when pulling 62dd88d on psafont:getallwhere
into e61e0ac on xapi-project:master.

@xapi-project xapi-project deleted a comment from coveralls Jul 4, 2024
@Vincent-lau
Copy link
Contributor

Looks good to me, as long as the use of SR record is ok....

Copy link
Member

@robhoes robhoes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, but otherwise this looks good.

ocaml/idl/datamodel_utils.ml Outdated Show resolved Hide resolved
@@ -1374,9 +1377,9 @@ let message_destroy (_ : printer) rpc session_id params =
| None, Some uuids ->
String.split_on_char ',' uuids
| Some _, Some _ ->
fail "Ambiguous arguments; need one of uuid, uuids"
failwith "Ambiguous arguments; need one of uuid, uuids"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's confusing that failwith is overridden in this module...

Making all the cases explicit avoids wrong behaviour when adding a new message

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This is useful to avoid fetching whole records in vm migrations with different
versions. This is because records can contain new variants, which can fail to
be serialized.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This reduces repetition, makes calls easier to read, and shorter.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Using records in cross-pool migration code is dangerous, as the code interacts
with potentially newer hosts. This means that fields in the record might be
different from what's expected. In particular adding an enum field can break
the deserialization, and removing a field as well.

With this change, only SR records are used. This is done to minimize the number
of calls done.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
@psafont psafont merged commit cbd156c into xapi-project:master Jul 16, 2024
15 checks passed
@psafont psafont deleted the getallwhere branch July 16, 2024 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants