Merge master to feature/trusted-certs#7010
Merged
minglumlu merged 87 commits intoxapi-project:feature/trusted-certsfrom Apr 15, 2026
Merged
Merge master to feature/trusted-certs#7010minglumlu merged 87 commits intoxapi-project:feature/trusted-certsfrom
minglumlu merged 87 commits intoxapi-project:feature/trusted-certsfrom
Conversation
Also introduce a batching delay on all Event.{from,next} calls, except
Event.from calls for tasks. This should help batch up multiple events from
field updates.
There are no extra delays for tasks, because these typically aren't immediately ready,
and when they are we want to immediately find out about it.
This is part of the "feature flag" that turns on the feature
(previously you could've manually edited xapi.conf to make these changes.)
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com> Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
This will enable short-circuiting internal API calls. When the FD is missing mark the call as Internal. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
This picks up an earlier pull request by @edwintorok that was abandoned. We need to be sure that the code we are calling it thread safe.
…#6881) Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
…#6881) (xapi-project#6964) Did not catch this because it is inside `#ifdef`
Removing this device class from the USB config would let admins configure VMs for digital signing purposes. These devices are also not very useful in Dom0. Signed-off-by: Tu Dinh <ngoc-tu.dinh@vates.tech>
Quicktest was listing potentially thousands of VDIs and querying its properties, just to find out whether the resulting list was empty or not. We don't need to produce the full list to check whether it is empty or not: we can stop once we have the 1st element. Replace the List with a Seq. Before: ``` time ./quicktest.exe -run-only Quicktest_vm_lifecycle [...] real 0m13.957 ``` After: ``` real 0m12.584s ``` Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Some (read-only) SRs can contain a large number of VDIs, but these cannot "leak" VDIs if we cannot create or destroy them. Before: ``` real 0m12.584s ``` After: ``` real 0m7.853s ``` Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Before: ``` real 0m7.853s ``` After: ``` real 0m7.746s ``` Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Before: ``` real 0m7.746s ``` After: ``` real 0m6.095s ``` Signed-off-by: Edwin Török <edwin.torok@citrix.com>
The difference is not really measurable. But I found out there is another pre-existing flag that can speed it up, `-default-sr`. Then the leak checks are only done on that SR, and not all the SRs. So the best way is to run this (at least during development): ``` time ./quicktest.exe -run-only Quicktest_vm_lifecycle -skip-stress -default-sr [...] real 0m1.709s ``` Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Removing this device class from the USB config would let admins configure VMs for digital signing purposes. These devices are also not very useful in Dom0.
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Instead of xmlrpc Signed-off-by: Edwin Török <edwin.torok@citrix.com>
…forward the API call directly like we do with the CLI for calls to the coordinator when we are the coordinator Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Instead of parsing the string backtrace, use the record that is available since OCaml 4.02.
This is more future-proof, and performance is also improved ~10x:
Before:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 2674.017426 (confidence: 2685.374519 to 2661.996895);
r² = Some 0.999347 }
```
After:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 263.881150 (confidence: 264.748572 to 263.176838);
r² = Some 0.999726 }
```
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
With a long recursive call there could be thousands of these.
Show just one in the logs.
There is a small <5% performance impact:
Before:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 263.881150 (confidence: 264.748572 to 263.176838);
r² = Some 0.999726 }
```
After:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 275.430030 (confidence: 275.818027 to 275.087604);
r² = Some 0.999938 }
```
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
forkexecd, resources and xapi-stdext-pervasives wasn't using it. Signed-off-by: Edwin Török <edwin.torok@citrix.com>
…se_with_backtrace Signed-off-by: Edwin Török <edwin.torok@citrix.com>
…ndency The backtrace wasn't retrieved before calling the logging function, so it was potentially lost. Use Printexc module instead. Signed-off-by: Edwin Török <edwin.torok@citrix.com>
…ject#6968) This is long overdue, but we finally got some QA resource to complete the test plan, and no major bugs were found. There is one improvement that could be made: when deleting a user/group (or otherwise changing permissions) on the XAPI side we could immediately expire any caches related to that user.
OCaml 4.02 added column location, and 4.11 added function names.
Drop any wrapper modules by dropping anything before the last `__` in
the string, if any.
For backward compatibility with python interop exceptions these new fields are optional.
Backtrace.t_of_sexp is only called in `xapi_xenopsd.ml`, and we require updating XAPI and xenopsd together already.
For everything else (the Task API) we'll accept backtraces without the additional info, but will start producing backtraces
with the additional info.
<10% performance impact:
Before:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 275.430030 (confidence: 275.818027 to 275.087604);
r² = Some 0.999938 }
```
After:
```
Backtrace.is_important (ns):
{ monotonic-clock per run = 292.372339 (confidence: 293.039727 to 291.671323);
r² = Some 0.999771 }
```
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Ensures that when make_rpc switches protocols between XMLRPC and JSONRPC, then timeboxed_rpc follows. Have to be careful to pass the correct values for subtask_of (outer task), and task_id (inner task). Signed-off-by: Edwin Török <edwin.torok@citrix.com>
When using `opam` to install all packages from XAPI some libraries failed to build because the internal `backtrace` library was only available in the `xapi-log` package. The build worked when using `dune` directly (or `koji`), because they build the entire repository instead of individual packages. To avoid conflicts with the `xapi-backtrace` installed from `xs-opam` make the backtrace library properly part of `xapi-log`, by exposing it as `xapi-log.backtrace`. Put the cram tests into the xapi-log.backtrace package too. Add rpclib as a dependency for xapi-log, it was a dependency of xapi-backtrace. Fixes: e4da294 ("[maintenance]: use internal backtrace library") Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>
…mporting the module. Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>
There is a minor change in the datamodel, the rest are SDK files. These changes will need porting to `26.1-lcm` afterwards.
…ons (xapi-project#6989) A new live patch update may drop live patch support for a component running with an old version. This means the old version is EOL in sense of live patching support. Previously, the logic collected all live patches that shared the same base build ID for the running component and treated them as cumulatively applicable. With this change, if the base build ID is marked EOL in the latest relevant update (for the same live patching component), the component is considered not live‑patchable
A few functions in message_forwarding need to do some cleanup when encountering an exception, but in doing so, the original backtrace is lost. This makes it difficult to track down the original location that raised the exception. I've considered other options instead of calling Backtrace.is_important: - Call `Backtrace.reraise e e`: this is more expensive than it needs to be. - Create a new function that can be easily used: This is awkward to backport, and in my attempts the resulting code was not great to read. So I ended up with the compromise of using Backtrace.is_important, which does exactly what's needed here, even though it's difficult to enforce correctly across the codebase. Signed-off-by: Pau Ruiz Safont <pau.safont@vates.tech>
QEMU 10.x has changed some of the arguments' format, de-couple the xenopsd and qemu builds by making the qemu-wrapper script compatible with both the old and new versions of qemu. Signed-off-by: Gerald Elder-Vass <gerald.elder-vass@citrix.com>
Signed-off-by: Gerald Elder-Vass <gerald.elder-vass@citrix.com>
In QEMU upstream commit 7851b21 the piix3-ide-xen was removed as it duplicated the piix3-ide device (after 0f84458 made them identical). Therefore the global option(s) for piix3-ide-xen will be ignored unless they are updated to apply to piix3-ide instead. Provide both piix3-ide-xen and piix3-ide options to maintain behaviour across versions during the upgrade period. Options which are not relevant or used will simply generate a warning in the daemon.log but are otherwise ignored. Signed-off-by: Gerald Elder-Vass <gerald.elder-vass@citrix.com>
Newer versions of QEMU accept slightly difference arguments than older versions. - chroot and runas were replaced with run-with - 'on'/'off' boolean values are accepted in most versions - global variables are machine/device dependent Adding global variables for both piix3-ide and piix3-ide-xen ensures the variables are present the relevant device, variables which do not apply to any devices in use will log a warning (in daemon.log) but are otherwise ignored. Example of the warning(s) produced: ``` <time> <host> qemu-dm-<domid>[<pid>]: qemu-dm-<domid>: warning: global piix3-ide-xen.subvendor_id has invalid class name <time> <host> qemu-dm-<domid>[<pid>]: qemu-dm-<domid>: warning: global piix3-ide-xen.subsystem_id has invalid class name ``` These changes were tested alongside QEMU version 4.2.1 and QEMU version 10.1.0, in both cases guests booted as expected and the respective warning appeared in the daemon log.
This was updated in xs-opam, but missing in this repo Signed-off-by: Pau Ruiz Safont <pau.safont@vates.tech>
…project#6995) A few functions in message_forwarding need to do some cleanup when encountering an exception, but in doing so, the original backtrace is lost. This makes it difficult to track down the original location that raised the exception. I've considered other options instead of calling Backtrace.is_important: - Call `Backtrace.reraise e e`: this is more expensive than it needs to be. - Create a new function that can be easily used: This is awkward to backport, and in my attempts the resulting code was not great to read. So I ended up with the compromise of using Backtrace.is_important, which does exactly what's needed here, even though it's difficult to enforce correctly across the codebase.
This was updated in xs-opam, but missing in this repo
Previously these dependences were added to the opam file, but those are autogenerated. Add the metadata to the source of truth Signed-off-by: Pau Ruiz Safont <pau.safont@vates.tech>
Previously these dependences were added to the opam file, but those are autogenerated. Add the metadata to the source of truth
When we are using a known memory configuration it includes the overhead. So we only have to add it when we create a new configuration. Localhost migration in a loop before and after the patch, memory reported by list_domains -memory -bytes: Before: id | uuid | state | tot bytes | max bytes | shadow bytes 17 | 8fc1ada1-... | B H | 4296003584 | 4336898048 | 36700160 18 | 8fc1ada1-... | B H | 4296003584 | 4376743936 | 36700160 19 | 8fc1ada1-... | B H | 4296003584 | 4416589824 | 36700160 20 | 8fc1ada1-... | B H | 4296003584 | 4456435712 | 36700160 21 | 8fc1ada1-... | B H | 4296003584 | 4496281600 | 36700160 22 | 8fc1ada1-... | B H | 4296003584 | 4536127488 | 36700160 23 | 8fc1ada1-... | B H | 4296003584 | 4575973376 | 36700160 24 | 8fc1ada1-... | B H | 4296003584 | 4615819264 | 36700160 25 | 8fc1ada1-... | B H | 4296003584 | 4655665152 | 36700160 26 | 8fc1ada1-... | B H | 4296003584 | 4695511040 | 36700160 After: id | uuid | state | tot bytes | max bytes | shadow bytes 27 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 28 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 29 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 30 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 31 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 32 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 33 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 34 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 35 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 36 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 Based on work by Edwin Torok. Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>
This fix in the previous patch should obsolete the workaround in the quickcheck. Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
When we are using a known memory configuration it includes the overhead. So we only have to add it when we create a new configuration. Localhost migration in a loop before and after the patch, memory reported by list_domains -memory -bytes: Before: ``` id | uuid | state | tot bytes | max bytes | shadow bytes 17 | 8fc1ada1-... | B H | 4296003584 | 4336898048 | 36700160 18 | 8fc1ada1-... | B H | 4296003584 | 4376743936 | 36700160 19 | 8fc1ada1-... | B H | 4296003584 | 4416589824 | 36700160 20 | 8fc1ada1-... | B H | 4296003584 | 4456435712 | 36700160 21 | 8fc1ada1-... | B H | 4296003584 | 4496281600 | 36700160 22 | 8fc1ada1-... | B H | 4296003584 | 4536127488 | 36700160 23 | 8fc1ada1-... | B H | 4296003584 | 4575973376 | 36700160 24 | 8fc1ada1-... | B H | 4296003584 | 4615819264 | 36700160 25 | 8fc1ada1-... | B H | 4296003584 | 4655665152 | 36700160 26 | 8fc1ada1-... | B H | 4296003584 | 4695511040 | 36700160 ``` After: ``` id | uuid | state | tot bytes | max bytes | shadow bytes 27 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 28 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 29 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 30 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 31 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 32 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 33 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 34 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 35 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 36 | 8fc1ada1-... | B H | 4256157696 | 4297052160 | 36700160 ``` Based on work by Edwin Torok.
…api-project#6971) Instead of xmlrpc. Targeting feature branch. Will also likely need a new `ocaml-rpc` release (the current one doesn't handle Base64 and Datetime in JSONRPC correctly).
Resolves conflicts in helpers: the feature branch added an rpcfun, while master has moved out some code to a function and applied a functional argument.
Resolves conflicts in helpers: the feature branch added an rpcfun, while
master has moved out some code to a function and applied a functional
argument.
Prerequisite of xapi-project#6987
Draft, waiting for testing and PR reviews to complete. Depends on: xapi-project#6974 xapi-project#6971 Also would be good to have an updated ocaml-rpc for the JSONRPC fixes in xs-opam. PRs for ocaml-rpc optimizations: mirage/ocaml-rpc#194 mirage/ocaml-rpc#193 mirage/ocaml-rpc#192 mirage/ocaml-rpc#184 The optimizations are not strictly required, but the first PR about fixing 4.14 build is, otherwise we can't take the new version into xs-opam.
…out" It changes behaviour of the audit_log endpoint This reverts commit ab802e6. Signed-off-by: Pau Ruiz Safont <pau.safont@vates.tech>
…out (xapi-project#7014) It changes behaviour of the audit_log endpoint This reverts commit ab802e6.
987092d to
4da9ef6
Compare
BengangY
approved these changes
Apr 15, 2026
changlei-li
approved these changes
Apr 15, 2026
31944c9
into
xapi-project:feature/trusted-certs
16 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
commit 4da9ef6bddeed813a9687d51116062ae9914c97a Merge: 1c4938316 043e6bde3 Author: Ming Lu <ming.lu@cloud.com> Date: Wed Apr 15 09:56:22 2026 +0800 Merge branch 'master' into private/mingl/feature/trusted-certs diff --cc ocaml/idl/datamodel_common.ml index 93a3fbc6f,0fe08b11c..25e689f4f --- a/ocaml/idl/datamodel_common.ml +++ b/ocaml/idl/datamodel_common.ml @@@ -3,21 -3,21 +3,21 @@@ open Datamodel_types open Lifecycle open Datamodel_roles (* IMPORTANT: Please bump schema vsn if you change/add/remove a _field_. You do not have to bump vsn if you change/add/remove a message When introducing a new release, bump the schema minor version to the next hundred to leave a gap for potential hotfixes needing to increment the schema version.*) let schema_major_vsn = 5 - let schema_minor_vsn = 794 -let schema_minor_vsn = 901 ++let schema_minor_vsn = 902 (* Historical schema versions just in case this is useful later *) let rio_schema_major_vsn = 5 let rio_schema_minor_vsn = 19 let miami_release_schema_major_vsn = 5 let miami_release_schema_minor_vsn = 35 diff --cc ocaml/idl/schematest.ml index e92f9e944,c963c8f11..a7fc9bd13 --- a/ocaml/idl/schematest.ml +++ b/ocaml/idl/schematest.ml @@@ -1,16 -1,16 +1,16 @@@ let hash x = Digest.string x |> Digest.to_hex (* BEWARE: if this changes, check that schema has been bumped accordingly in ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *) - let last_known_schema_hash = "ce90c659723cbcd5265e4dd856802b74" -let last_known_schema_hash = "32bbba07579ca8844fa6162164530268" ++let last_known_schema_hash = "a64e7e1133a10add1b39d22cd6bbc352" let current_schema_hash : string = let open Datamodel_types in let hash_of_obj x = List.map rpc_of_content x.contents |> List.map Jsonrpc.to_string |> String.concat "" |> hash in Datamodel.all_system |> List.map hash_of_obj |> String.concat ":" |> hash