Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync mater into feature/go_sdk #5634

Merged
merged 131 commits into from
May 16, 2024
Merged

Conversation

duobei
Copy link
Contributor

@duobei duobei commented May 16, 2024

A conflict resolved:

git show 9ee34d42f6215eb17bc7a59608bc96359130bb40
commit 9ee34d42f6215eb17bc7a59608bc96359130bb40 (HEAD -> feature/go_sdk)
Merge: ce89f1e71 4dd835175
Author: Luca Zhang <feiya.zhang@cloud.com>
Date:   Thu May 16 11:00:50 2024 +0800

    Merge branch 'master' into feature/go_sdk

diff --cc .github/workflows/generate-and-build-sdks.yml
index be83ba57f,9657b1f9b..78ec2579a
--- a/.github/workflows/generate-and-build-sdks.yml
+++ b/.github/workflows/generate-and-build-sdks.yml
@@@ -24,9 -24,14 +24,17 @@@ jobs
          shell: bash
          run: opam exec -- make sdk
  
 +      - name: Run CI for SDKs
 +        uses: ./.github/workflows/sdk-ci
 +
+       - name: Store C SDK source
+         uses: actions/upload-artifact@v4
+         with:
+           name: SDK_Source_C
+           path: |
+             _build/install/default/xapi/sdk/c/*
+             !_build/install/default/xapi/sdk/c/dune
+ 
        - name: Store C# SDK source
          uses: actions/upload-artifact@v4
          with:

GabrielBuica and others added 30 commits March 27, 2024 11:03
Add standard http attributes to from opentelemetry such as:

- `http.request.method`
- `http.request.header.content-type`;
- `http.request.body.size`;
- `http.request.header.user-agent`;
- `http.request.header.(key)`.

This improves debuggability associated with client requests.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
With *hybrid source format in export, the following cases are supported:
1. nbdhybrid: QCOW2 -> NBD device in dom0 -> exported file
2. nbdhybrid: VHD -> NBD device in dom0 -> exported file
3. hybrid: VHD -> blktap device in dom0 -> exported file

The case 2 above can't support an optional parameter "base".
This parameter holds the ID of another VHD VDI. When it is passed, the
export will only write the differences between "source" and "base" to
the destination file.

As a short-term solution, in case 2 above, when the "base" is passed,
the source format is changed to "hybrid" in this commit.
This can work because:
1. the comparsion on blocks required by "base" is supported by "hybrid";
2. the raw data from NBD device and blktap (Frankentap) device are same;
3. the sparseness information of the source required in case 2 can be
   get by either NBD interface (nbdhybrid) or VHD parsing (hybrid).

Signed-off-by: Ming Lu <ming.lu@cloud.com>
waitpid_nohang closes the socket by calling waitpid and then calls
clear_nonblock on that socket, which fails.

waitpid_nohang's policy around closing the socket is not obvious enough.
Make it more explicit by not calling waitpid but inlining code and
looking at the different cases. The behaviour is still to close the
socket when the child process has terminated and not otherwise.

Logging can be re-enabled if we have to come back to this.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
…384483

CA-384483: Can't export VDI to VHD file with base VDI
Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>
Signed-off-by: Benjamin Reis <benjamin.reis@vates.tech>
Slightly different from missing file, add missing path.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Clean even if user press Ctrl-C, not only sending TERM signal.
Remove possible stale unix socket to allow to run multiple times.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Make sure we get the correct output from program launched.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Make sure program receives the string we are sending.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Requires libxml2-devel installed.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
(cherry picked from commit 0008395)
The C SDK build was failing with a recent GCC on Fedora39 like this:
```
src/xen_common.c: In function ‘xen_session_logout’:
src/xen_common.c:298:5: error: ‘xen_call_’ accessing 16 bytes in a region of size 0 [-Werror=stringop-overflow=]
  298 |     xen_call_(session, "session.logout", params, 0, NULL, NULL);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Use `NULL` instead of a VLA of size 0.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
(cherry picked from commit bfe78bd)
…dora-fix

Port of "Fix C SDK build on Fedora39" to `master`
Domains using TPMs need to have SWTPM always available. Encode this information
in the service file so the services are shut down at the correct time while
powering off the hosts.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
I am not sure where this restriction came from. It was introduced in the
same commit that introduced the metadata-export feature itself, without
explanation. Full exports of snapshots are already allowed.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
This allows clients to know which snapshots they knowingly just created and act
on them instead of looking again into the db and try to divine which snapshots
the call created.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This allows clients to detect that the newer device type exclusions for export
are available.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
This allows to orchestrators to copy devices like VBDs from the VM, export the
metadata without the disk, import the VM to any pool, and finally add the
saved VBDs to the new VM before booting it.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This allows to easily test the http endpoint. Also advertises the metadata
parameter on these calls

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Use the helper functions to process parameters, use atomics instead of ad-hoc
references for counting in possibly-parallel case

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Tooling complains about using 1970 as the date otherwise

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
…ernal headers from the public ones).

Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
GabrielBuica and others added 26 commits April 30, 2024 13:55
Instrument:

- `forkhelpers.ml`,
- `fecomms.ml`

to create spans around functions when a parent span is supplied as
`tracing`.

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Currently the `observe` mode of the `tracing` library do not work
corrently resulting in the logs being spammed by this warning.

Comment it out so that the logs do not become too big (for the time
being).

Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
tests: Allow the alcotest_suite to run
The API call VM.set-has-vendor-device used to be a lincensed feature but
it no longer is. As a first step to simplify when Windows VMs
automatically or not update device drivers, remove the license checks in
the code. The feature flag still remains in place.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
The VCustom value was only used for the has-vendor-device field in a VM
and contained code rather than a simple value. We are removing this.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
Remove the function and use VM.create_from_record directly.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
…/CP-48195-instrument-forkexecd-client

CP-48195: Instrument client side of `forkexecd`
- Use `Types.checkError` instead of throwing a generic `XenAPIException`. This ensures the `Types.XYZ` family of exceptions are being used
- Use `@JsonValue` to ensure base class objects are deserialised as a simple opaque_ref string, as opposed to a mapping of each field. This ensures the API's behaviour is unchanged.
- Parse the results of `task.getResult` calls. The jsonrpc method returns value payloads of the form `"value" : "<value>OpaqueRef:XYZ</value>"` so we need to strip the surrounding XML

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
…e build @check

Bytecode builds for `http_lib` are disabled due to '(modes best)',
and that means that anything that depends on it must have it disabled too to avoid this warning.

Avoids these kinds of warnings:
```
File "_none_", line 1:
Error: Module `Buf_io' is unavailable (required by `Http_svr')
```

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…ries

There were 3 modules with conflicting names with compiler libraries: Watch,
Debuginfo and Stats. Debuginfo was renamed, the others's libraries were changed
to be wrapped.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Pinning the libraries runs dune subst, which needs a project name, define it

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Do not output loglines that are part of the normal operation. Use debug for
them, they are not usually logged, but can be enabled if need be by changing
the loglevel

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
New version got released

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
The query on HTTP endpoint /updates will return the available updates in
JSON format. Prior to the changes in this commit, if a query arrives
when another query is being handled, a "GET_UPDATES_IN_PROGRESS" error
will be returned immediately. This behaviour is not friendly to GUI
client XenCenter.

In this commit, the behaviour is changed to wait and retry in handling
the query in xapi since the "*_IN_PROGGRESS" error is a transient
failure. Tolerating it in xapi (server) side avoids error handling in
client side.

With the change, the "GET_UPDATES_IN_PROGRESS" will not be an error
exposed to users any more. Therefore it is removed.

Signed-off-by: Ming Lu <ming.lu@cloud.com>
…389319

CA-389319: Wait and retry for GET_UPDATES_IN_PROGRESS
As part of a start, resources are allocated for a VM in "scheduled_to.."
fields. These need to be cleared if the start fails. It turned out that
this was incomplete for PCI slots and those were leaking. This patch
tries to be more systematical about it.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
This is mostly copied from the jekyll version at xapi-project.github.io and
amended for Hugo. The main difference is that all XenAPI pages have now been
integrated with the main menu in the left side bar, including the menu of the
XenAPI classes.

Some automation is left to do in order to take the `doc/data` files from a
build of `ocaml/idl` and update the class pages using the script
`doc/make-class-pages.py`. Also the overview of xapi releases is still missing,
as well as some of the API guides.

Finally, there is some overlap with markdown files in `ocaml/doc`, which needs
to be sorted out.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
From Xen 4.19 onwards the legacy paths disappeared and the only valid path for
pygrub is /usr/libexec/xen/bin/pygrub. This path has always been preferred, but
now it's mandatory.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
@duobei duobei changed the title Sync mater into Feature/go_sdk Sync mater into feature/go_sdk May 16, 2024
@duobei duobei merged commit e205457 into xapi-project:feature/go_sdk May 16, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet