Commits on Apr 28, 2023

  1. multifd: We already account for this packet on the multifd thread 

    Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    291b612 View commit details
    Browse the repository at this point in the history

  2. migration: Move ram_stats to its own file migration-stats.[ch] 

    There is already include/qemu/stats.h, so stats.h was a bad idea.
We want this file to not depend on anything else, we will move all the
migration counters/stats to this struct.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    3fb55f4 View commit details
    Browse the repository at this point in the history

  3. migration: Rename ram_counters to mig_stats 

    migration_stats is just too long, and it is going to have more than
ram counters in the near future.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    fc6fcdd View commit details
    Browse the repository at this point in the history

  4. migration: Rename RAMStats to MigrationAtomicStats 

    It is lousely based on MigrationStats, but that name is taken, so this
is the best one that I came with.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>

---

If you have any good suggestion for the name, I am all ears.
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    bb6a3af View commit details
    Browse the repository at this point in the history

  5. migration/rdma: Split the zero page case from acct_update_position 

    Now that we have atomic counters, we can do it on the place that we
need it, no need to do it inside ram.c.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    2585c1d View commit details
    Browse the repository at this point in the history

  6. migration/rdma: Unfold last user of acct_update_position() 

    Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    02c1fa1 View commit details
    Browse the repository at this point in the history

  7. migration: Drop unused parameter for migration_tls_get_creds() 

    It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    32bcb96 View commit details
    Browse the repository at this point in the history

  8. migration: Drop unused parameter for migration_tls_client_create() 

    It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
    Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    c20062e View commit details
    Browse the repository at this point in the history

  9. qtest/migration-test.c: Add tests with compress enabled 

    There has never been tests for migration with compress enabled.

Add suitable tests, testing with compress-wait-thread = false
too.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    5d3b094 View commit details
    Browse the repository at this point in the history

  10. qtest/migration-test.c: Add postcopy tests with compress enabled 

    Add postcopy tests with compress enabled to ensure nothing breaks
with the refactoring in the next commits.

preempt+compress is blocked, so no test needed for that case.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    94c3c8d View commit details
    Browse the repository at this point in the history

  11. ram.c: Let the compress threads return a CompressResult enum 

    This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    280304a View commit details
    Browse the repository at this point in the history

  12. ram.c: Dont change param->block in the compress thread 

    Instead introduce a extra parameter to trigger the compress thread.
Now, when the compress thread is done, we know what RAMBlock and
offset it did compress.

This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    7ab0eb0 View commit details
    Browse the repository at this point in the history

  13. ram.c: Reset result after sending queued data 

    And take the param->mutex lock for the whole section to ensure
thread-safety.
Now, it is explicitly clear if there is no queued data to send.
Before, this was handled by param->file stream being empty and thus
qemu_put_qemu_file() not sending anything.

This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    7fe37c4 View commit details
    Browse the repository at this point in the history

  14. ram.c: Do not call save_page_header() from compress threads 

    save_page_header() accesses several global variables, so calling it
from multiple threads is pretty ugly.

Instead, call save_page_header() before writing out the compressed
data from the compress buffer to the migration stream.

This also makes the core compress code more independend from ram.c.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    b4be604 View commit details
    Browse the repository at this point in the history

  15. ram.c: Call update_compress_thread_counts from compress_send_queued_data 

    This makes the core compress code more independend from ram.c.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    85f44f9 View commit details
    Browse the repository at this point in the history

  16. ram.c: Remove last ram.c dependency from the core compress code 

    Make compression interfaces take send_queued_data() as an argument.
Remove save_page_use_compression() from flush_compressed_data().

This removes the last ram.c dependency from the core compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    9b1efb4 View commit details
    Browse the repository at this point in the history

  17. ram.c: Move core compression code into its own file 

    No functional changes intended.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    9e9bf8c View commit details
    Browse the repository at this point in the history

  18. ram.c: Move core decompression code into its own file 

    No functional changes intended.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    0c51904 View commit details
    Browse the repository at this point in the history

  19. ram compress: Assert that the file buffer matches the result 

    Before this series, "nothing to send" was handled by the file buffer
being empty. Now it is tracked via param->result.

Assert that the file buffer state matches the result.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    973704d View commit details
    Browse the repository at this point in the history

  20. ram-compress.c: Make target independent 

    Make ram-compress.c target independent.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    5766ce8 View commit details
    Browse the repository at this point in the history

  21. migration: Initialize and cleanup decompression in migration.c 

    This fixes compress with colo.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
    @Lukey3332
    Lukey3332 authored and Juan Quintela committed Apr 28, 2023
    Copy the full SHA
    05ecac6 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2023

  1. Merge tag 'migration-20230428-pull-request' of https://gitlab.com/jua… 

    …n.quintela/qemu into staging

Migration Pull request (20230429 vintage)

Hi

In this series:
- compression code cleanup (lukas)
  nice, nice, nice.
- drop useless parameters from migration_tls* (juan)
- first part of remove QEMUFileHooks series (juan)

Please apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmRMF90ACgkQ9IfvGFhy
# 1yPVUBAApRj2d6K6T0FwW8axL0LUD+7QXmMnTkhO9/QJDBHGban9hWuaH4n3Om+0
# 89tHl5Fo6bgOJMagUGfIuF1Xt9tJSFZMp3OMFxrV2+yhLqHQ7LlJSIFrO+ILeRQJ
# 3kKzTzBEb9LNVFPKrD2SyEvhGxxD1kHfnNydKdU3XqvyAWPXqr1BonxsF06+i5ud
# WRhEJ9iL9eFP4CkCPOMHxBhfme6gP+NkxaB6R2Eb30jNTP1U9nTlKaV2fGLVn/zY
# VRC1bV9Ujy3wGeKBp8Lm9FIa8yBIf/HImI+nWGFROYslECMAR0kMfEU+aWY5y8By
# t0xDD0OEh2QB7rjGpx1B3Pb2He5YznzQNG4kmjNqWcH+PWurRbBaN++1r1DHrQzk
# fhfsT3A9eWUM06JsMm2Y+IhHwzT+PddmjWjHDU/kPNn1+GnxFytC55Ie5ew62PLu
# jWtOTERSg38e6m2gdFM5PLnfP5aLmjH0An78CDFhVy8f8I0R89tIC915p4QECuaa
# IIetvv70NF92EOXEtV9IEt8XvCsOomXrUCCtQMSAthHsh37Vn3W2U8fmXluAclua
# AW/mllj1AqEaUHVvCyST5bq0xqzE90dqjUB474oKi+wtPIzpXb8LP0xVZvcU8zWo
# 53MNFybvcDoZdva1JynzUvug06k5o2t91wIkLyUvigZZQ+3Osrg=
# =rl9X
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 28 Apr 2023 08:00:45 PM BST
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [undefined]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20230428-pull-request' of https://gitlab.com/juan.quintela/qemu: (21 commits)
  migration: Initialize and cleanup decompression in migration.c
  ram-compress.c: Make target independent
  ram compress: Assert that the file buffer matches the result
  ram.c: Move core decompression code into its own file
  ram.c: Move core compression code into its own file
  ram.c: Remove last ram.c dependency from the core compress code
  ram.c: Call update_compress_thread_counts from compress_send_queued_data
  ram.c: Do not call save_page_header() from compress threads
  ram.c: Reset result after sending queued data
  ram.c: Dont change param->block in the compress thread
  ram.c: Let the compress threads return a CompressResult enum
  qtest/migration-test.c: Add postcopy tests with compress enabled
  qtest/migration-test.c: Add tests with compress enabled
  migration: Drop unused parameter for migration_tls_client_create()
  migration: Drop unused parameter for migration_tls_get_creds()
  migration/rdma: Unfold last user of acct_update_position()
  migration/rdma: Split the zero page case from acct_update_position
  migration: Rename RAMStats to MigrationAtomicStats
  migration: Rename ram_counters to mig_stats
  migration: Move ram_stats to its own file migration-stats.[ch]
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
    @rth7680
    rth7680 committed Apr 29, 2023
    Copy the full SHA
    b93c11d View commit details
    Browse the repository at this point in the history