Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --duplicated-nevra "keep-last" option, and --delayed-dump #325

Merged

Commits on Nov 1, 2022

  1. Make the --update old_metadata override hack simpler

    Even though we are destructively changing the recycled cr_Packages data,
    we can keep the structure self-contained so the upper level logic will
    automatically take care of the allocated memory.
    
    The recycled metadata cr_Packages use one chunk per all packages
    (CR_PACKAGE_SINGLE_CHUNK).  That's why (pkg->chunk == NULL) and we can
    re-use it for a new chunk.  We just need to change the type of pkg to
    !CR_PACKAGE_SINGLE_CHUNK so the cr_package_free() cleans the new
    chunk.
    
    This allows us to stop taking care of the locally allocated location_*
    variables, because they are _cleanup_free_.  This makes even the buffer
    handling simpler.
    praiskup committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    0bf32d7 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2022

  1. Don't remove old metadata packages with --update

    Revamped 'pkg_from_md' that was unused for some time, but renamed to
    'clean'.
    praiskup committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    ed9ce71 View commit details
    Browse the repository at this point in the history
  2. Better 'res' and 'pkg' allocation handling

    The 'task_cleanup' target was reached without free-ing the 'pkg'
    structure in one of the error cases before.
    
    The 'res' was allocated on two places.
    praiskup committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    c85bbf9 View commit details
    Browse the repository at this point in the history
  3. New helper wait_for_incremented_ids()

    Will be needed later for the "skip" mechanism.
    praiskup committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    8db3a6a View commit details
    Browse the repository at this point in the history
  4. New (hidden) --delayed-dump option

    Normally we dump (to XML and DB) directly from the pool of threading
    workers, in a streaming fashion (as soon as the metadata about packages
    are determined it is written).
    
    The new option causes that the metadata are loaded first (still in
    parallel), and then written at one time (by one thread for now).
    Since all the metadata (cr_Package) is available in the memory in one
    time, we can do some additional processing and filtering (before we
    eventually perform the dump).
    
    This is an optional behavior because the delayed XML chunk preparation
    slows down the overall multi-threaded processing time.
    praiskup committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    24f0a34 View commit details
    Browse the repository at this point in the history
  5. New --duplicated-nevra <solution> option

    Commit 8fb99cd added a detection
    mechanism for duplicated NEVRA packages and an useful warning.
    
    The new option '--duplicated-nevra keep' keeps the default behavior (if
    specified) — all the duplicate packages are dumped into metadata.
    
    When '--duplicated-nevra keep-last' argument is used, only the last
    NEVRA (ordered by build time, then by its ocation path) is dumped to
    the metadata and the former duplicates are skipped.
    
    Relates: https://pagure.io/copr/copr/issue/840
    Fixes: 325
    praiskup committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    856bdb8 View commit details
    Browse the repository at this point in the history