Replicate does not refresh cached api specs.

[PotentialIngenuity]

Summary

I am performing a replication using an upstream pulp with the pulp_label_select option enabled. I also have the policy set as labeled. I am getting this error about the URL parameters.

Steps to reproduce

Initialize an upstream definition like so:

  {
    "pulp_href": "/pulp/api/v3/upstream-pulps/01969319-e8df-7ed6-93bc-768aa9742e6a/",
    "prn": "prn:core.upstreampulp:01969319-e8df-7ed6-93bc-768aa9742e6a",
    "pulp_created": "2025-05-02T22:24:30.945414Z",
    "pulp_last_updated": "2025-05-02T22:24:30.945457Z",
    "name": "repo01-adp",
    "base_url": "https://repo.company.com",
    "api_root": "/pulp/",
    "domain": null,
    "ca_cert": null,
    "client_cert": null,
    "tls_validation": false,
    "hidden_fields": [
      {
        "name": "client_key",
        "is_set": false
      },
      {
        "name": "username",
        "is_set": true
      },
      {
        "name": "password",
        "is_set": true
      }
    ],
    "q_select": "pulp_label_select='content_type=adp-common,content_type=adp-platform,content_type=adp-core'",
    "last_replication": null,
    "policy": "labeled"
  },

Expected behavior

replicate should initiate and complete successfully

Stacktrace/Error log

May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]: pulp [849cbddb64804f81980787c20f339f52]: pulpcore.tasking.tasks:INFO: Task[pulpcore.app.tasks.replica.replicate_distributions] 0196931a-0885-73e4-a7e6-9f823cb3b82a failed (OpenAPIError: Parameter [q] not available for distributions_container_container_list.) in domain: default
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]: pulp [849cbddb64804f81980787c20f339f52]: pulpcore.tasking.tasks:INFO:   File "/usr/local/lib/python3.9/site-packages/pulpcore/tasking/tasks.py", line 71, in _execute_task
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     result = func(*args, **kwargs)
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulpcore/app/tasks/replica.py", line 83, in replicate_distributions
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     for distro in distros:
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulpcore/app/replica.py", line 96, in upstream_distributions
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     distributions = self.distribution_ctx_cls(self.pulp_ctx).list(list_size, offset, params)
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulp_glue/common/context.py", line 909, in list
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     entities.append(next(iterator))
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulp_glue/common/context.py", line 880, in list_iterator
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     response: t.Mapping[str, t.Any] = self.call("list", parameters=payload)
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulp_glue/common/context.py", line 680, in call
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     return self.pulp_ctx.call(
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulp_glue/common/context.py", line 450, in call
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     result = self.api.call(
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:   File "/usr/local/lib/python3.9/site-packages/pulp_glue/common/openapi.py", line 617, in call
May  2 17:24:39 172.18.0.1 docker-pulp_worker-1[58721]:     raise OpenAPIError(

Pulp and pulp-cli version info

pulp core: 3.75.2
pulp-cli: 0.32.1
pulp-glue: 0.32.1

Additonal context

[PotentialIngenuity]

I see this will probably be fixed by pulp/pulp-cli#1182

[PotentialIngenuity]

@mdellweg Is that PR related to this issue I am seeing?

[mdellweg]

I don't think so.
This sounds more like an issue of the replica talking to the upstream server.
Can you add some version info for both?

[PotentialIngenuity]

My setup

repo01 (source/upstream) -> repo02 (target/replica)

Both
pulp-core 3.75.2

repo01
python 3.11
pulp-cli 0.31.1
pulp-glue 0.31.1 squeezer limit

repo02
python 3.9
pulp-cli 0.32.1
pulp-glue 0.32.1

[mdellweg]

Being in the same pulpcore version is surprising but let's get this to the right audience.

[PotentialIngenuity]

@mdellweg any updates?

[mdellweg]

Looking at https://pulpproject.org/pulp_container/restapi/#tag/Distributions:-Container/operation/distributions_container_container_list
there is a "q" parameter. Can you compare the pulp_container versions too?
Can you maybe delete the cached api.json files on the replica installation? (I'd suspect them to be somewhere in ~pulp/.cache/)

[PotentialIngenuity]

pulp_container is 2.24.1 on both instances.

I cleard the cache and re-ran the replication and it ended with the same error message.

[mdellweg]

OK, so i found the file here: ./var/lib/pulp/.cache/squeezer/http___localhost_5001__pulp_api_v3_docs_api.jsonapi.json But this may depend on how you run your installation.

[PotentialIngenuity]

I did a find and deleted the few that showed up. The replication ran successfully.

find / -name *.jsonapi.json

[mdellweg]

Thanks for trying. I believe this actually brings up an issue.