pipeline-manager: add a config API endpoint to get a list of canned demo URLs

[lalithsuresh]

Signed-off-by: Lalith Suresh lalith@feldera.com

Is this a user-visible change (yes/no): yes

[snkas]

• Canned demo endpoint now returns a list of JSON objects of type:

  [
    {
        "title": "...",
        "description": "...",
        "url": "..."
    },
    ...
  ]
  

• Added a first canned demo: warehouse

• Rebased against main (kept the Cargo.lock changes)

• Minor comment and text changes

[snkas]

Command to run it directly:

cargo run --bin=pipeline-manager --features pg-embed -- --dev-mode --canned-demos='{"title": "Warehouse", "description": "Use case of a warehouse that stores product items which are bought and sold.", "url": "https://raw.githubusercontent.com/feldera/feldera/features/demo/demos/warehouse/demo.json"}'

... and then you can check the endpoint using:

curl -X GET 'http://localhost:8080/v0/config/demos' | jq

[lalithsuresh]

@snkas My understanding was that the title and description will be encoded in the json file that describes the demo. I don't think we need to repeat that in the CLI argument here. Besides, it'll be a pain to render via Helm with all the escaping. :)

[snkas]

@lalithsuresh , @Karakatiza666 had a strong preference to have the name and description be part of the demo list itself I think for UI placeholders while the demo JSON is loaded in. I agree it becomes a pain in terms of escaping when passing serialized JSON... We can keep it a list of URLs in this first version and then revisit it later on. What do you think @Karakatiza666 ?

[snkas]

Reverted to just a list of strings (URLs).

[snkas]

Split off the canned demo warehouse to separate PR: #1401

Command to run it directly:

cargo run --bin=pipeline-manager --features pg-embed -- --dev-mode --canned-demos=https://raw.githubusercontent.com/feldera/feldera/canned-demo-warehouse/demo/demos/warehouse/demo.json

... and then you can check the endpoint using:

curl -X GET 'http://localhost:8080/v0/config/demos' | jq

[snkas]

Ready for final review

Notes:

• There is not yet a --canned-demos <...> argument added to the pipeline-manager in the docker-compose.yml because (1) test-docker-compose-stable will not work as the previous image does not have the argument, and (2) there are not yet any canned demos merged into main.
  As such, the demos tab will be empty in the docker compose for now when building from source.
• My idea is the following: leave the docker-compose.yml have no --canned-demos <...> argument until the next release, at which point afterwards (after the release) I update the docker-compose.yml with the --canned-demos <...> of all the canned demos we have merged into main till then.
• Added CHANGELOG entry
• @lalithsuresh : I kept the Cargo.lock changes from your original commit

[ryzhyk]

Does this have to be under /config? Seems like demos should be a separate tag.

[snkas]

The reasoning is I think that it is part of the configuration with which you start the pipeline-manager (it is provided as a command-line argument), as such you fetch it from the config.

[ryzhyk]

Yeah, I get it. I do think it's a bit of a stretch, but I'm ok with it.

[snkas]

Rebased with main, awaiting CI to pass to merge.

Changes:

• Rebasing
• "self-describe" -> "describe"
• Removed "canned"