Skip to content
This repository was archived by the owner on Jul 3, 2023. It is now read-only.
This repository was archived by the owner on Jul 3, 2023. It is now read-only.

API hardening TODOs #357

@lalithsuresh

Description

@lalithsuresh

Having attempted to interact with the REST API from the commandline (curl + websocat), here are some rough issues I've ran into that I'll try to coordinate with @gz to fix:

  • Currently, there is no way to ergonomically query objects in the API using information supplied by the user. It often requires users to create objects, then do list queries (read more state than needed), parse the results, and then do follow up queries. For example, when trying to use the HTTP input transport, the endpoint at the pipeline uses a generated name (connector-). To find that endpoint from a commandline, I need toGET /configs, use jq to search for the uuid of the entry that has config="<stream name>", use an out of band process to find the pipeline port, and then use the retrieved UUID and port to reach <pipeline url>:<port>/input_endpoint/connector-<uuid>.
  • The above problem comes down to revisiting object naming and identifiers, uniqueness requirements, search in the API, with the server side generating identifiers. This also affects idempotency expectations (e.g., when rerunning python/test.py, we create projects, pipelines etc with the same shape every time).
  • The server needs to generate UUIDs instead of an incrementing counter.
  • Component and schema evolution (e.g., adding new transport config entries requires rebuilding the python client, and adding new types of connectors requires an enum type to be available at the client).
  • API versioning.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions