Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client: make streaming functions easier to consume #46528

Open
thaJeztah opened this issue Sep 21, 2023 · 0 comments
Open

client: make streaming functions easier to consume #46528

thaJeztah opened this issue Sep 21, 2023 · 0 comments
Labels
area/api area/cli kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. kind/refactor PR's that refactor, or clean-up code

Comments

@thaJeztah
Copy link
Member

Description

This is just a quick draft, related to #46527, but there's been other discussions in this area that I may add here. This ticket should also be added to an epic (to be created) about improving the client (or providing a better sdk / better tools for functionality that's currently living in docker/cli, but should be more easy to consume).

The client.ImagePull (as well as other functions that use a streaming endpoint)
function is very hard to consume. To use these functions, a great amount of
boiler-plating code, as well as custom logic is needed, and the utilities to
help with this (pkg/jsonmessage and/or pkg/progress, which both seem to be
serving similar functionality), are poorly written;

Looking at this (client.ImagePull) example:

  • The error returned is only for the initial request (which is "reasonable",
    and makes sense for some situations, but is definitely not intuitive).
  • A result of that is that not handling the stream means "cancelling" the
    pull.
  • To get errors that occur during the pull, it's required to process the
    stream using the jsonmessage package. And the utilities in this package
    are tightly integrated with presenting the stream (i.e., error-handling
    is part of "presenting" the stream).
  • Because of this coupling, it also expects things like "do we have a terminal
    attached"? (and if so: its file-descriptor), which is irrelevant for many
    scenarios where the pull is to be used for purposes other than a pull on
    the command-line.
  • if we get an error, it's a JSONError, which does have a status-code
    (although it appears to not be used) and cannot be handled by the errdefs
    package.

We should make this better. Some options to consider:

  • (lower hanging fruit): add an "synchronous" option, and move the stream-
    handling logic into the ImagePull function. This makes the function
    handle the pull from start to finish, printing the output (if io.writers
    are passed for this), and returns any error that occurs while pulling,
    which could be either the initial error (making the request), or errors
    during the pull.
  • It would also make sure to return errors with a correct errdefs type
    for further processing.
  • For the longer term, we must rewrite the pkg/jsonmessage, as well as
    the pkg/progress packages: remove the duplication, and make sure we
    have a canonical implementation where needed.
  • But also separate handling the stream from presenting the stream;
    presentation should be separate, and something the consumer should be
    able to have full control over.
  • This would be smilar to the auxCallback which was added at some point,
    but instead, any presentation function / callback should be optional
    (possibly some default with printing to a vanilla io.writer: TBD).
@thaJeztah thaJeztah added area/api area/cli kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. kind/refactor PR's that refactor, or clean-up code labels Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api area/cli kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. kind/refactor PR's that refactor, or clean-up code
Projects
None yet
Development

No branches or pull requests

1 participant