Skip to content
This repository was archived by the owner on Dec 21, 2021. It is now read-only.

Conversation

@tumppi
Copy link

@tumppi tumppi commented Feb 3, 2021

Field detection support was removed from backend a while ago so Stream.detectFields() call is not working. This PR adds this simple logic using FieldDetector into the client itself.

@linear
Copy link

linear bot commented Feb 3, 2021

NET-182 Replace API call with field detection implementation in JS client

In the JS client, there's a method that calls an API endpoint detectFields which has been removed.

The JS client should instead implement the field detection on client level:

  • Get a resend of the latest 1 message from the stream
  • Inspect the message contents and deduce the field schema from that
  • Save the config on the stream

The inspection logic used to be in the broker and was something like this:

https://github.com/streamr-dev/broker/blob/cc24ac3ee223f442a63a3f23ada6de21d4d6acb4/src/websocket/FieldDetector.js#L24-L40

@tumppi tumppi requested a review from timoxley February 3, 2021 13:30
@tumppi
Copy link
Author

tumppi commented Feb 12, 2021

The logic is so simple that it's now part of existing detectFields.

@timoxley timoxley force-pushed the NET-182-detect-fields branch from 39d6fc1 to 2ce2157 Compare February 17, 2021 19:47
@timoxley
Copy link
Contributor

Rebased, added types. LGTM.

getEndpointUrl(this._client.options.restUrl, 'streams', this.id, 'detectFields'),
this._client.session,
)
// Get last message of the stream to be used for field detecting
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should try to be a bit more fault tolerant, or smarter, about handling cases where the data is shaped differently e.g. grab the last n messages and base the type on a combination of the n, or raise a warning if they aren't all of the same time.

None of this is enforced really though so maybe it doesn't matter.

})

// Save field config back to the stream
this.config.fields = fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, what happens if detectFields is called twice in parallel and the async calls resolve out of order? Going forward perhaps could ensure only a single call can be outstanding

@timoxley timoxley self-requested a review February 18, 2021 16:25
@timoxley timoxley merged commit b50eb2a into 5.x Feb 18, 2021
@timoxley timoxley deleted the NET-182-detect-fields branch February 18, 2021 16:26
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants