feat: Add dynamic type by scbrickley · Pull Request #5212 · influxdata/flux

scbrickley · 2022-09-19T17:44:33Z

Rebased and modified version of #4762.

Added some tests to make sure that retrieving a dynamic value via index or member access still returns a dynamic value. Calling Dynamic() on any non-dynamic value produces an UnexpectedKind panic (similar to the other methods in the value interface). Similarly, attempting to call Int(), UInt(), Float(), etc. on a dynamic type also produces an UnexpectedKind panic. This is to enforce the rule that dynamic values must be explicitly cast to the appropriate type in order to be used as such.

libflux/flux-core/src/semantic/tests.rs

onelson · 2022-09-20T18:04:44Z

semantic/monotype.go


+// NewDynamicType will construct a new Dynamic MonoType
+func NewDynamicType() MonoType {
+	builder := flatbuffers.NewBuilder(32) // FIXME: how big?


The size of this is still an open question. I'm not that familiar with flatbuffers, but it does seem to me like it should be possible to mark this as 0-sized so long as whatever fields are associated with the node size themselves.
I'm thinking in terms of the BaseNode fields we see like the location of the node, etc.

values/dynamic.go

scbrickley · 2022-09-21T18:49:04Z

libflux/flux-core/src/semantic/nodes.rs

+        if let MonoType::Dynamic(_) = &t {
+            self.typ = t.clone();
+            return Ok(());
+        }
+


@onelson This and the accompanying change to Expression::Index.infer() were my solution to get the new type inference tests to pass. This seems... hacky though. Open to suggestions for other solutions.

I think that looks fair. I'm no expert though. My reading of this says that when r is dynamic, assign the type of dynamic to expression accessing the property (which is what we want).

The fact you're cloning t to do this is okay, I think? Otherwise you could look to assign a new instance of the type (but I'm not sure why that'd matter). There's a flatbuffer pointer inside which I'm not sure how to construct by hand 😭

Someone else with more experience in this are should probably check this out. Maybe Jonathan has input.

Edit: As an aside, in the vectorize code, we frequently wrap the originally inferred type of things by cloning the existing monotype and passing it into a wrapper type.

The difference between this situation and the vectorize code is in this case we're talking about a whole new value instead of replacing existing expressions to change their typing.

I'd probably want to get @Marwes to have some eyes on this. Type inference is a strange area of code and doesn't really work in ways that are procedural at all times so this condition may not be valid.

I, unfortunately, don't have any input on this.

Doing more targeted checks like this seems better to me generally, but I could maybe see issues depending on the order that type inference runs. That is, this requires that we know that the type is dynamic when we reach this point in the code, we can't discover that it is dynamic later and then pass this check. I don't know if that is actually a problem though

Did we ever figure this out? It seems to me like it's possible this condition might not work correctly if we haven't yet determined that the original object was dynamic.

Just to make sure, if it turns out this fails, we can just turn dynamic off and this section of code will just not matter, correct?

I think the deal is all the fns that will construct Dynamic items will be in experimental, but iirc there's no reasonable way to feature flag this stuff.

It does follow that if user's don't construct dynamic items, this match arm won't matter.

Building on what Owen said, nothing currently uses the dynamic type. The functions that will use it will come from the other issues in the epic. I think the patches for those issues will be easier to test.

@jsternberg Curious about your thoughts on this. I don't want to merge this if you have serious doubts. However, this is currently blocking other work, so if there's a specific change I can make to get this PR across the line, I'm happy to make it.

jsternberg

I just need @Marwes to give feedback on the type inference stuff. The type inference code is solid and I don't want it to be hacky so it needs to be solid and backed by the algorithm.

The compiler code is presently incredibly hacky and is already set for a refactor. The way dynamic would be implemented in a newer compiler would just be to have it built in from the ground up as an inherent part of the reflected value which is already how the vm proposal proposes things. So I'm just not really concerned with changes to the compiler to accommodate dynamic.

jsternberg · 2022-09-22T14:58:18Z

libflux/flux-core/src/semantic/nodes.rs

+        if let MonoType::Dynamic(_) = &t {
+            self.typ = t.clone();
+            return Ok(());
+        }
+


I'd probably want to get @Marwes to have some eyes on this. Type inference is a strange area of code and doesn't really work in ways that are procedural at all times so this condition may not be valid.

I, unfortunately, don't have any input on this.

libflux/flux-core/src/semantic/types.rs

scbrickley · 2022-09-26T14:23:16Z

@Marwes could you weigh in on the questions Jonathan raised above?

onelson

I share some of the hesitation re: inference but I hope we will still have the room to make adjustments as we implement the other bits on top of this new type.

My feeling is it'll be much easier to smoke out problems re: inference once we can use Dynamic in acceptance tests.

jsternberg

I think my own personal concerns are fine and this can be merged. I'd still defer to @Marwes's review. If he approves, I think we're good.

scbrickley force-pushed the dynamic-type branch from 0cd6888 to 3280fa1 Compare September 19, 2022 17:45

pierwill mentioned this pull request Sep 19, 2022

Dynamic types pierwill/flux-redex#4

Open

scbrickley force-pushed the dynamic-type branch 3 times, most recently from 7c799aa to a1c883b Compare September 20, 2022 14:30

scbrickley marked this pull request as ready for review September 20, 2022 14:31

scbrickley requested a review from a team as a code owner September 20, 2022 14:31

scbrickley requested review from jsternberg and onelson and removed request for a team September 20, 2022 14:31

scbrickley force-pushed the dynamic-type branch 4 times, most recently from d982098 to 4f608db Compare September 20, 2022 18:12

onelson reviewed Sep 20, 2022

View reviewed changes

scbrickley force-pushed the dynamic-type branch from 6545b5c to e618539 Compare September 21, 2022 18:46

scbrickley requested a review from onelson September 21, 2022 18:47

scbrickley commented Sep 21, 2022

View reviewed changes

jsternberg suggested changes Sep 22, 2022

View reviewed changes

jsternberg requested a review from Marwes September 22, 2022 15:01

scbrickley requested a review from jsternberg September 26, 2022 19:40

scbrickley force-pushed the dynamic-type branch from e311659 to 177dfba Compare September 26, 2022 19:52

scbrickley added 2 commits September 26, 2022 16:01

feat: Add dynamic type

04c39c7

chore: make generate

af90651

scbrickley force-pushed the dynamic-type branch from 177dfba to af90651 Compare September 26, 2022 20:01

onelson approved these changes Sep 27, 2022

View reviewed changes

jsternberg approved these changes Sep 27, 2022

View reviewed changes

Marwes approved these changes Sep 28, 2022

View reviewed changes

scbrickley merged commit a4af06c into master Sep 28, 2022

This was referenced Sep 30, 2022

feat(dynamic): add dynamic.asArray builtin #5239

Merged

Runtime bits for dynamic index and member expressions #5246

Closed

sanderson mentioned this pull request Oct 3, 2022

Document Flux dynamic types influxdata/docs-v2#4510

Closed

2 tasks

onelson mentioned this pull request Oct 3, 2022

feat(dynamic): runtime support for member expressions, remove index support #5258

Merged

5 tasks

jacobmarble deleted the dynamic-type branch January 4, 2024 17:05

Conversation

scbrickley commented Sep 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

scbrickley Sep 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

onelson Sep 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsternberg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

scbrickley commented Sep 26, 2022

Uh oh!

onelson left a comment

Choose a reason for hiding this comment

Uh oh!

jsternberg left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

scbrickley commented Sep 19, 2022 •

edited

Loading

scbrickley Sep 21, 2022 •

edited

Loading

onelson Sep 21, 2022 •

edited

Loading