The Great Rejiggering (1.0.0) #104

LPGhatguy · 2018-05-26T01:32:19Z

Closes #127, closes #13, closes #146, closes #154, closes #158, closes #180, closes #24, closes #47, closes #48, closes #56, closes #58, closes #7, closes #8, closes #84, closes #88, closes #99.

This is an early PR that aims to rewrite almost every single line of code in Roact.

This PR is meant to serve as a synchronization point for a lot of things that I've wanted to refactor and fix about Roact, notably including most of the above issues. It's also a sprint to try to get Roact's test coverage to as close to 100% line and branch coverage as possible.

Missing Existing Features (Done!)

Portals
Refs
Context
Host events

Docs

Rename features
- Primitive components -> Host components
Document bindings
Document fragments
Document setGlobalConfig
Document validateProps
Include "Added in " for new APIs
Add deprecation/rename detail for Roact.update (formerly Roact.reconcile)

New Features

Misc

idiomic

Went through and wrote down some thoughts as I went from top to bottom.

idiomic · 2018-06-05T04:29:36Z

lib/Reconciler.lua

-	if ref == nil then
-		return
+local function DEBUG_warn(...)
+	if DEBUG_LOGS then


This if statement can be moved outside the function so that it isn't constantly evaluated. Once this if statement and the one below are moved out, then they can be combined since their condition is the same. The local DEBUG_warn can be set to an empty function if DEBUG_LOGS == false or to warn if DEBUG_LOGS == true

idiomic · 2018-06-05T04:46:47Z

lib/Type.lua

+
+local Symbol = require(script.Parent.Symbol)
+
+local Type = newproxy(true)


Why do we need the type.is function? Type.of(element) == Type.Element would make it look nice next to the typeof(key) == "string". Perhaps adding "string", "boolean", "function" and what not to Type would also be a good idea. If we have a type enumeration, lets have all the types in there? It would be nice to use Type.of like so: Type.of(myVariable) == Type.String. Using Type.of also allows developers to use types as a key in lookup tables without altering their syntax (otherwise myVariable[Type])

Thinking about it more, Type.is does allow polymorphism without casts though. I wont remove the previous comment since it is good food for thought.

That's a good idea, it would make the typechecks more in line with those using typeof as well.

I don't think I want it to return any types except internal Roact ones, since for those you can just use typeof and be more clear.

idiomic · 2018-06-05T04:58:39Z

lib/Reconciler.lua

-]]
-function Reconciler.unmount(instanceHandle)
-	local element = instanceHandle._element
+local function taskMountNode(details)


Where is details being constructed? I think the checks below should be done when details is constructed, and then the type of the details table can be set to Type.Details to signify that it conforms to the Details type structure, which type can be checked here.

The function that calls the task* functions is the one creating the data, so I think doing the checks here (it's basically a constructor) is probably fine.

idiomic · 2018-06-05T05:07:35Z

lib/Reconciler.lua

+	assert(typeof(parentRbx) == "Instance" or typeof(parentRbx) == "nil")
+	assert(typeof(isTreeRoot) == "boolean")
+
+	return function(tree)


I haven't finished reading the code, I'm just commenting as I'm going. That being said, if this is what a task looks like, I have a recommendation. A task could be a table with two things, a function to call, and the parameters to call it with. This had a couple advantages to a task being a function, all based upon the key difference of reusing a function. First, a function isn't created for every task. Second, the identity of the function doesn't change between tasks of the same type, allowing a system to do comparisons (think: prioritization or running same types of tasks one after another). Third, the task scheduler can see the parameters (I'm not sure how much of an advantage this is, but it is another degree of freedom).

The first iteration of this was actually closer in line to that, where I had a task type as a string and a table of parameters with a giant switch statement.

I think you're right that having function identity would be useful here, but I was a bit weary about allocating another table for every task. Maybe just mutating the input details object to add a fn member would be fine?

For most of the transforms related to data, I think we're going to need a lot more data, especially when it comes to deduplicating renders. That sort of stuff needs to be sorted top-to-bottom, so the generic task queue idea might not survive!

Adding a function would be fine, although I'd feel better about assigning the description a task type. It seems to me to be the problem that types attempt to fix -- to decide which operations are valid for the given data, to allow comparisons between types, and to put a stamp of truth on specific characteristics (has the data it needs, of the right sub-types).

idiomic · 2018-06-05T05:22:52Z

lib/Type.lua

+
+	return typeof(value) == "table" and value[Type] == typeMarker
+end
+


Where is the assigning of types to a table being done? There should be a single definition and a single function that verifies that a table type is valid. Since we have no static checking, we can either have greedy or lazy dynamic type verification. Since verifying types when a value is assigned can do too much checking, I'd recommend only verifying when needed. Greedy dynamic checking is valuable for debugging though since it allows the developer to find exactly when a type was invalidated. Perhaps a flag to switch between the two types? Greedy dynamic checking would have to be done by metatables or necessitate special setters for objects with types.

Also if we are using lazy evaluation with the Type.is function, then the type stored in the table becomes meaningless. The expected type is passed into the Type.is function and the given table may or may not match the type assigned to it. If we use Type.of or greedy evaluation, then the table's type would become meaningful again.

idiomic · 2018-06-05T09:09:47Z

lib/Reconciler.lua

+		taskIndex = 1,
+		tasksRunning = false,
+		connections = {},
+		mounted = true,


Note that this tree truly isn't mounted yet, and if the root node fails to mount, this tree isn't ever truly mounted.

idiomic · 2018-06-05T09:10:40Z

lib/Reconciler.lua

+		tasks = {},
+		taskIndex = 1,
+		tasksRunning = false,
+		connections = {},


What are these connections? I don't think I've seen them used yet.

Oops, that was an oversight; we don't track a connection object with BindToRenderStep, which is what I was anticipating.

idiomic · 2018-06-05T09:16:44Z

lib/Reconciler.lua

+	mount = mountTree,
+	unmount = unmountTree,
+	reconcileTree = reconcileTree
+}


Overall there is a lot of emphasis on task management in this file. While that is good, this file is going to get huge when the rest of the reconciler code is added in. I'd recommend moving the task oriented code into another file. The reconciler should create task types and associate priorities to them, which the task handler... handles.

The other PR I was working on, #102, does something like that with a Scheduler object. Once I get a better grip on exactly how task scheduling is going to work, I'd definitely like to do that.

idiomic · 2018-06-05T09:20:12Z

lib/Reconciler.lua

+return {
+	mount = mountTree,
+	unmount = unmountTree,
+	reconcileTree = reconcileTree


tree is removed in the other two function names, but not reconcileTree. mount takes a node as an argument but for some reason its function is mountTree. unmount actually takes a tree, so I don't completely understand why tree was removed from its name. I'd almost say drop the "tree" in all these function names.

This is a debug API, not entirely concerned with the outward semantics except that I can run example code against it!

idiomic · 2018-06-05T09:23:17Z

lib/Reconciler.lua


-		if not success then
-			local source = element.source or DEFAULT_SOURCE
+	return tree


rather than returning a new "tree" which contains internal state, it may not be a terrible idea to make an association table to turn the root node into the internal tree state. This would also fix the "tree" naming issues mentioned below, as all reconciler functions would just take the root node.

That's how Roact works today, and I think that was a mistake I'd like to rectify. A handle to a disjoint tree should be explicitly handled differently than a component instance!

… setting

… holding mutable render data

I think I still want to move tasks back into being objects.

Only mounting is ported to this system right now, I'll port the others in a moment>'

LPGhatguy · 2018-06-30T00:44:03Z

Current tension in the new reconciler is transitioning from what is currently TaskScheduler to a new idea of something like RenderScheduler, which explicitly organizes tasks and task-related data by tree depth.

This new idea is a lot less generic, but has the nice advantage that tasks at a given depth can be performed in any order, so we don't have to do as much data structure trickery.

An open question is still how to schedule something to happen after a group of tasks without incurring significant overhead. With TaskScheduler, you can just push a functional task into the queue after pushing dependencies, but since render tasks in RenderScheduler won't have a total order and aren't performed in the order that they're pushed, I'm less certain how to perform them.

The extra complexity that React took on with fibers is starting to make a little bit more sense. We could do something similar with coroutines if we wanted to; maybe we should.

…se entirely

setState suspension

… updates are introduced and the loop continues

… and messaging when users misuse setState

…urs several function call across several modules. Improves perf for mounting and updating leaf nodes

* add and test validateProps for new reconciler * add assertCalledWithDeepEqual to spies this is the ugliest name ever for this method, but idk what to call it * use assertCalledWithDeepEqual in validateProps testing * reorder validateProps invocation to after defaultProps + test * reindent validateProps tests * gate validateProps behind GlobalConfig setting

* Replace pcall with xpcall * Introduce xpcall for properties set via bindings

* Add messages to all asserts * Add flag to the test runner. Technically, none of our tests yet rely on the asserts (which is surprising), but we still want hem on for tests * Replace GlobalConfig functionality with something closer to the way we do debug stuff * Replace all DEBUG checks with new GlobalConfig approach * Remove the only-set-once logic from GlobalConfig * Remove public exposure of GlobalConfig's function, initialize config before tests

* Remove references to 'primitive' components/elements * Adjust some wording around kinds of components, fix broken link * functional component -> function component * Avoid usage of the verb 'reconcile' where not directly applicable * Deprecate Roact.reconcile in favor of Roact.update in master (#194) * Update CHANGELOG * Remove undocumented APIs `getGlobalConfigValue` and `Element` (#195) * Roact.reconcile replaced with Roact.update, uses of reconcile changed to update. Occasional mentions of 'the Roact reconciler' have been changed to simply 'Roact' * Add version info to new apis, more details to validateProps, added section for setGlobalConfig * Added the 'enable tracebacks' message to other places that try to use element tracebacks * Change misleading code in Roact.Change example (#196) * PR comments: mention deprecated names, fix links * Rename propertyValidation config key to propValidation, add code snippet for enabling it to docs

…l optimization

* Move virtual tree fields to internal sub-table * Minor code cleanliness feedback

* Test to verify ref.current == ref:getValue(), fix up docs * Update the 'Added in...' clauses in the api reference * update example I guess * A bit of touchup

ZoteTheMighty

1.0 here we come!

LPGhatguy added 6 commits May 25, 2018 18:31

Break everything

5b02e2c

Merge branch 'master' into new-reconciler

db994a1

Add test client and begin prototyping reconciler features

e38d807

Optionally toggle logs

e0cc9ad

Better RenderStepped connection name

154e467

Merge master

724b4b5

LPGhatguy mentioned this pull request Jun 4, 2018

Errors put Roact in a bad state #106

Closed

3 tasks

LPGhatguy added 4 commits June 4, 2018 14:58

Iterate a bit on this idea, introduce concept of Tree and Node

17245a2

Fancy new unmount function

60fa010

Tinker with task system, now always functions instead of tables

c0e1218

Simplify task scheduling

fd8f36c

idiomic reviewed Jun 5, 2018

View reviewed changes

LPGhatguy added 2 commits June 5, 2018 14:09

Fix broken client after key became required

ccac1c4

Move debug check functions in a level; eventually this will be a tree…

3c239e3

… setting

idiomic mentioned this pull request Jun 6, 2018

Remove createElement #109

Closed

LPGhatguy added 11 commits June 12, 2018 11:17

Tinkering work

5542702

Add empty 'renderer' files

a899f15

Node depth, config parameter, typechecking more things

8f89cbf

Document reconciler data structures, add 'scheduledRenders' field for…

900d008

… holding mutable render data

Merge branch 'master' into new-reconciler

6ae6436

Merge latest master

58b16fb

Re-skip Component tests

75fa632

Break out scheduler into TaskScheduler.

264b381

I think I still want to move tasks back into being objects.

Create task object with separation between validation+perform phases.

45dd45f

Only mounting is ported to this system right now, I'll port the others in a moment>'

Port everything over to newer task scheduler

33ea0b1

Fix destruction code for TaskScheduler

81b8653

LPGhatguy added 3 commits June 29, 2018 18:32

Implemented a bucketed priority queue, I'm not sure if this makes sen…

ec41e59

…se entirely

Blow up RenderScheduler

509b9c1

Nonsense, I'm stalled a bit here

a7961da

ZoteTheMighty and others added 21 commits February 27, 2019 14:41

WIP: Setstate suspension (#183)

6e74824

setState suspension

Add some setState suspension edge case tests

20e606e

Add test that makes sure prior state updates are cleared when pending…

22fdbef

… updates are introduced and the loop continues

Add loop guard and error messaging to improve test failure situations…

46fa93b

… and messaging when users misuse setState

Fix previously mentioned bug with infinitely updating state

844d189

Add test to throw when setState recursion limit is reached

8860c5c

Slight perf improvement: avoid reconciling nil children, since it inc…

ea822f6

…urs several function call across several modules. Improves perf for mounting and updating leaf nodes

Add Rotriever manifest

cffb5ea

Use lowercase config file name

b70197b

Xpcall (#187)

1a113eb

* Replace pcall with xpcall * Introduce xpcall for properties set via bindings

Fix error messages from setting invalid props (#191)

7bdc808

Merge branch 'master' into new-reconciler

5722d14

Clean up TODOs and add GH issue links

598b24a

Fix some minor TODOs and remove some outdated ones. Add TODO for smal…

e3166fb

…l optimization

Remove unused require

68880db

Hide tree internals (#201)

4540e6e

* Move virtual tree fields to internal sub-table * Minor code cleanliness feedback

Merge CHANGELOG from master

a587a49

Prime changelog for 1.0.0

383cdce

LPGhatguy changed the title ~~WIP: The Great Rejiggering~~ The Great Rejiggering (1.0.0) May 3, 2019

LPGhatguy and others added 5 commits May 3, 2019 14:17

Merge branch 'master' into new-reconciler

c13e342

Bless ref:getValue(), update version numbers in docs (#202)

8289d15

* Test to verify ref.current == ref:getValue(), fix up docs * Update the 'Added in...' clauses in the api reference * update example I guess * A bit of touchup

Update changelog

229d6a8

Miscellaneous repo housekeeping

520f8e6

restored event suspension (#203)

ebfe578

ZoteTheMighty approved these changes May 3, 2019

View reviewed changes

LPGhatguy merged commit 635f1cd into master May 3, 2019

cliffchapmanrbx deleted the new-reconciler branch June 11, 2020 18:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The Great Rejiggering (1.0.0) #104

The Great Rejiggering (1.0.0) #104

LPGhatguy commented May 26, 2018 •

edited by ZoteTheMighty

idiomic left a comment

idiomic Jun 5, 2018

idiomic Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

idiomic Jun 5, 2018

idiomic Jun 5, 2018

idiomic Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

idiomic Jun 5, 2018

LPGhatguy Jun 5, 2018

LPGhatguy commented Jun 30, 2018

ZoteTheMighty left a comment


		local Symbol = require(script.Parent.Symbol)

		local Type = newproxy(true)


		return typeof(value) == "table" and value[Type] == typeMarker
		end

The Great Rejiggering (1.0.0) #104

The Great Rejiggering (1.0.0) #104

Conversation

LPGhatguy commented May 26, 2018 • edited by ZoteTheMighty

Missing Existing Features (Done!)

Docs

New Features

Misc

idiomic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LPGhatguy commented Jun 30, 2018

ZoteTheMighty left a comment

Choose a reason for hiding this comment

LPGhatguy commented May 26, 2018 •

edited by ZoteTheMighty