RecipeMetadata and prefetching optimizations #151

interim17 · 2025-11-11T03:36:18Z

Problem

Changes discussed with @meganrm for state management and some optimizations to prefetching of input options and recipe data.

Basically part 2 of #150 and should address the TODOs I added there.

Solution

Instead of RecipeManifest or RecipeManifest/RecipeData ===> we use:RecipeMetadata and RecipeData

RecipeMetadata is whatever we can get on our first trip to server when we call getAllDocsFromCollection.
That means no actual recipe and no unpacked editableFields.

Regardless how we would like to shape this data up front, it makes sense to differentiate these data since they load at very different rates.

Splitting the firebase utils into a batch meta data loader, and a per-ID utility that gets the details (it's slower), makes loading the page smoother.

We can also get rid of isLoading state for now and derive it from the presence of either the input options, or a given recipe object.

Default result URLs are in the metadata, but I think we should prevent loading them beacuse it looks weird to have the viewer populate before the recipe panel does, so I throttled them to go together.

…hub.com/mesoscope/cellpack-client into feature/ssot-refactor

…ching

github-actions · 2025-11-11T03:36:37Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	20.23%	367 / 1814
🔵	Statements	20.23%	367 / 1814
🔵	Functions	40%	24 / 60
🔵	Branches	71.05%	81 / 114

File Coverage

File	Stmts	% Branch	% Funcs	% Lines	Uncovered Lines
Changed Files
src/components/PackingInput/index.tsx	0%	0%	0%	0%	1-2, 12-16, 26-30, 32-35, 37-40, 43-45, 47-49, 51, 54-56, 58-68, 70-71, 73-80, 82, 84, 86
src/state/store.ts	0%	0%	0%	0%	1, 51, 53-59, 61-63, 65, 67-70, 72-86, 88-89, 91-92, 94-96, 99-101, 104, 107-114, 116-118, 120-122, 124-127, 129-137, 139-150, 152-163, 165-167, 169, 171-176, 178-187, 189-192, 195-204, 207-213, 215-216, 218-226, 228-245, 247-263, 266-273, 275-280, 282-287, 289-293, 295-299, 301-302, 306-308, 310-313, 315-318, 320-323, 325-328, 330-341, 344-359
src/types/index.ts	100%	100%	100%	100%
src/utils/firebase.ts	25.98%	62.5%	25%	25.98%	35-36, 41-42, 76-78, 81-91, 94-98, 101-109, 113-126, 129-134, 137-139, 142-166, 174-184, 186-197, 204-216, 230-231, 234-244, 246-255, 257, 259-263, 265-270

Generated in workflow #169

github-actions · 2025-11-11T03:36:55Z

PR Preview Action v1.6.2
Preview removed because the pull request was closed.
2025-11-13 20:05 UTC

interim17 · 2025-11-11T03:37:38Z

src/components/PackingInput/index.tsx

+            {!recipeObj ? (
+                loadingText
+            ) : (
+                <Tabs defaultActiveKey="1" className="recipe-content">
+                    <Tabs.TabPane tab="Edit" key="1">
+                        <RecipeForm onStartPacking={handleStartPacking} />
+                    </Tabs.TabPane>
+                    <Tabs.TabPane tab="Full Recipe" key="2">
+                        <JSONViewer title="Recipe" content={recipeObj} />
+                    </Tabs.TabPane>
+                </Tabs>
+            )}


There isn't a good reason to prevent this panel from loading once the first recipe has been fetched, but I figured showing the loading marker was in line with our overall design pattern, and it should only be visible very briefly if the user selects a recipe right after the dropdown populates.

interim17 · 2025-11-11T03:38:28Z

src/state/store.ts

+            const { inputOptions, loadRecipe } = get();
+
+            const optionList = Object.values(inputOptions || {});
+            if (optionList.length === 0) return;
+
+            const recipeIds = optionList
+                .map(o => o?.recipeId)
+                .filter(id => id && !get().recipes[id]);
+
+            // Make sure our default initial is in the options we queried
+            const initialIdToLoad =
+                recipeIds.includes(INITIAL_RECIPE_ID) ? INITIAL_RECIPE_ID : recipeIds[0];
+
+            // Ensure the bootstrap recipe is loaded & selected
+            if (!get().recipes[initialIdToLoad]) {
+                await loadRecipe(initialIdToLoad);
            }
-            get().selectRecipe(recipeToLoad);
+
+            // Load remaining recipes in the background (don’t block)
+            const remainingRecipesToLoad = recipeIds.filter(
+                id => id !== initialIdToLoad && !get().recipes[id]
+            );
+            await Promise.all(remainingRecipesToLoad.map((id) => loadRecipe(id)));


Sorry for churn here, but I do think this is better.

Hopefully reads well and it loads the initial recipe before moving on to the rest.

Copilot

Pull Request Overview

This PR refactors the recipe data loading architecture by splitting RecipeManifest into two distinct types: RecipeMetadata (fast-loading fields available immediately from the collection query) and RecipeData (slower-loading fields like recipe content and editable fields). The refactoring optimizes page load performance by fetching metadata in bulk while deferring individual recipe details until needed.

Key changes:

Split data types to differentiate between quickly-available metadata and slower recipe details
Removed isLoading state in favor of deriving loading status from data presence
Implemented throttling to prevent result viewer from loading before recipe panel
Updated bootstrap logic to load initial recipe first, then remaining recipes in background

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/utils/firebase.ts	Refactored data fetching functions to separate metadata batch loading from individual recipe data loading
src/types/index.ts	Split `RecipeManifest` into `RecipeMetadata` and reorganized `RecipeData` interface
src/state/store.ts	Removed `isLoading` state, updated selectors and loading logic to work with new data types
src/components/PackingInput/index.tsx	Replaced `isLoading` check with derived loading state based on data presence
src/components/Dropdown/index.tsx	Updated type reference from `RecipeManifest` to `RecipeMetadata`

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/types/index.ts

src/utils/firebase.ts

src/state/store.ts

…cellpack-client into fix/prefetch

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…nto feature/ssot-refactor

ascibisz

I like the distinction you came up with of RecipeMetadata and RecipeData. Everything looks good and works well for me!

meganrm · 2025-11-12T23:01:21Z

src/components/Dropdown/index.tsx

    placeholder: string;
    defaultValue?: string;
-    options: Dictionary<RecipeManifest>;
+    options: Dictionary<RecipeMetadata>;


personally I still like the term manifest but I'm not super attached to it. it's what we use in other apps like CFE for this same type of data

Oh sure I like manifest too, for some reason I thought you said at our 1:1 that you wanted it called metadata.

oh, I think I was just using that as a category term, sorry!

meganrm · 2025-11-12T23:06:21Z

src/types/index.ts

    recipeId: string;
-    configId: string;
-    displayName: string;
+    defaultRecipeData: ViewableRecipe;


since you're calling this whole object RecipeData, having defaultRecipeData inside it is a little unclear. I think I would just call this defaultRecipe. or originalRecipe (and then the edits are spicy 😅🍗)

…nto feature/ssot-refactor

…cellpack-client into fix/prefetch

…nto fix/prefetch

Copilot

Pull Request Overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-13T19:46:51Z

src/utils/firebase.ts

+ */
+const getRecipeDataFromFirebase = async (recipeId: string, editableFieldIds: string[]): Promise<RecipeData> => {
+    const defaultRecipe = await getFirebaseRecipe(recipeId);
+    const editableFields = await getEditableFieldsList(editableFieldIds) || [];


The || [] fallback is unnecessary because getEditableFieldsList already returns undefined when the input array is empty, and the function signature is Promise<EditableField[] | undefined>. The || [] will not catch the undefined case. Instead, handle the undefined explicitly: const editableFields = await getEditableFieldsList(editableFieldIds); if (!editableFields) return ...; or change the return type expectation.

Suggested change

const editableFields = await getEditableFieldsList(editableFieldIds) || [];

const editableFields = await getEditableFieldsList(editableFieldIds);

if (!editableFields) {

return {

recipeId,

defaultRecipe,

editableFields: [],

edits: {}

}

}

Copilot · 2025-11-13T19:46:52Z

src/state/store.ts

+            const editedValue = lodashGet(rec.edits, path);
+            if (editedValue !== undefined) {
+                if (typeof editedValue === "string" || typeof editedValue === "number") {
+                    return editedValue;
+                }
                return undefined;
            }


Using lodashGet on rec.edits is incorrect because edits is a flat Record<string, string | number> where keys are paths, not a nested object. The path should be used directly as a key: const editedValue = rec.edits[path];

…nto fix/prefetch

Copilot

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/types/index.ts

src/state/store.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

interim17 added 4 commits November 10, 2025 11:06

use object instead of string as single source of truth

e07c9c6

Merge branch 'feature/show-precomputed-results-tweaks' of https://git…

2decf90

…hub.com/mesoscope/cellpack-client into feature/ssot-refactor

use displayName

3a02c07

split manifest into RecipeMetadata and RecipeData and refactor prefet…

0c886b5

…ching

interim17 commented Nov 11, 2025

View reviewed changes

interim17 mentioned this pull request Nov 11, 2025

Feature/ssot refactor #150

Merged

interim17 requested a review from Copilot November 11, 2025 03:41

Copilot AI reviewed Nov 11, 2025

View reviewed changes

src/types/index.ts Outdated Show resolved Hide resolved

src/utils/firebase.ts Outdated Show resolved Hide resolved

src/state/store.ts Outdated Show resolved Hide resolved

src/state/store.ts Outdated Show resolved Hide resolved

src/state/store.ts Outdated Show resolved Hide resolved

interim17 added 2 commits November 10, 2025 19:51

add second call of editRecipe for gradient value

ed152a6

Merge branch 'feature/ssot-refactor' of https://github.com/mesoscope/…

adbd622

…cellpack-client into fix/prefetch

interim17 changed the title ~~split manifest into RecipeMetadata and RecipeData and refactor prefet…~~ RecipeMetadata and prefetching optimizations Nov 11, 2025

interim17 and others added 4 commits November 10, 2025 19:55

Apply suggestion from @Copilot

df1c6df

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

fc79530

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

22a5672

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestions from code review

5d3a689

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

interim17 marked this pull request as ready for review November 11, 2025 04:01

interim17 requested review from ascibisz, meganrm and rugeli November 11, 2025 04:01

interim17 added 2 commits November 11, 2025 11:51

move building current recipe object from util into selector

9de4abf

Merge branch 'main' of https://github.com/mesoscope/cellpack-client i…

656b14a

…nto feature/ssot-refactor

ascibisz approved these changes Nov 12, 2025

View reviewed changes

interim17 added 4 commits November 12, 2025 09:31

revert to using utility to build recipe objects

faf266d

Merge branch 'feature/ssot-refactor' into fix/prefetch

9fd8319

fix merge artifact

fb23431

add documenting comment to buildRecipeObject

8f5f11a

meganrm reviewed Nov 12, 2025

View reviewed changes

interim17 added 7 commits November 12, 2025 19:46

Merge branch 'main' of https://github.com/mesoscope/cellpack-client i…

98e37c5

…nto feature/ssot-refactor

rename and adjust typing of recipe object utility

f6f6dca

wrap loading text in container to position it

d5e6f1c

RecipeMetadata -> RecipeManifest

725b614

defaultRecipeData -> defaultRecipe

fe4a840

Merge branch 'feature/ssot-refactor' of https://github.com/mesoscope/…

2e789a4

…cellpack-client into fix/prefetch

fix defaultRecipe naming

f23c4fa

meganrm approved these changes Nov 13, 2025

View reviewed changes

Merge branch 'main' of https://github.com/AllenCell/cellpack-client i…

3442ed7

…nto fix/prefetch

Base automatically changed from feature/ssot-refactor to main November 13, 2025 19:37

meganrm requested a review from Copilot November 13, 2025 19:45

Copilot AI reviewed Nov 13, 2025

View reviewed changes

Merge branch 'main' of https://github.com/AllenCell/cellpack-client i…

273c4e1

…nto fix/prefetch

meganrm requested a review from Copilot November 13, 2025 19:54

Copilot AI reviewed Nov 13, 2025

View reviewed changes

src/types/index.ts Outdated Show resolved Hide resolved

src/state/store.ts Outdated Show resolved Hide resolved

meganrm reviewed Nov 13, 2025

View reviewed changes

src/state/store.ts Outdated Show resolved Hide resolved

rugeli approved these changes Nov 13, 2025

View reviewed changes

meganrm and others added 3 commits November 13, 2025 12:00

dont export locally used hooks

b832ba2

remove error from merge conflict

efa33c5

Update src/state/store.ts

73b06cb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

meganrm merged commit 61ae362 into main Nov 13, 2025
2 checks passed

meganrm deleted the fix/prefetch branch November 13, 2025 20:04

-    const editableFields = await getEditableFieldsList(editableFieldIds) || [];
+    const editableFields = await getEditableFieldsList(editableFieldIds);
+    if (!editableFields) {
+        return {
+            recipeId,
+            defaultRecipe,
+            editableFields: [],
+            edits: {}
+        }
+    }

RecipeMetadata and prefetching optimizations #151

RecipeMetadata and prefetching optimizations #151

Uh oh!

Conversation

interim17 commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Uh oh!

github-actions bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report

Uh oh!

github-actions bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

interim17 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

interim17 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ascibisz left a comment

Choose a reason for hiding this comment

Uh oh!

meganrm Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

interim17 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

meganrm Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

meganrm Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

interim17 commented Nov 11, 2025 •

edited

Loading

github-actions bot commented Nov 11, 2025 •

edited

Loading

github-actions bot commented Nov 11, 2025 •

edited

Loading