[RFC] atom effects

[dai-shi]

Continuing from discussions in the discord channel.
Recoil proposes Atom Effects.
We don't need to provide the same API, and probably it's impossible as there are some conceptual differences between jotai and recoil. (It is interesting to me that things get to be seen differently in jotai from recoil, because originally jotai is modeled after recoil.)

However, there are good use cases. It seems like there are three possibilities for each use case.

1. can be implemented as derived atoms
2. can't be implemented as derived atoms, but something can be added in jotai core
3. can never be implemented in jotai

Collecting use cases might be important. Please share your ideas.

For 1, we could create new functions in jotai/utils: withEffect and atomWithEffect.

Any thoughts? volunteers?

[Aslemammad]

@dai-shi I do my job with derived atoms, or even write-only atoms, these kinds of atoms could listen to other atoms and do things after updating, they are too easy to implement. I'm a volunteer. I'm thinking of option 1.

[Aslemammad]

Here's the withEffect that I'm thinking about:

withEffect(anAtom, onSet: (get,set) => void) 
// IDK if we need the newVal and oldVal like recoil( I'm not big fan of it, 
// because I think withEffect should run after every update of the anAtom)
withEffect(anAtom, onSet: (get, set, {newVal, oldVal})

and if we want to use atomWithEffect we can make a function that creates the atom with the withEffect's onSet.

Considering recoil's Atom effects:

• node: we can reach it by referencing the atom
• trigger: do we actually need It? if we do, can someone explain to me why?
• setSelf: we can achieve it by the set of onSet: (get, set)
• resetSelf: we can do set(anAtom.init)

[dai-shi]

Let me see how we could implement examples in recoil docs.

Logging Example

https://recoiljs.org/docs/guides/atom-effects#logging-example

const baseAtom = atom(null)
const currentUserIdAtom = atom(
  get => get(baseAtom),
  (get, set, arg) => {
    set(baseAtom, arg)
    console.log('Current user ID:', get(baseAtom))
  }
)

Note: The log is shown, when the update is scheduled, not committed.

History Example

https://recoiljs.org/docs/guides/atom-effects#history-example

const history = [];
const baseAtom = atom(null)
const userInfoAtom = atom(
  get => get(baseAtom),
  (get, set, arg) => {
    const oldValue = get(baseAtom)
    set(baseAtom, arg)
    const newValue = get(baseAtom)
    history.push({
      label: `${JSON.serialize(oldValue)} -> ${JSON.serialize(newValue)}`,
      undo: () => {
        set(baseAtom, oldValue);
      },
    })
  }
)

Note: This would only work if baseAtom is not async.

[Aslemammad]

@dai-shi I think jotai is easier than recoil now, that's awesome.

[dai-shi]

(Well, I didn't test them, and the behavior can be different.

[dai-shi]

Continuing #211 (comment)

State Synchronization Example

https://recoiljs.org/docs/guides/atom-effects#state-synchronization-example

const baseAtom = atom(null)
const userInfoAtom = atom(
  get => get(baseAtom) ?? myRemoteStorage.get(userID),
  (get, set, arg) => {
    if (arg === 'initialize') {
      set(baseAtom, get(userInfoAtom))
      myRemoteStorage.onChange(userID, userInfo => {
        set(baseAtom, userInfo);
      })
    } else if (arg === 'cleanup') {
      myRemoteStorage.onChange(userID, null);
    }
  }
)

  // in component
  const [userInfo, dispatch] = useAtom(userInfoAtom)
  useEffect(() => {
    dispatch('initialize')
    return () => dispatch('cleanup')
  }, [])

So, we don't technically have atom effects. It's not equivalent.

Write-Through Cache Example

https://recoiljs.org/docs/guides/atom-effects#write-through-cache-example

const baseAtom = atom(null)
const userInfoAtom = atom(
  get => get(baseAtom) ?? myRemoteStorage.get(userID),
  (get, set, arg) => {
    if (arg.type === 'initialize') {
      set(baseAtom, get(userInfoAtom))
      myRemoteStorage.onChange(userID, userInfo => {
        set(baseAtom, userInfo);
      })
    } else if (arg.type === 'cleanup') {
      myRemoteStorage.onChange(userID, null);
    } else if (arg.type === 'set') {
      set(baseAtom, arg.value)
      myRemoteStorage.set(userID, arg.value)
    }
  }
)

  // in component
  const [userInfo, dispatch] = useAtom(userInfoAtom)
  const setUserInfo = value => dispatch({ type: 'set', value })
  useEffect(() => {
    dispatch('initialize')
    return () => dispatch('cleanup')
  }, [])

Local Storage Persistence

https://recoiljs.org/docs/guides/atom-effects#local-storage-persistence

See: https://github.com/pmndrs/jotai/blob/master/docs/persistence.md

Browser URL History Persistence

https://recoiljs.org/docs/guides/atom-effects#browser-url-history-persistence

Your ideas...

[dai-shi]

While I would try to avoid adding a new feature in core, this seems unavoidable.

• We discussed some patterns in Good pattern for updating atoms based on subscription #234, but useEffect based solution only works for non-base atoms
• We started working on feat: jotai/query (powered by react-query) #248, and support for subscription for base atoms seems crucial

---

So, here's the proposal.

import { atom } from 'jotai'

const dataAtom = atom(null)
dataAtom.effects = [
  (get, set) => {
    const unsubscribe = someStore.subscribe((nextData) => {
      set(dataAtom, nextData)
    })
    return unsubscribe
  }
]

// effects are invoked in commit phase, when it first have a dependent
// and will be cleaned up when there are no dependents (even if it's a very short period.)
// `get` can read only from the committed state, wip state is just ignored.

[Aslemammad]

@dai-shi Nice way for effects.

[dai-shi]

@Aslemammad suggested that effects sound like useEffect, and runs every commits. That wasn't really my intention. It's more like useEffect(..., []). I don't see any use cases for running functions every time, because that is the purpose of derived atoms with write function (which is invoked in the commit phase.)

If anyone has an idea about the use case of running functions on every changes, let us know.

Given that there can be misunderstanding with useEffect() and it's different from Recoil's Atom Effects, I would propose to rename it.

---

New proposal.

import { atom } from 'jotai'

const dataAtom = atom(null)
dataAtom.onmount = [
  (get, set) => {
    const unsubscribe = someStore.subscribe((nextData) => {
      set(dataAtom, nextData)
    })
    return unsubscribe
  }
]

// functions in `onmount` are invoked in the commit phase when it first have a dependent,
// and will be cleaned up when there are no dependents (even if it's a very short period.)
// `get` can read only from the committed state, wip state is just ignored.
// for async atoms, `get` will throw a promise in the pending state (tentative: we would change if React changes it.)

I wonder if we want/need to make it a list.

dataAtom.onmount = (get, set) => { ... }

This looks more familiar, doesn't it?

[Aslemammad]

I think it's better now, and I think we can make it .onMount or just .mount. I prefer the first one. And I think yes, I can get things done with derived atoms for use cases like useEffect(), but onMount, is a really good similar to useEffect(,[]).

[dai-shi]

Hm, okay debugLabel is already a camel case.

Here's version 3.

import { atom } from 'jotai'

const dataAtom = atom(null)
dataAtom.onMount = (get, set) => {
  const unsubscribe = someStore.subscribe((nextData) => {
    set(dataAtom, nextData)
  })
  return unsubscribe
}

[dai-shi]

I have been a bit less confident with the proposed api, and now I understand why. There's a pitfall with get reading stale value (as I noted in code comment in #211 (comment)), and it's giving too much power while we already give much power in write.

In short, it was not minimalistic.

Here's the version 4.

import { atom } from 'jotai'

const dataAtom = atom(null)
dataAtom.onMount = (setAtom) => {
  const unsubscribe = someStore.subscribe((nextData) => {
    setAtom(nextData)
  })
  return unsubscribe
}

You may wonder how to update two atom values in one subscription. In this case, use a derived atom.

const textAtom = atom('')
const countAtom = atom(0)
const derivedAtom = atom(
  null,
  (_get, set, nextText) => {
    set(textAtom, nextText)
    set(countAtom, nextText.length)
  }
)
derivedAtom.onMount = (setAtom) => {
  const unsubscribe = someStore.subscribe((nextData) => {
    setAtom(nextData)
  })
  return unsubscribe
}

[dzcpy]

Is it possible to use tools like fast-json-patch (https://github.com/Starcounter-Jack/JSON-Patch) to maintain the history of an atom? So only the difference of each modification is saved. I'm working on a huge object and need to save all modifications in history. Keeping the entire object is a bit too inefficient.
Can anyone shed some light on the best practice? Many thanks!

[dai-shi]

@dzcpy I think if we use two atoms, the history can be implemented. No need for onMount in this case.
The good way to think at first is how you would do it with useState. Then, it's straightforward to migrate to atom and useAtom. I can help on it.
Would you open a new discussion / issue?

[dzcpy]

@dai-shi Thanks for your reply. I'll think about it and open a new issue if I can't solve it

[ChristopherWirtOfficial]

@dai-shi I feel like either I'm not understanding, or most of the responses here are missing the underlying sentiment of effects by suggesting that the answer is to put the interaction into hook-space. If I have jotai base atoms I want to update with results of some derived atom when that derived atom has reactive updates, my understanding is this would currently be done in a hook like:

const myDerivedValue = useAtomValue(MyDerivedAtom);
const [value, setValue] = useAtom(MyBaseAtom);

useEffect(() => {
    setValue(massageValue(myDerivedValue));
}, [myDerivedValue]);

This has the problem of coupling Jotai-exclusive state logic with your React components, which imo kneecaps the library.

I'm admittedly a Jotai beginner compared to my experience with Recoil, so I'm hoping this is just a learning opportunity for me. But if you're doing something some kind of sync between atoms, you now have to choose a point in your component hook-space code that's authoritative over the decision to sync those two atoms, which seems counter to Jotai's philosophy.

[dai-shi]

When we designed .onMount(), we specifically avoid supporting .onUpdate() which can lead wrong usage more than proper usage. It's exactly this case.

You code snippet,

const myDerivedValue = useAtomValue(MyDerivedAtom);
const [value, setValue] = useAtom(MyBaseAtom);

useEffect(() => {
    setValue(massageValue(myDerivedValue));
}, [myDerivedValue]);

is not good practice, not even with jotai, but with react. It requires two renders to get the final value and users would see intermediate state. Basically, it's delayed.

What's better with jotai is you define a writable derived atom and update two atoms at the same time. This doesn't require hook-spece solution and no delay.

[ChristopherWirtOfficial]

That example was arbitrary, and my underlying point is that i wouldn't want to approach the problem with useEffect. I'll try and throw together a better example.

For what it's worth, your response isn't all that helpful for me, or to the question posed. The trivial example i showed can be done differently, but the sentiment of wanting effects that accomplish similar patterns is still useful. The place where i do my writes isn't necessarily the place that makes sense to couple those particular items. A complex derived atom may have several ways of reactively updating, something that makes jotai great. Having to march my knowledge that something in a derived atom has changed up to the place where i make those base changes feels like it's kneecapping.

Again, i may just be misunderstanding here, hopefully this is a more complete summary of the perceived problem I'm having. If I'm truly just approaching Jotai wrong, then that's alright. But based on the core principles of atomic state, my use case feels very intuitive, and somewhat reasonable.

[dai-shi]

I think I misunderstood your question then. I thought it's related to [RFC] atom effects. Please open a new discussion with smallest possible concrete example.

[ShravanSunder]

@dai-shi What were your reservations about onSet which would run on every commit? Its something recoil does, and could be useful for external sychronizations etc.

[dai-shi]

#211 (comment)
I think it will lead to 99% misusage. There are some use cases like external synchronizations with derived atoms, but it should be rare. We should prefer enhancing write to add "onSet" logic.

That doesn't mean I gave it up though. Maybe, we can try something with unstable_ prefix for the very rare cases. Do you have specific use cases in mind?

[ShravanSunder]

@dai-shi Its mostly regards synchronization with an external store or queue.

• For example having a indexdb that is being batch written to. Kinda of like a write through cache. So when the application loads onMount, data is available again. After which its just write through with the onSet.
• The other idea is having a transaction queue. Changes to atoms send a transaction to the queue. The transaction is sent to the background (for a slow backend or process).

[dai-shi]

@ShravanSunder Let's first see if those use cases can be covered by a writable atom, which is preferable and more importantly, better in performance.

[rileyjshaw]

@dai-shi do you have a recommendation for how to call an effect outside of React based on derived state? For instance:

const postAtom = atom({ author: 'rileyjshaw', topic: 'Jotai' });
const authorAtom = selectAtom(postAtom, post => post.author);
// Call an effect whenever the value of `authorAtom` changes.

I tried the following, but unless effectAtom is mounted in React, the effect doesn’t run:

// Does not work, and feels a bit messy.
const effectAtom = atom(get => {
  const author = get(authorAtom);
  console.log(author); // <- Desired effect.
});

I realize that in this small example I could add an effect into authorAtom. But my question is whether it’s possible to call an effect from an unmounted atom, or some sort of atom listener.

In other words

All of your examples above use a baseAtom which is not directly consumed by React components. All of the effects occur in the directly consumed atom. Is there a way to flip this, so that the effect function lives independently of React? Thanks!

---

Edit 1:

I just re-read some of the above posts, and it looks like this feature was intentionally excluded from Jotai.

When we designed .onMount(), we specifically avoid supporting .onUpdate() which can lead wrong usage more than proper usage.

I’m curious about how you would structure the following requirement in a writable atom, as you suggest here: #211 (comment).

Let’s say I have a bunch of users which are dynamically loaded. I can also change my own username, which is a nested property under the users map. I want to persist my name to localStorage:

const myUserId = '123';
// Contents will eventually be overwritten.
const usersAtom = atom({
  [myUserId]: localStorage.getItem('name') ?? '',
});
const myNameAtom = focusAtom(usersAtom, optic => optic.prop(myUserId).prop('name'));

function App () {
  const setUsers = useSetAtom(usersAtom);
  const [myName, setMyName] = useAtom(myNameAtom);

  // Overwrite the contents of usersAtom every 10 seconds.
  useEffect(async () => {
    const intervalId = setInterval(() => {
      const users = await fetch('/users');
      setUsers(JSON.parse(users));
    }, 10000);
    return () => clearInterval(intervalId);
  }, []);

  return <input value={myName} onChange={e => setMyName(e.target.value)} />
} 

Since myNameAtom is derived, I don’t think useAtomWithStorage etc. will be useful here. I know of two solutions:

1. Add another derived atom, and include it in the render function just to run its effects:

const myNameStorageEffectAtom = atom(get => {
  const name = get(myNameAtom);
  localStorage.setItem('name', name);
  return null;
});

function App() {
  useAtomValue(myNameStorageEffectAtom);
  // ...
}

2. Add a React effect:

function App() {
  const [myName, setMyName] = useAtom(myNameAtom);
  useEffect(() => {
    localStorage.setItem('name', myName);
  }, [myName]);
  // ...
}

Both solutions add an undesired render dependency, and couple the side-effect to React. I hope there’s a third solution that I don’t know about, like so:

3. Listen to changes on the atom from outside of React:

// Just an example… I know this doesn’t exist:
myNameAtom.onUpdate(newName => localStorage.setItem('name', newName));

---

Edit 2:

I just found #750, and I get the feeling that this is a use case you’re not interested in supporting. Fair enough!

Both issues are a bit old, so if you’re open to discussing this further I think it would be a really useful feature. I like the minimal surface area approach that @ahfarmer outlined in #750, specifically:

import { effect } from "jotai";

const destroy = effect((get, set) => {
  // automatically re-runs whenever one of my get() values changes 
  // this is similar to an atom, except that it has set(), and does not have its own value
});

.onUpdate would work as well, and might match the existing .onMount format better.

---

Sorry for the long issue, and thanks for all of the work that you do on Jotai, Zustand, Valtio, etc! Looking forward to hearing your thoughts when you get a chance, but no rush of course.

[dai-shi]

@rileyjshaw
Yes, it's not a technical limitation, but an API design to avoid misusage.
I do sometimes want .onUpdate when I create some complex utils for deriving atoms.
It would be better to open a new discussion instead of discussing in this old issue. Feel free to open one, but please don't expect anything will happen soon. I've been holding this idea for more than a year.

Meanwhile, v2 API has store API, so you can do this:

myStore.sub(myNameAtom, () => {
  const maybeNewValue = myStore.get(myNameAtom);
  localStorage.setItem('name', maybeNewName));
});

It can also lead to misusage, but at least there's something you can do, that wasn't possible previously.

[rileyjshaw]

Thanks for responding so quickly @dai-shi! And sorry for commenting on an old issue. I won’t open a new discussion unless you think it’s important; I trust that you’ve got all the context you need and are keeping it in mind.

The snippet that you posted from v2 appears close to what I was looking for, so I’ll start looking into v2’s store implementation. I think the clarity that I love so much about Jotai’s minimalism / separated concerns might just not extend to external effects. Whether I’m writing a null returning component, or an extra store, it feels like it’s adding some unrelated conceptual overhead vs. @ahfarmer’s proposal. But as you said, you’ve been holding this idea for over a year and I trust your vision 🙂

Thanks again for all your work!! I just signed up as a monthly sponsor.

[dmaskasky]

@rileyjshaw

Playing around with existing Jotai api. I absolutely love this library and other work from @dai-shi ❤️.

Here's what I came up with for atomEffect. Please note, like @dai-shi mentioned, atomEffect api introduces some pretty ugly design patterns and should be avoided when possible.

import { Atom, atom, Getter, Setter } from "jotai";

/**
 * creates an effect atom which watches atoms with get and
 * synchronously updates atoms with set.
 */
export function atomEffect(effectFn: Effector) {
  const watchedValuesMapAtom = atom(new Map<Atom<any>, any>());

  const cleanupAtom = atom<CleanupFn | null>(null);

  const isFirstRun = (get: Getter) => {
    const watchedValuesMap = get(watchedValuesMapAtom);
    return watchedValuesMap.size === 0;
  };

  const someValuesChanged = (get: Getter) => {
    const watchedValuesMap = get(watchedValuesMapAtom);
    return Array.from(watchedValuesMap.keys()).some(
      (anAtom) => !Object.is(get(anAtom), watchedValuesMap.get(anAtom))
    );
  };

  const updateWatched = (get: Getter) => {
    const watchedValuesMap = get(watchedValuesMapAtom);
    Array.from(watchedValuesMap.keys()).forEach((anAtom) => {
      watchedValuesMap.set(anAtom, get(anAtom));
    });
  };

  const getFn = (get: Getter) => {
    const watchedValuesMap = get(watchedValuesMapAtom);
    const getter = <T>(anAtom: Atom<T>): T => {
      const currentValue = get(anAtom);
      watchedValuesMap.set(anAtom, currentValue);
      return currentValue;
    };
    return getter;
  };

  const getSetCollector: GetSetCollector = (get) => (set) => {
    if (!isFirstRun(get) && !someValuesChanged(get)) {
      return;
    }
    updateWatched(get);
    get(cleanupAtom)?.();
    const cleanup = effectFn(getFn(get), set);
    set(cleanupAtom, () => cleanup);
  };

  return atom((get) => {
    const injectEffect = get(effectAtom);
    if (typeof injectEffect === "function") {
      injectEffect(getSetCollector(get));
    }
  });
}

type Effector = (get: Getter, set: Setter) => void | CleanupFn;
type CleanupFn = () => void;
type GetSetCollector = (get: Getter) => SetCollector;
type SetCollector = (set: Setter) => void;
type InjectEffect = (setCollector: SetCollector) => void;

function createSetInjector(set: Setter) {
  return (setCollector: SetCollector) => {
    setCollector(set);
  };
}

const effectAtom = atom(
  null as null | InjectEffect,
  (get, set: Setter, _?: InjectEffect | void) => {
    const value = get(effectAtom);
    if (typeof value === "function") return;
    const setInjector = createSetInjector(set);
    set(effectAtom, setInjector);
    effectAtom.init = setInjector;
  }
);
effectAtom.onMount = (setAtom) => {
  setAtom();
};

Usage

const updateComponentWhenCodeChangesAtom = atomEffect((get, set) => {
  const code = get(codeAtom); // watched
  set(updateComponentAtom, code); // applied
  return () => {
    // cleanup
  }
});

Register

atomEffects must be registered somewhere to work and can be registered multiple times without any duplicate behavior issues.

Register effect in the read function of another atom

const componentAtom = atom(get => {
  get(updateComponentWhenCodeChangesAtom);
  get(_componentAtom);
});

Register effect with a read hook

useAtom(updateComponentWhenCodeChangesAtom)

Notes

1. Because we don't have access to Jotai internals, we are forced to keep watched atom references in a map without any way to garbage collect them. This will result in a memory leak.
2. Not sure about whether this solution would be concurrent mode safe.
3. atomEffect opens the door for another utility atomWithLazy 😊 (see below)

AtomWithLazy

This pattern can be useful when dealing with atoms whose values require an expensive computation. lazyEvaluationFn is invoked on first read access of the atom and synchronously updates the atom's value.

import { atom, Getter, PrimitiveAtom } from "jotai";
import { atomEffect } from "./atomEffect";

export { atomWithLazy };

function atomWithLazy<T>(
  lazyEvaluationFn: (get: Getter) => T
): PrimitiveAtom<T | null>;

function atomWithLazy<T, I>(
  lazyEvaluationFn: (get: Getter) => T,
  initialValue: I
): PrimitiveAtom<T | I>;

function atomWithLazy<T, I = null>(
  lazyEvaluationFn: (get: Getter) => T,
  initialValue: I | null = null
) {
  const valueAtom = atom(initialValue as I | T);
  const firstRunAtom = atom(true);
  const effectAtom = atomEffect((get, set) => {
    const firstRun = get(firstRunAtom);
    if (!firstRun) return;
    set(firstRunAtom, false);
    const result = lazyEvaluationFn(get);
    lazyAtom.init = result;
    set(valueAtom, result);
  });
  const lazyAtom = atom(
    (get) => {
      get(effectAtom);
      return get(valueAtom);
    },
    (get, set, value: I | T) => {
      set(valueAtom, value);
    }
  ) as typeof valueAtom;
  lazyAtom.init = initialValue as I;
  return lazyAtom;
}

Usage

const globals = { React };
const codeMap = lessons.map(({ code }) => atom(code));
const initialValue: ComponentEntry = { Component: () => null, code: '' };

const componentAtom = atomWithLazy(
  (get) => {
    try {
      const effectAtom = atomEffect((get, set) => {         // <---- a lazily instantiated atomEffect 😁
        const codeAtom = codeMap[lessonId];
        const code = get(codeAtom);
        set(componentAtom, code);
      });
      get(effectAtom);
      const code = get(codeMap[lessonId]);
      const Component = evalutateCode(code, globals) as React.FC;
      return { Component, code } as ComponentEntry;
    } catch (e) {
      return initialValue;
    }
  },
  initialValue
);

Notes on atomWithLazy

1. lazyEvaluation run's once for the atom, regardless of the number of mount / unmount events.
2. There is a slight visual flicker as the first lazily evaluated atom being lazily loaded returns it's initial value before immediately returning its lazily evaluated value. Although this might be preventable with a DeferredPromise.
3. Since atomEffect is mounted only once, the flicker only happens on the first usage of atomEffect or atomWithLazy.
4. Its possible to nest an atomEffect inside a lazyAtom to lazily register an atomEffect.

[dai-shi]

@dmaskasky

As far as I understand, it looks like a nice hack. The code can probably simplified a little bit further.

Because we don't have access to Jotai internals, we are forced to keep watched atom references in a map without any way to garbage collect them. This will result in a memory leak.

Even Jotai internals don't know all atoms. How about initialize the map on mount and remove it on unmount? Naively, effects don't run for not-mounted atoms though, or we could clean up for each time.
https://github.com/jotaijs/jotai-cache and atomWithObservable code might be of a hint.

Not sure about whether this solution would be concurrent mode safe.

Jotai v1 had a risk, but in v2, we update atom values outside React render cycle if the atom is already mounted. So, the question is if we can somehow deal with not-yet-mounted atoms. atomWithObservable is a real hack on this. It has unstable_timeout to mitigate it, but without it, there can theoretically be memory leaks in concurrent rendering.

If you could solve those two issues, would you be interested in publishing jotai-effect package?

[dai-shi]

One small but important fix:

// this creates a shared map across multiple stores/Providers.
const watchedValuesMapAtom = atom(new Map<Atom<any>, any>());

// instead, do this:
const watchedValuesMapAtom = atom(() => new Map<Atom<any>, any>());

[dmaskasky]

If you could solve those two issues, would you be interested in publishing jotai-effect package?

Yes. I'd love to contribute. 😊

I just have a few questions before I begin:

type GeneralEffector = (get: Getter, set: Setter) => void;
type SpecificEffector<Value> = (get: Getter) => Value;

 // current api
type AtomEffect = (effector: GeneralEffector) => Atom;
type AtomWithLazy<Value> = (effector: SpecificEffector<Value>, initialValue: Value)

1. What are your thoughts on the api as proposed? Personally, I like how atomEffects are their own thing. They can read from and update multiple atoms. But I don't like how they must be registered separately if they only update a single atom. So there might be a case to tie it to a single atom and restrict it to only update a single atom. Other options are:
   a. withAtomEffect(anAtom: Atom, effector: SpecificEffector): Atom
   b. anAtom.effect(effector: SpecificEffector)
   c. atom((get: Getter) => Value, (get: Getter, set: Setter, ...args: Args) => Result, effector: SpecificEffector) 👎
2. Do prefer like atomEffect or atomWithEffect?
3. Do you think it makes sense to put atomWithLazy in jotai-effect or should belong in its own package?
4. Are there any other applications similar to atomWithLazy that could go in this package?

[dai-shi]

1,2: My preference would be atomWithEffect, but I don't understand the capability fully, so I'd defer the final decision. How compatible would it be with Recoil one?
3: Haven't looked into atomWithLazy yet, but it can be in jotai-effect.
4: No idea at the moment.

Please focus on the effect one first and once we agree on it, we can move on for others.

I wonder if we should set in sync. I guess effect should/could run in async. And, that reminds me, we have an internal/undocumented feature called setSelf. If we could use it, it's gonna be super clean.

[dmaskasky]

Some thoughts

Thinking on this more,

I found this thread because the problem I was trying to solve was that I needed to update an atom when another atom (atomWithStorage) was initialized / changed. I think this is a problem worthy of a solution.

But it might better to restrict the api a bit. While atomEffect could update multiple atoms, I'm not yet convinced they should. However, one could achieve the same effect by writing to a writable atom which can update multiple atoms.

atomEffect is basically a derived read-only atom with a set parameter in the getter. I think the main function of an atomEffect is to perform side-effects when data changes. I don't think we should call it atomWithEffect since an atomEffect's return value is not the focus of the utility. I might even type it's effector to enforce returning void.

Experimentation

Below is some experimentation along with some thoughts.

// atomEffect doesn't do anything until it is mounted somewhere
// but where should it be mounted? Inside `gtAtom` or 'lteAtom` or both?  😕
// Will this be confusing?
atomEffect((get, set) => {
  const value = get(valueAtom);
  if (value > 0)
    set(gtAtom, expensiveGTComputation(value));
  else
    set(lteAtom, expensiveLTEComputation(value));
});
  ...
// what if gtAtom is unmounted?
// Now, lteAtom also stops getting updated, hmm...
const gtAtom = atom(get => {
  get(atomEffect);
  return get(_gtAtom);
});
  ...
const result = useAtomValue(atomEffect);
// when would its value ever be important / useful?
console.log(result); // undefined;  🙁

const actionAtom = atom(null, (get, set) => ({
  increment: () => {
    set(countAtom, count => count + 1);
  },
  decrement: () => {
    set(countAtom, count => count - 1);
  }
}));

const actionHackAtom = atomEffect((get, set) => {
  return set(actionAtom);
});
  ...
// neat, but probably outside the patterns we would want to support
// note, that this component will update when any values watched by
// the atomEffect changes. In this case, none, but it can be difficult to tell,
// and feels like a misuse of useAtomValue.
const { increment, decrement) = useAtomValue(actionHackAtom);

withAtomEffect might be better suited for updating a single atom's value when other atoms change.

// it might be nice to associate atom effects to any existing atom
const storageAtom = atomWithStorage('key', value);
const storageAtomWithEffect = withAtomEffect(storageAtom, get => {
  if (get(countAtom) > 0) return `get(idAtom)-${count}`;
  return get(storageAtom);
});

// this fixes the problem of where to mount atomEffects
// not sure on the use case, but withAtomEffect can be chained, but would be order-dependent
// if chained, each run should probably update the value of the host atom before the next effect runs.
// I think this is how Jotai currently behaves when atoms are set and later read in an atom setter.

Recap

atomEffect:
Pros

1. can be used for state synchonization and side-effects <-- this is huge
2. can set multiple atoms at once.
3. similar to how useEffect can set multiple states at once.
4. simpler than setting a writable atom to work around the one-atom-write-per-effect policy
5. mounting in a derived atom couples the effect to the mount of that atom.

Cons

1. atomEffects need to be mounted somewhere. It can be unclear where to mount them as their mount point couples the atomEffect to the lifecycle of the mounting component, despite the atomEffect potentially setting multiple atoms.
2. the value of an atomEffect does not have any obvious meaning.
3. atomEffects have the potential to introduce some weird patterns such as setStates being returned from useAtomValue.
4. atomEffects have too much power. Then can get and set any atom in the Getter fn, which is supposed to be pure. This could enable some bad coding patterns.

withAtomEffect
Pros

1. bound to a single atom. #single-responsibility
2. eliminates confusion with where to mount
3. although slightly more difficult, updating multiple atoms at once is still supported with a writable atom. The extra friction is useful for guiding users to better code practices. However, the writable atom can also be seen as a single-responsibiliity action atom and therefore considered an acceptable pattern.
4. the value of the effector function is no longer worthless since its now used to update the atom to which its bound.
5. can be associated to any atom
6. simpler api

Cons

1. chaining withAtomEffect could be difficult to follow.
2. not ideal for state synchronization (outside Jotai) or side-effects (ie. setInterval)

Conclusion

While atomEffect does have its drawbacks, I can't ignore its utility for state synchronization and side-effects. atomWithLazy would not have been possible without atomEffect.

However, for updating a single atom's value, I feel withAtomEffect is a much more sensible api.

[dai-shi]

In general, changing another atom when some atom changes is a bad practice as it's not "pure" or "declarative". So, it's technically a side effect. I would highly recommend to avoid such usage.

However, in an edge case, it's useful and for exactly the case with atomWithStorage. (btw, I think we could improve atomWithStorage, or develop a new util specific for storage.)

I feel withAtomEffect is a much more sensible api.

Yeah, that looks cool.

An interesting bit:

const fooAtom = atom(
  (get, { setSelf }) => {
    const value = get(barAtom) * 2;
    Promise.resolve().then(() => {
      setSelf(value);
    });
    return value;
  },
  (get, set, value) => {
    set(bazAtom, value + 1);
  }
);

[dmaskasky]

idk, to me the code below feels more imperative and less ergonomic for some reason.

const countAtom = atom(0);
const cleanupAtom = atom(null);
const countEffectAtom = atom(
  (get, { setSelf }) => {
    get(watchedAtom);
    Promise.resolve().then(setSelf);
  },
  (get, set) => {
    const cleanup = get(cleanupAtom);
    cleanup?.();
    const intervalId = setInterval(() => {
      set(countAtom, count => count + 1)
    }, 1000);
    const cleanupFn = () => clearInterval(intervalId);
    set(cleanupAtom, () => cleanupFn);
  }
);

const countWithEffect = atom((get) => {
  get(countEffectAtom); // register the effect
  return get(countAtom);
}, (get, set, value) => {
  set(countAtom, value);
});

I get that useEffect is the recommended approach today for side-effects. But doesn't it feel weird to bind a global state model to the component's (not the atom's) lifecycle? I would use onMount, but we can't watch atom values in the onMount function.

withAtomEffect v2

To make withAtomEffect able to support side-effects, we can modify the api a bit. As with useEffect, the effect executes after all next state for the Jotai atom graph has evaluated. This should let us make to call setSelf synchronously in the effector as the entire effector function would be async.

const countAtom = atom(0);
const countWithEffect: Atom = withAtomEffect(countAtom, (get, { setSelf }) => {
  get(watched);
  const intervalId = setTimeout(() => {
    setSelf(count => count + 1);
  }, []);
  return () => {
    clearTimeout(intervalId);
  };
});

Open questions.

1. setSelf is synchronous in the effector, but would still be asynchronous in the standard Jotai api. Is this a concern?
2. should an effect have an abort controller if it fires after current state has been evaluated? If not, should setSelf still live in an object?

[dai-shi]

I mean using setSelf is non-declarative, and not recommended in general. I think it should be an API for third-party library authors. (not for app developers.)

setSelf is synchronous in the effector, but would still be asynchronous in the standard Jotai api. Is this a concern?

setSelf can't be called in sync by design. If something expects to run in sync, it doesn't work as expected.

should an effect have an abort controller if it fires after current state has been evaluated? If not, should setSelf still live in an object?

what does "live in an object" mean? btw, we have { signal } for atom with promises.

[dmaskasky]

Since setSelf is commonly used by developers, I moved it out of options and moved options to the third arg. Hopefully, this function signature is ok.

I don't see any way to make get work correctly when the effector is run with Promise.resolve().then. Due to Jotai internals, I can make setSelf run next in the event loop, but not the entire effector.

type SetSelf<Args, Result> = (...args: Args) => Promise<Result>;

type CleanupFn = () => Promise<void> | void;

type PromiseOrValue<Value> = Value | Promise<Value>;

type Effector<Args, Result> = (
  get: Getter,
  setSelf: SetSelf<Args, Result>
) => PromiseOrValue<CleanupFn> | PromiseOrValue<void>

type AtomWithEffect<Value, Args, Result> = WritableAtom<Value, Args, Result>;

type WithAtomEffect<Value, Args, Result> = 
  (anAtom: Atom<Value>, effector: Effector<Args, Result>) => AtomWithEffect<Value, Args, Result>

[dai-shi]

Hm, I guess setSelf won't be of help in this case. Let's start with withAtomEffect use case. I'll open up a new discussion to ideate.

[dmaskasky]

All,

This thread taught me that there are more folks out there who would be interested in an atomEffect utility, and gave me the inspiration and motivation to build it.

You may find the docs here: https://jotai.org/docs/integrations/effect

Thank you