Manifest V3

[mikecann]

Am I correct in thinking that this library will no longer work in Manifest v3 with persistant background pages going away?

Edit:

https://developer.chrome.com/extensions/migrating_to_manifest_v3

Basically background pages are going away being replaced with service workers.

Its not exactly univerally loved as an idea: https://groups.google.com/a/chromium.org/forum/#!topic/chromium-extensions/0Af4aqQcY1Q

[tshaddix]

@mikecann Thanks for calling this out. I'll have to look into this.

I have tested using this on a non-persistent background page and it worked fine last time I checked. Mostly because chrome will keep the event page running if it is doing something (like connecting via a chrome port, which this package does on every injected page). Additionally, it saves it's state frequently, so it can handle reloads fairly well.

That said, I haven't explicitly tested this with the new v3 spec. We will need to do that. I'm pretty confident we can get it working without too much issue.

[mikecann]

@tshaddix ye I think service workers will randomly disconnect.. Not sure what happens with ports.. Its really annoying TBH

[tshaddix]

Alright I'm still working on testing this here: https://github.com/tshaddix/webext-redux-examples/tree/manifest-v3

As of now, it is not currently working. That said, I can't figure out if it's actually an issue with something in webext-redux or another issue with current Canary build. The current developer workflow for service worker based extensions is terrible IMHO.

I've read through quite a few write ups about the new v3 changes and I'm not seeing any reason this library should not work fine with it. The only major thing that I see changing is the recommended frequency of redux store saves to chrome storage.

I'll keep trying to debug this example extension...

[mikecann]

Ye, theres not that much love for the service worker background page at the moment.. Thanks for doing some research on this, let us know if you manage to get to the bottom of it. I think theres quite a few extensions relying on this lib :)

[tshaddix]

You got it! I'll keep messing with it.

[christospappas]

@tshaddix I checked out your branch linked above and tested it in Chrome 84.0.4147.21 dev. Clicks are registered and the counter updates across tabs. Is there something in particular that was broken when you were testing things or has the latest Chrome build fixed things?

[tshaddix]

@christospappas That's great to hear! It sounds like the latest Chrome build fixed things then. I tried the latest build about a month ago and was still running into issues.

I'll double check to make sure my local copy of the branch is good to go and then I guess we'll just continue to keep an eye on it as the Chrome dev builds progress.

Thank you for your help!

[mikecann]

I havent checked the code but does this account for random service worker disconnects?

[tshaddix]

So this package does not offer persistence at this point (maybe it should?), but you can set up a listener on store state changes to persist the store in memory, and then re-instate the store on extension load:

chrome.storage.local.get([
  'state'
], ({state}) => {
  // initiate redux store with saved state or new
  store = Store(state || {});

  // setup webext-redux
  wrapStore(store, {portName: PORT_NAME});

  /**
   * Save the current store state to local storage
   */
  const saveState = () => {
    if (!store) {
      return;
    }

    const state = store.getState();

    chrome.storage.local.set({
      state
    });
  };

  // On new state, persist to local storage (throttle function from lodash)
  const throttledSave = throttle(saveState, 1000, {trailing: true, leading: true});
  store.subscribe(throttledSave);
});

[Alino]

hi can you please tell me, where should we place this code to load the state from chrome.local.storage?
I have put it just in background page (I tried this, but it get's executed only once after the extension is reloaded) but I guess we should put it inside an "on extension load" listener? What's exactly that? Thank you

Edit: to make it clear why I am asking is because I have an issue that when I switch to other tab, my redux store get's overwritten with new rootReducer for the new tab.

For example here is a state when tab with id 1371 is active:

{
1371isInjected: false,
1371isPanelOpen: true
}

after switching to tab id 2334:

{
2334isInjected: false,
2334isPanelOpen: true
}

data about tab 1371 is gone. Should I load previous state every time a tab is switched from chrome.local.storage and merge it with current state? Or is there better solution, am I doing something wrong?

EDIT2:
so I found out that this disappearing state is happening because of dynamic keys due to tab id.
so I am rewriting my store to look like this:

{
    tabs: {
        2334: {
            isInjected: false,
            isPanelOpen: false
        },
        1371: {
            isInjected: true,
            isPanelOpen: true
        }
    }
}

now my store persists when switching tabs.

[anotherstarburst]

Just a heads up that I've managed to implement this (it took a bit of fiddling though). For anyone struggling I made use of the https://github.com/KELiON/redux-async-initial-state library as redux doesn't play nice out of the box with the promises that are returned from browser.storage.local.get('state'). The usage of the library in my case is something like:

const enhancers = [
  asyncInitialState.middleware(getInitialState),
  ...other enhancers e.g. saga
];

However, in my travels I have concluded that it's worth adding

browser.runtime.onSuspend.addListener(() => {
  saveState();
});

for non-persistent background scripts (which as I understand it is the way the future is looking in v3 as this thread discusses) to ensure the latest state is persisted before the background processes is suspended

[Techdojo]

@mikecann - thanks for opening this issue. I'm just starting to look in anger at rewriting my previous Chrome extension to be compliant with the new MV3 spec and I had exactly the same question as I used the previous version of this library to provide a react / redux framework in my popup page and it worked a treat and I was hoping to use the latest version as well.

FWIW - I'm happy to help out with testing or implementing any fixes to make sure that everything still works in a MV3 world 👍

[hindmost]

For those who still awaiting this library to work in Manifest V3: that will never happen, at least as long as it uses its current architecture. This architecture, let's call it client-server, assumes that the state of extension is permanently stored in a background script (server) and sending to other components (popup etc) by request. The trouble is that this approach is incompatible with event-based model used in Manifest V3. In this model background scripts do not live/run permanently and may be unloaded at any moment (that can't be known beforehand). So with this library, sooner or later, the state of extension will be lost - it's inevitable unless you save the state in chrome.storage every time it changes and load it from there every time it's needed.

Here the question arises: if we have to regularly save the state in chrome.storage anyway, then why do we need such intermediate as Webext Redux at all? Let's just store the state immediately in chrome.storage!

However, chrome.storage does not work in Redux way. So if you accustomed to thinking in Redux, you would like to somehow make chrome.storage compatible with it. That's why I wrote Reduxed Chrome Storage library. This is the only way to get Redux working in Manifest V3. And at the same time it is a unified way to use Redux in all modern browsers' extensions (as storage API has full cross-browser support).

[fregante]

this library to work in Manifest V3: that will never happen

@hindmost I disagree.

you save the state in chrome.storage every time it changes and load it from there every time it's needed.

Right, this is the solution. Service Workers are not immediately discarded. Doing so while they're being used would be counterproductive. One should assume that the SW will be open or will be re-opened when receiving a message.

why do we need such intermediate as Webext Redux at all? Let's just store the state immediately in chrome.storage!

chrome.storage does not work in Redux way

You answered yourself.

I don't see why this can't work. I do think that a background-less implementation could be faster, but you'll have to compare a possibly-in-memory call of a background implementation to a likely-not-in-memory implementation of raw storage.get calls. I don't know if the browser always reads from disk in this case.

[hindmost]

For some reason I haven't receive notification about the above comment. That's why my so late response.

@fregante I don't see why you started talking about implementation performance here (did I ever mention performance?). The issue with this library is not that it's slower or faster than something else. The issue is that it can't guarantee the state to be always up to date without regular and manual backup in chrome.storage. Since Webext Redux doesn't deal with chrome.storage at all, user has to do this backup himself/herself in each place (component) where Webext Redux is used.

And you used some of my quotes out of context. "chrome.storage does not work in Redux way" isn't the end of paragraph. After these words I suggest a solution of this problem ready to use.

[eduardoacskimlinks]

Hi Foks,

We are exploring migrating to V3 in our extension to align with Chrome extension requirements. I follow most of the comments on this issue which were helpful to give me an overall picture of the situation for this library.

Based on the recent posts between @fregante and @hindmost, I am concerned that the library doesn't have any mechanisms for persists the store if the browser decides to stop the service worker. Have you considered providing a persistence layer against chrome.storage as part of the library?

[yiziz]

@tshaddix looks like this is still an unresolved issue and a pretty major one given that multiple browsers are moving from persistent to event based extensions. I'm doing research on how to deal with this in my own chrome extension, and I'm a bit concerned about continuing to use this library. How dedicated is the webext-redux and goguardian team towards implementing a solution? Is goguardian still using this library in its suite of products?

[yiziz]

@tshaddix everything seems to work as long as the initial service working is active. however, once the service worker is terminated or idle, restarting the service worker does not automatically reconnect the selectors. The service worker loads the correct initial state and redux actions can still be sent, but until I refresh the tab, the selectors do not respect a redux state change.

I was able to address this by re-assigning the store variable with a new Store object and then calling ReactDOM.render into the existing/same container. While this does make the selectors work as expected (so far), I worry that there will be unexpected side effects that I haven't encountered yet.

[tshaddix]

@yiziz Interesting - okay. I will dive into this a bit more. Thank you for looking into this.

To answer your previous question:

How dedicated is the webext-redux and goguardian team towards implementing a solution? Is goguardian still using this library in its suite of products?

This package is maintained by contributors from the open source community and myself. GoGuardian does not support this package in any sort of official capacity. Unfortunately, my free time has been extremely sparse over the last couple years, but I still have every intention of supporting this package.

[tshaddix]

@eduardoacskimlinks Great question - in the past I've always just pointed people towards the demo for persisting the store state, but at this time with the new v3 spec, it's probably best to make it an official functionality of the package. I think this will naturally be required to support v3.

[eduardoacskimlinks]

Hey folks, I started the migration to manifest V3. I found that articles suggest bundling service workers with Webpack 5 to use the options target: node for reasons like DOM and XHR requests are unavailable in service workers. However, the moment I use this option, packages like redux, among others, doesn't build properly. Have you experienced this issue? How did you solve it?

[ymn]

Hey folks, I started the migration to manifest V3. I found that articles suggest bundling service workers with Webpack 5 to use the options target: node for reasons like DOM and XHR requests are unavailable in service workers. However, the moment I use this option, packages like redux, among others, doesn't build properly. Have you experienced this issue? How did you solve it?

Service workers don't have access to DOM (https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/#documents) and XMLHttpRequest is deprecated in service workers you need to use Fetch api (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)

[eduardoacskimlinks]

Hey folks, I started the migration to manifest V3. I found that articles suggest bundling service workers with Webpack 5 to use the options target: node for reasons like DOM and XHR requests are unavailable in service workers. However, the moment I use this option, packages like redux, among others, doesn't build properly. Have you experienced this issue? How did you solve it?

Service workers don't have access to DOM (https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/#documents), and XMLHttpRequest is deprecated in service workers you need to use Fetch API (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)

Hi Ymn, I appreciate your response; Perhaps, my discussion topic was unclear in my first message. I am aware of the situation with service workers regarding DOM and XHR requests ("Deprecated" is an understatement since it's no longer available)

My question is regarding the bundling process using Webpack 5 and the target: node option, which is recommended when building a service worker. This is causing current problems to import libraries like Redux on the JavaScript setup, so I am wondering if any of you during the migration experienced this issue and how you solved it

[sghsri]

Any update on this?

[nopol10]

Any update on this?

switching to https://github.com/hindmost/reduxed-chrome-storage worked out for me

[tshaddix]

Unfortunately, the amount of free time I have available continues to dwindle. Realistically, I won't be able to make any major modifications to this package in the near future. It's not something I'm thrilled about, but I just don't have enough time in the day anymore.

That said, I'm more than happy to review PRs if someone else attempts to migrate this package to V3. Alternatively, as @nopol10 pointed out, @hindmost has built a great package with a focus on event-driven extensions. I'd highly recommend considering it as an alternative.

[eduardoacskimlinks]

Any update on this?

switching to https://github.com/hindmost/reduxed-chrome-storage worked out for me

Do you have an example for using reduxed-chrome-storage? I am uncertain this solution will work properly as it's not handling the different changes for the background script for example, what happens when the extension goes idle and goes back? as well the management of the store section is a bit obscured so I am curious if you have use the library mention in a commercial application

[nopol10]

switching to https://github.com/hindmost/reduxed-chrome-storage worked out for me

Do you have an example for using reduxed-chrome-storage? I am uncertain this solution will work properly as it's not handling the different changes for the background script for example, what happens when the extension goes idle and goes back? as well the management of the store section is a bit obscured so I am curious if you have use the library mention in a commercial application

with reduxed-chrome-storage, dispatches from content scripts happen directly in the content scripts and the state is stored in chrome storage so even if the background service worker script is not active, it is ok.

see if this helps:
https://github.com/nopol10/nekocap/blob/389303d33af11d0408b04c3a74e7353645146767/src/common/store/store.tsx

https://github.com/nopol10/nekocap/blob/7245452d8aff6cffe4e2d3d06947522d6a799eb3/src/extension/background/index.tsx

[eduardoacskimlinks]

switching to https://github.com/hindmost/reduxed-chrome-storage worked out for me

Do you have an example for using reduxed-chrome-storage? I am uncertain this solution will work properly as it's not handling the different changes for the background script for example, what happens when the extension goes idle and goes back? as well the management of the store section is a bit obscured so I am curious if you have use the library mention in a commercial application

with reduxed-chrome-storage, dispatches from content scripts happen directly in the content scripts and the state is stored in chrome storage so even if the background service worker script is not active, it is ok.

see if this helps: https://github.com/nopol10/nekocap/blob/389303d33af11d0408b04c3a74e7353645146767/src/common/store/store.tsx

https://github.com/nopol10/nekocap/blob/7245452d8aff6cffe4e2d3d06947522d6a799eb3/src/extension/background/index.tsx

@nopol10 Have you faced the issue where the view stops receiving state updates to re-render after coming back from idle? And how did you solve it using reduxed-chrome-storage?

Flow illustrative:

Before going idle:

• content script Action --> Background Action --> store (state updated) --> View (new state sent back to all content scripts)
  After going iddle:
• content script action --> Background Action --> store (state updated) --x View (no state pass back to the any content script)

[nopol10]

From reduxed-chrome-storage's medium post (https://levelup.gitconnected.com/using-redux-in-event-driven-chrome-extensions-problem-solution-30eed1207a42):

Actions do not go through the background script any more. Content script has its own store, background service worker has its own store and all of them access the same state that is synchronized in chrome.storage so you shouldn't encounter the issue you mentioned.

[eduardoacskimlinks]

https://levelup.gitconnected.com/using-redux-in-event-driven-chrome-extensions-problem-solution-30eed1207a42

@nopol10 Thanks, that article was the missing piece of the puzzle as there wasn't real relation between webext-redux and redux-chrome-storage as they serve different purposes. Consequently, we need to change the application architecture if we want to support it.

There is but one last question on my side. I went through your repository to understand your implementation of the library. It seems that you share the same storage for all your scripts which is interesting as well you wait for the provider to be ready.

I wonder if you face some concurrency problems as the new model has a single point of failure which is chrome.storage (reduxed-chrome-storage) as we can have action competing to update the same information on the shared storage. The most obvious example is managing tab information.

[nopol10]

I did encounter an issue in redux-saga where putting (dispatching) and selecting from the state immediately yielded an outdated copy of the state but I'm not sure if this only happens in redux-saga or in Redux as a whole as a result of using reduxed-chrome-storage.
Otherwise I don't think I've encountered any other issues and it works just as redux should.

[codeyourwayup]

I am having the same issue, following up this thread.

[eduardoacskimlinks]

Hi @frankgo81 @tshaddix, I am currently working on an architectural proposal change for this library as migrating towards another library like reduxed-chrome-storage will be costly for us.

Hopefully, I should have the proposals available in a PR by the end of next week

[eduardoacskimlinks]

@tshaddix and @frankgo81 as promised, you can find the preliminary PR to fix the problems that webext-redux is facing in the current codebase.

There were two ideas on the table, implement the reconnect pattern and move away from connecting.

For context, the reconnect pattern didn't work because after the content script reconnected successfully, the background script went idle, sometimes creating an undesired loop situation.

Therefore, moving away from the connected architecture into a broadcast model with content-script state initialisation seems the most effective way to approach the problem.

Feedback on the approach is welcome on #282, particularly around edge cases that may affect the implementation

[z0ccc]

@tshaddix Will one of these PR's be approved?

[colbat]

Hi everyone, first of all, thanks for the great work on webext-redux. I use it for a long time for the Substital extension (actually since it was still called react-chrome-redux) and it successfully serves 200k users on Chrome, Firefox, and Edge.

I'm on the process to update the extension to Manifest v3 and I came across all the challenges described in this thread.

I think found a solution to make it work as it is with Manifest v3 and service workers. Turns out that the current version of the library already works well with non-persistent service workers (at least as far as I saw). We only need to adapt our code architecture a bit.

For context, I was already using chrome.storage to store my redux store since I wanted to preserve user settings.
Even though I didn't have any issue with the state persistence, I had some problems when the service worker was going idle.

The problem I was facing: After the service worker went idle, interacting with the popup (which connected to the redux store on load) was triggering an error: Port error: Could not establish connection. Receiving end does not exist. After the service worker was awake, closing the popup and opening it again didn't cause any issue.

Why this error: I believe it happened because at the time the popup loads, the connection with the store is not yet created. And why was the connection with the store not yet created? Because the store is initialized with values from the storage, and getting these values (with chrome.storage.local.get(...)) is asynchronous. Basically, it attempted to connect to the redux store, but the redux store didn't have the time yet to be initialized in the service worker.

Solution: Waiting for the redux store to be initialized from the storage (in the service worker) before connecting to it from other parts of the extensions (like from the popup for example).

The flow is the following:

1. Listen for when the popup (or the content script) is loaded.
2. When it's loaded: in the service worker, get the state from the storage and initialize the store. (wrapStore etc.).
3. Send a message from the service worker that the store was initialized.
4. In the popup (or content script), listen to this message, and then connect to the store (new Store(...)).
5. At this stage, in the popup, we can use the store and could also render a React app for example.

In the service worker, I subscribe to every state change and save the new state to the storage.

I've created a repository that demonstrates the solution: https://github.com/colbat/webext-redux-event-driven and where you can find more details in the code with comments. This is a simple counter with a popup.

I hope that helps, and happy to get any feedback.

[eduardoacskimlinks]

@colbat Thanks for contributing to addressing the current challenges for webext-redux. Your solutions sound interesting @RomanSavarin also proposed another solution which seems similar to yours. Feel free to have a look #282 (comment)

We don't use pop-ups on our side, so your insights on this matter will be valuable in how we can make my PR better to support fully Manifest v3 using this library.

Finally, I must warn you that we haven't heard from the owner of the repository in months, so we decided to fork the project to continue using the library

[tdriley]

I'm testing an alternative to using this library which avoids a lot of rewriting/refactoring by keeping Redux and the Redux reducer functionality roughly the same. It works like this:

(Note: I'm on old versions redux/react-redux, so using createStore for the time being, I'll update when the Chrome store MV2 deprecation date isn't impending so ominously).

1. Each extension part (Background, Popup, Options [maybe content scripts too?]) loads the contents of storage.local as soon as it starts (React app mounting or whatever). In my case I have some HOCs using Provider + connect and some Functional components using useStore.
2. Each part sets up its own Redux store with a dummy reducer and an initial data object that is the data loaded in 1.☝️ deep-merged into the store defaults. So, something like this:

    const loadedData = await browser.storage.local.get()
    const data = deepMerge(storeDefaults, loadedData)
    const store = createStore((state, action)=> Object.assign({}, state, action.newState), data)

    // Use store in your app

4. Each part also needs to add a storage change listener like this:

    browser.storage.onChanged.addListener(changes=> {
        browser.storage.local.get().then(newState=> {
            // "STORAGE_CHANGE" does not need to be an actual action, Redux just requires 'type' to be defined
            store.dispatch({ type: "STORAGE_CHANGE", newState: newState })
        })
    })

5. Then at the end of your main reducer function, save the new state to storage.local instead of returning it:

	// Action-specific stuff has happened already

    browser.storage.local.set(newState)

6. Now throughout any of your extension parts where you need to dispatch changes, you can call the reducer func instead of store.dispatch() (I've renamed mine storageReducer to make more sense), like this:

storageReducer({ type: "MY_EXISTING_ACTION", someData: someData })`

I'm still in the early stages of testing but this seems to work for me, and the storage.onChange event wakes up the background service worker if it has suspended. Every time any of the extension parts start, they get the latest data from storage.local and when other parts change the data, they load the latest again. I did not need to change any of the action-specific reducer functionality.

Interested in anyone's thoughts on how badly I've messed up.

[eduardoacskimlinks]

Hey @tdriley,

I believe leveraging webext-redux to manage state persistence within the browser cache is an excellent approach. I appreciate how clearly everything is explained in your comment on #244 (comment) and taking the time to share with the community.

Although we have moved away from the library due to the lack of maintainers and the changes brought by manifest v3 to the architecture (event-driven model), it remains a viable option, as you described.

However, I'd like to share some considerations based on my practical experience dealing with browser storage and manifest v3:

1. The persistence of the entire state in the browser cache can become problematic as the state grows.
2. There's a single point of failure: if browser.storage.local.set fails, the state will become inconsistent.
3. Depending on the complexity of your extension, ensure effective handling of concurrency when dealing with browser storage. For instance, what happens when point 5 triggers point 4 in conjunction with other user actions?

Please take all of this with a grain of salt as it's my own experience

[tdriley]

I believe leveraging webext-redux to manage state persistence within the browser cache is an excellent approach.

@eduardoacskimlinks Thanks! However, what I was trying to do is use only redux and react-redux and move away from webext-redux(for the same reasons you mention).

You're right about concurrency, I will need to do more research and testing around that.

[eduardoacskimlinks]

I see; that was my mistake. I should have caught up on the first comment. That's exciting, @tdriley! Please share your final implementation. I would love to see what the final solution looks like.