Reduce fs_accounts option size, possible?

[lkraav]

🚀 feature request

Description

Option table optimization dictates to look at autoload payload first.

SELECT 'Autoload kiB' as name, ROUND(SUM(LENGTH(option_value)) / 1024) as value FROM wp_options WHERE autoload='yes'
UNION
SELECT 'Autoload count', count(*) FROM wp_options WHERE autoload='yes'
UNION
(SELECT option_name, ROUND(LENGTH(option_value) / 1024) FROM wp_options WHERE autoload='yes' ORDER BY length(option_value) DESC LIMIT 10)

fs_accounts is easily one of the heaviest options recorded. Only WC Memberships has a heavier payload, but it has the excuse of being the backbone of the business.

Autoload kiB 646
Autoload count 1,821
wc_memberships_rules 96
fs_accounts 86
woocommerce_tracker_ua 33
wp_user_roles 30
rewrite_rules 19
redux_builder_amp 18
_transient_wpv_transient_meta_keys_visible512 17
itsec-storage 15
wc_customer_order_csv_export_orders_custom_formats 11
wsblc_options 11

Describe the solution you'd like

Please find a way to reduce autoload option payload. Significantly. 10x less kB would be a nice goal.

Describe alternatives you've considered

Yes, I will just disable Freemius wherever possible. I don't want to and neither do plugin vendors.

Your thoughts?

[sc0ttkclark]

I feel like some fs_accounts data is transient-worthy, while some makes sense to stay in the option itself. That might help solve this as transients can be offloaded more easily off to object caching solutions like Redis.

[vovafeldman]

@lkraav our initial thesis was "speed over size", i.e., it's "cheaper" to load a single record vs. multiple, as the intuition was that we need the data we store and the natural split will be based on plugins (a record per plugin). As the SDK progressed and we added many advanced features, we are now naturally storing significantly more details, and most likely, not everything is needed to be auto loaded.

So thanks for bringing that up 👍 we'll prioritize an audit of the SDK and rethink our storage layer design.

@sc0ttkclark and definitely, good chance some can be offloaded to transients too.

[ned-bs]

any update on this?
fs_update is second biggest autoload in my website.

[small-fork]

Any update or progress on this? I have two necessary plugins on my website now switching to Freemius but I'm hesitant to update because of this issue.

It looks like no activity on this for quite some time.

[vovafeldman]

Hey @bsahins @small-fork 👋 Thanks for checking in. We already made some healthy progress with the storage layer optimization in Dec 2022 (you can check the improvements here) and plan to continue the work in April/May.

[small-fork]

@vovafeldman Thank you so much for the update. I really appreciate it!

[ben72]

@vovafeldman Hi there! My fs_accounts is 75526 bytes big. Any way I can reduce it somehow?

[ben72]

@vovafeldman Any progress on this? My fs_accounts is now 69346 bytes large. When I look into it there are a lot of information about old plugins I no longer use. Is there a way to safely remove that data?

[fazhek]

Testing on staging site - removing fs_account from database clears data inside it from unused plugins, and the fs_accounts appears again with fresh data, much smaller in size. I am still unsure is it safe to proceed with this on a live site?

[vovafeldman]

@ben72, thanks for mentioning the old plugins data. We originally planned to redesign the entire storage layer (in addition to the optimizations that were already made in this branch). But it seems very risky, so as a start, we'll add a Garbage Collector for deleting data of plugins no longer in use.

@fazhek, just removing fs_account is a bad practice and can generate unexpected behavior.

[ben72]

@vovafeldman That seems reasonable. When can we expect to see the garbage collector? If you need to see an old fs_account with probably lots of garbage you can have mine.

[Kyleri]

I'm liking the garbage collector idea; my fs_account is ALSO hugetastic & I'd really rather be able to make it a LOT smaller. Thanks!

[splaquet]

The size of fs_accounts autoload has been horrible for years... and unless I'm grossly misunderstanding something, loading ANY of the Freemius plugins NOT associated with our installed plugin, makes absolutely zero sense.

Like many others on this thread, the fs_accounts options table is second largest.

[vovafeldman]

I know it's been a while, but we are almost there!

The GC (Garbage Collector) is ready and past regression testing. Due to the significance of the change, before we release it to the public, we first production test it on one of our plugins.

If you want to try it, here's the relevant branch: feature/garbage-collector-configurables

Make sure to set the WP_FS__ENABLE_GARBAGE_COLLECTOR define to true and WP_FS__GARBAGE_COLLECTOR_EXPIRATION_TIME_SECS to the length period for which you would consider that an inactive plugin is considered uninstalled (the default value is four weeks; we don't recommend to set it less than a week).

Once you have those set, the GC will clean uninstalled plugins data after the time you defined by WP_FS__GARBAGE_COLLECTOR_EXPIRATION_TIME_SECS

[swashata]

Update: we have released v2.6.0 with the Garbage collector. Please check the release note to find out more information: https://github.com/Freemius/wordpress-sdk/releases/tag/2.6.0

Thank you.