mastodoncustomemojis: out of order 🚧 ⚠️

[AlfredSK]

Expected behavior

When mastodoncustomemojis is enabled Mastodon custom emojis should show up as emojis.

Actual behavior

They don't (anymore).

Quote of @MrPetovan

There’s an outstanding caching issue where an empty emoji array will be cached for a specific domain, and all subsequent posts content from this domain will be rendered and in turn cached without the emojis replaced.

Steps to reproduce the problem

Discussion: https://libranet.de/display/98d99eb4155b7dc5ee677b2548461761
Test: https://libranet.de/display/52acf9cd63646e85c319d5dd562f01fc058a33c8

Friendica version you encountered the problem

2018.08-dev

Friendica source (git, zip)

git

PHP version

7.0.30

SQL version

10.0.34-MariaDB

[AlfredSK]

@MrPetovan Is there anything else I have to do? Because after git-pulling it still doesn't work.
https://libranet.de/display/52acf9cdf6e01ca9c12fc58650667ff762b5eeb1

[MrPetovan]

Please try to temporarily set system.ignore_cache to 1, reload the page and set back system.ignore_cache to 0.

[AlfredSK]

Where do I have to add that config key? In my local.ini.php? And if so, is [system] the right section?

[MrPetovan]

Yes, or you can use bin/console config system ignore_cache 1|0.

[AlfredSK]

The syntax of the command seems to be wrong. But temporarily configuring it in local.ini.php did the trick.
Thanks!

[MrPetovan]

I forgot the config keyword.

[MrPetovan]

I still see emojis shortcodes that should have been replaced by images. It always happen when the post content is generated by the worker, so I added some backend log, let's see what it finds.

[MrPetovan]

It looks like the put_item_in_cache hook isn't called when post content is rendered from the backend. If I disable the cache and look at /network or /display, the put_item_in_cache() hook the addon is riding on is correctly called, but if posts are coming in from the backend, they will be rendered without the hook being called at all.

@annando Any idea where the item content could be generated outside of put_item_in_cache()?

[annando]

Could you try to work with the daemon? (Just to compare possible differences). I already recognised problems with the loading of hooks.

[MrPetovan]

How do I work with the daemon?

[annando]

Just type bin/daemon.php start in the command line. No need to remove the crontab entries, these worker calls are ignored while the daemon is running.

[MrPetovan]

Hmm, I'm getting this error in the standard output, could explain the Twitter malfunction on my side:

PHP Notice:  Undefined property: stdClass::$id_str in addon/twitter/twitter.php on line 1005
PHP Stack trace:
PHP   1. {main}() bin/worker.php:0
PHP   2. Friendica\Core\Worker::processQueue() bin/worker.php:61
PHP   3. Friendica\Core\Worker::execute() src/Core/Worker.php:100
PHP   4. Friendica\Core\Worker::execFunction() src/Core/Worker.php:279
PHP   5. twitter_sync_run() src/Core/Worker.php:348
PHP   6. twitter_fetchhometimeline() addon/twitter/twitter_sync.php:29
PHP   7. twitter_fetch_own_contact() addon/twitter/twitter.php:1597
PHP   8. twitter_fetch_contact() addon/twitter/twitter.php:1780

[annando]

I'm currently am looking into it - and I guess I found something.

[MrPetovan]

Just updated, let's see.

[MrPetovan]

No dice.

Mastodon: https://mastodon.social/@trwnh/100637035823943470
Friendica: https://friendica.mrpetovan.com/display/52acf9cdaee68a8fa63680f2e59ed2bf3bb1def9

[annando]

It is working here after I reset the cache. I have a suggestion: We could store the emoji data in config values and we could update it on a daily base - but only store it when not empty.

[MrPetovan]

New potential issue: For this post: Is the original author in author-link or the booster? Because Sean doesn't have any custom emojis on social.nasqueron.org, which would prevent any replacement.

https://friendica.mrpetovan.com/display/e0f7609611df156d17358c9df3e93e00c6d16b57

[MrPetovan]

Why in config value if it expires every day? Currently the TTL for populated emojis is a week, which is too long. We could shorten it for a day or even an hour. For empty arrays the TTL is 30 minutes but it could be reduced to 5. The API call is quick and they are spread across multiple instances anyway.

[annando]

The cache expires. So upon a temporary server failure the data is gone. This doesn't happen with config keys.

BTW: after ignoring the cache, the post you mentioned above looked fine on my system.

[MrPetovan]

It isn't a problem if the data is gone, it would be re-fetched from the remote servers.

[annando]

For the next release it would probably an idea to fetch the emoji list during the regular server discoveries in the core. Then we could store this value in the gserver table.

[tobiasd]

For the next release it would probably an idea to fetch the emoji list during the regular server discoveries in the core. Then we could store this value in the ```gserver``` table.

What for? I don't think these emojis serve any purpose. Having an addon to deal with them is fine if the admin of a node feels otherwise. In fact I think an addon is the perfect place for this functionality.

[annando]

This fetching upon displaying the content does slow down the system and increases the network load. Having it only in a background process at a place where we already are querying this server would reduce the network load.

Of course the displaying of these emojis would still not be done automatically.

[MrPetovan]

It's only a < 200ms call for Mastodon.social for example, it doesn't slow down the system that much, especially since the call is cached for hour. Also it doesn't slow down the system if the addon isn't activated. I agree with @tobiasd here, we don't need to bring out the big guns in the Core. The API call is fast and is cached, which means a post can still be processed in less time than it takes to simply load @tobiasd 's own profile page without the addon.

This has never been about performance.

[annando]

It's not elegant ;-)

[MrPetovan]

Friendica is not elegant. Yet here we are! This is optional and a marginal performance hit, you can let it go, I promise.

[MrPetovan]

Confirmed it works this morning: https://friendica.mrpetovan.com/display/52acf9cd9788a6d64bb0960f6b367c8f29bd2ab7