Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reconsider placing uploaded fonts within the uploads directory #59417

Open
Tracked by #60528
johnbillion opened this issue Feb 27, 2024 · 57 comments · May be fixed by #60354
Open
Tracked by #60528

Reconsider placing uploaded fonts within the uploads directory #59417

johnbillion opened this issue Feb 27, 2024 · 57 comments · May be fixed by #60354
Assignees
Labels
[Feature] Font Library Needs Decision Needs a decision to be actionable or relevant [Priority] High Used to indicate top priority items that need quick attention [Status] In Progress Tracking issues with work in progress

Comments

@johnbillion
Copy link
Member

johnbillion commented Feb 27, 2024

In #53965 the decision was made to use the wp-content/fonts directory for uploaded fonts rather than wp-content/uploads/fonts.

Following this discussion in Slack about the Font Library, this decision should be reassessed. There are pros and cons for using wp-content/fonts versus using a fonts directory within the uploads directory, some of which are technical.

Notes

Arguments in favour of wp-content/fonts

Arguments in favour of wp-content/uploads/fonts

  • Directories outside of wp-content/uploads may not be writable, either as a security measure or due to offloading uploaded media to a CDN
    • This is potentially a blocker
    • A discussion on this topic has been started in the #hosting channel in Slack
    • This has been raised previously
    • The decision to store fonts outside the uploads folder is problematic for immutable file systems and set ups in which user uploads are offloaded. It's not simply a case of making the directory writable, plugins will need to be updated, servers reconfigured.
    • This could affect high impact hosts such as WordPress VIP, any site using security hardening plugins, or any site using DISALLOW_FILE_MODS
  • Font files are uploaded by users, therefore it makes sense to place the uploaded files within the uploads directory
  • Personal data export files are placed within the uploads directory. Even though there is no specific discussion on that decision in the core Trac ticket for that feature, it's likely this was done for the technical reasons above.

Please comment with any further arguments in favour or against either location.

@priethor priethor added [Priority] High Used to indicate top priority items that need quick attention Needs Design Needs design efforts. Needs Decision Needs a decision to be actionable or relevant and removed Needs Design Needs design efforts. labels Feb 27, 2024
@priethor
Copy link
Contributor

priethor commented Feb 27, 2024

I believe the fonts location should be outside uploads for the above reasons, which outweigh the cons. Furthermore, I would argue that users don't upload the fonts; they are downloaded from external directories just like themes and plugins.

Since the primary reasons for using the uploads folder seem to be practical reasons to ensure compatibility, why not use it only as a fallback or allow a setting to configure it?

@hellofromtonya
Copy link
Contributor

For historical context, during 6.4's merge consideration, this concern was raised. But the discussion didn't continue forward and IMO was unresolved.

Why? The primary focus was on the REST API's architecture as it was the biggest blocker and highest priority at that time. I suspect it consumed the focus and times, leaving less for deeper reviews and follow-ups.

@costdev
Copy link
Contributor

costdev commented Feb 27, 2024

I believe the fonts location should be outside uploads for the above reasons, which outweigh the cons. Furthermore, I would argue that users don't upload the fonts; they are downloaded from external directories just like themes and plugins (maybe wp-content should be considered instead?).

I agree. I think from an organisational perspective, wp-contents/fonts is the correct choice. IMO, we should be looking at what's needed to make that work, and the lead time needed on all fronts to make sure it's usable at release.

uploads, plugins, mu-plugins, themes, upgrade and upgrade-temp-backup are directories located in wp-content. Deployment setups will largely only have uploads as a writable directory there. I understand why this would lead us to putting a fonts subdirectory in uploads instead to avoid technical implications, though I think that would be us leaning into what's easier and immediately workable, though it's also a permanent decision for BC. To me, it feels more like we need to work on a solid communication strategy and affording appropriate timeframes to facilitate this change, than on the implementation.

If we were introducing the wp-content/uploads directory, and all of the same technical implications existed, how would we go about introducing this new writable directory, communicating this to hosts, developers and users, with enough time for identifying issues and for everyone to do what needs done to make it work?

@hellofromtonya
Copy link
Contributor

hellofromtonya commented Feb 27, 2024

I would argue that users don't upload the fonts; they are downloaded from external directories just like themes and plugins (maybe wp-content should be considered instead?).

I'm not sure I understand @priethor. Can you share more?

Media (images, videos, audio, PDFs, etc.) can also be first downloaded from somewhere (a camera, recording device, external media providers, etc.). It's the same for fonts as a user can upload a font file that they have stored on their computer.

@swissspidy
Copy link
Member

Initial response from asking in the #hosting channel:

Both Pantheon and WordPress.com VIP change the default location to wp-content/uploads/fonts because the current location doesn't work for them. See https://github.com/pantheon-systems/pantheon-mu-plugin/blob/main/inc/fonts.php and Automattic/vip-go-mu-plugins#5265.

If they have to add workarounds to make the feature work for them, that's an indicator that the default location is maybe not the best.

@costdev
Copy link
Contributor

costdev commented Feb 27, 2024

If they have to add workarounds to make the feature work for them, that's an indicator that the default location is maybe not the best.

I agree that it's perhaps an indicator, though I'd like to know about the roadblocks to them and others making the new wp-content/fonts directory writable instead of filtering to wp-content/uploads/fonts. I think it's important that we document the reason(s) why one possible path is chosen over another.

@johnbillion
Copy link
Member Author

This is because the uploads directory is mounted on a completely separate filesystem to facilitate horizontal scaling, replication, CDN etc. It's not just about making other directories writable.

Most WordPress hosts that provide scalable architecture work in the same way, and it's nothing new. Some docs here for Altis Cloud as an example: https://docs.altis-dxp.com/cloud/s3-storage/.

@andreilupu
Copy link
Contributor

I think that any uploadable should go to wp-content/uploads/ because this is the premise that we all followed by either plugin/themes developers, hosts, etc.

Introducing uploaded files outside of wp-content/uploads/ will introduce a new level of complexity. I can only imagine how many troubles this will bring to

  • multi-sites with front-end editors & upload capabilities.
  • external storages
  • CDNs (while any production site will want to cdn the fonts)
  • Docker instances that assume wp-content/uploads/ as a separate volume.
  • Hosts that use git as file versioning/backup systems; they usually treat wp-content/uploads/ differently.

@youknowriad
Copy link
Contributor

The "pros" of using wp-content/fonts seem to be more philosophical, whereas the "cons" seem to be more technical.

This argument is marked as in wp-content/uploads/fonts folder. I'd say that for me, it's more an argument for wp-content/fonts. Technical challenges can be overcome, philosophical aspects shape WordPress forever.

@youknowriad
Copy link
Contributor

It could be that using wp-content/fonts contributes to Font Manager uses incorrect URLs on multisite #59352, although that needs further investigation

This one is independent of this discussion so we should remove it from the arguments in favor of one or the other.

@youknowriad
Copy link
Contributor

I think that any uploadable should go to wp-content/uploads/ because this is the premise that we all followed by either plugin/themes developers, hosts, etc.

Themes, plugins are also things users can upload.

@aaronjorbin
Copy link
Member

if we go for the philisophical choice of wp-content/fonts, isn't that saying that the users of WordPress should bear the weight of this decision and the potential for this valuable functionality not working out of the box? That feels to me like it goes against the philosophies of WordPress.

Great software should work with little configuration and setup. WordPress is designed to get you up and running and fully functional in no longer than five minutes. You shouldn’t have to battle to use the standard functionality of WordPress.

https://wordpress.org/about/philosophy/

@youknowriad
Copy link
Contributor

youknowriad commented Feb 27, 2024

if we go for the philisophical choice of wp-content/fonts, isn't that saying that the users of WordPress should bear the weight of this decision and the potential for this valuable functionality not working out of the box? That feels to me like it goes against the philosophies of WordPress.

That's not exactly what I'm saying. I'm saying that we should overcome the technical challenges to make the philosophical choices work. (specially since they seem like fixable issues for what I've seen so far)

@johnbillion
Copy link
Member Author

The "pros" of using wp-content/fonts seem to be more philosophical, whereas the "cons" seem to be more technical.

This argument is marked as in wp-content/uploads/fonts folder. I'd say that for me, it's more an argument for wp-content/fonts. Technical challenges can be overcome, philosophical aspects shape WordPress forever.

I've moved this into the Notes section 👍

It could be that using wp-content/fonts contributes to Font Manager uses incorrect URLs on multisite #59352, although that needs further investigation

This one is independent of this discussion so we should remove it from the arguments in favor of one or the other.

I've removed this 👍

@cbirdsong
Copy link

We should keep uploads for things you insert into posts.

Is this actually an established expectation that is documented anywhere? That does not really square with what I generally see in the /uploads/ folder in practice. Here is what I found while surveying the sites I regularly work on:

  • a woocommerce_uploads folder
  • folders full of PDFs created by plugins that generate PDFs for various reasons
  • plugin data and logs for WP All Import, WP All Export and WP Migrate DB
  • backup copies of images processed by Shortpixel and Smush
  • files uploaded to Gravity Forms and WPForms by site visitors
  • page-specific CSS generated by a theme

If the uploads folder is only supposed to be for things you add to posts, then that ship sailed a long time ago.

If we end up adding folders to media library and someone adds a "fonts" folder for some reason.

I don't think this is a strong argument since it could be an issue with literally anything inside the /uploads/ folder. For that reason I imagine that any organizational structure added to the media library would only exist at the database level, like taxonomies.

@andreilupu
Copy link
Contributor

I think that any uploadable should go to wp-content/uploads/ because this is the premise that we all followed by either plugin/themes developers, hosts, etc.

Themes, plugins are also things users can upload.

I was trying to express the fact that WP products(plugins/themes) presumed that uploadables go in wp-content/uploads/. Examples:

  • An import/export plugin like Migrate Db or Duplicator, while they try to clone a WP site will zip the uploads folder entirely and not mess with it at all. A change like this will force them to go a level up above uploads and not all the hosts allow such a thing.
  • The vast number of premium themes have a system of Export/Import layouts and I believe they will have similar problems.

Probably everything I wrote can be solved from the technical point of view(with some complexity costs) but this doesn't strive for simplicity.

@youknowriad
Copy link
Contributor

youknowriad commented Feb 28, 2024

Is this actually an established expectation that is documented anywhere? That does not really square with what I generally see in the /uploads/ folder in practice. Here is what I found while surveying the sites I regularly work on:

the crux of the problem seem to be that there's no documented expectation for either case:

  • Core uses uploads for media + personal data export paths.
  • Core uses non wp-content (not just uploads) for uploadable entities as well.

Which means, the decision comes down to what we think is right mostly rather than history/consistency.

@azaozz
Copy link
Contributor

azaozz commented Feb 28, 2024

It's quite unfortunate that this issue was opened now, few days before WP 6.5 RC1, many months after the code in question was added and the original decision was made. This makes it very hard to test any new theories and/or ideas.

I'll try to answer and/or clarify some of the above points.

Additional arguments in favour of wp-content/fonts

  • There are many plugins that do all sorts of things with the files and directories that are in /uploads. Some may move them, or display them in a particular way, or even copy them to a remote location and delete them. Adding a non-standard directory (standard names there are year/month) with "special" files would likely be unexpected by some of these plugins. This could potentially result in all kinds of breakage, from unexpectedly displaying font files and allowing users to perform unexpected actions on them, to potentially "mangling" the URIs to them. Seems this may potentially affect a large amount of WP users. Also see the above example of how "littered" the /uploads directory may get. Not sure it would be a good idea for WP itself to add to that "messiness". (Side note: I understand that /uploads is the only location where plugins can be sure they will be able to add or store files. However it seems in most cases these are temporary files that plugins have failed to "clean up").

  • Seems that large sites and some agencies may want to keep the fonts directory under version control (fonts get updated too). As far as I see it would be pretty bad to have such directory in /uploads.

Individual points

The decision to store fonts outside the uploads folder is problematic for immutable file systems and set ups in which user uploads are offloaded. It's not simply a case of making the directory writable, plugins will need to be updated, servers reconfigured.

Frankly I'm a bit unsure what "immutable file systems" means here (no files are ever updated?). Not sure how common this setup may be but seems it prevents the WP (security related) auto-updates from being applied. Seems this is a pretty bad idea unless there is enough personnel to apply updates "manually" or to create and maintain an auto-updates system outside of WP.

Seems that when there is enough personnel for manual auto-updates or to maintain a proprietary auto-updates system, it won't be such a big burden to adjust the server settings to allow use of wp-content/fonts? Also seems that in some of these cases the site owners may decide to have the /fonts directory under version control thus preventing any WP users from installing any fonts through the UI (same as plugins and themes).

This could affect high impact hosts such as WordPress VIP, any site using security hardening plugins, or any site using DISALLOW_FILE_MODS

Same as above. As far as I understand DISALLOW_FILE_MODS should be set in the cases where the site owners do not want the WP users to install or update any WP files through the UI and/or when WP core, plugins, and themes are under version control. Note that in these cases wp-content/fonts may be under version control too.

Font files are uploaded by users, therefore it makes sense to place the uploaded files within the uploads directory

That's not exactly the case afaik. Generally plugins and themes can also be uploaded (zipped) by the users. That seems very rare for the plugins and themes hosted on wp.org but somewhat common for "commercial" plugins and themes afaik.

To me it seems the main points here are whether WP should "add a /fonts sub-directory to (the messiness) in wp-content/uploads and "hope for the best" (i.e. hope that all the plugins that create that messiness would play nice, now and in the future), or have a dedicated directory in wp-content that will ensure fonts will work properly (including the possibility to be under version control) but would require some sites to make some changes. Note that there is exactly one month time for these changes to be implemented.

Seems the only thing missing from the current implementation (#59294) is a way to enable uploading of fonts when DISALLOW_FILE_MODS is set but the wp-content/fonts directory is configured to allow uploading and storing of font files. See: #59294 (comment).

@costdev
Copy link
Contributor

costdev commented Feb 28, 2024

An import/export plugin like Migrate Db or Duplicator, while they try to clone a WP site will zip the uploads folder entirely and not mess with it at all. A change like this will force them to go a level up above uploads and not all the hosts allow such a thing.

Import/Export/Backup plugins also tend to offer plugin/theme backups, which would require access to wp-content subdirectories. Do you know what happens to the plugins you mentioned in these cases?

Themes, plugins are also things users can upload.

Indeed, and while they could be called an "uploadable", as they are uploadable, their relationship to the site is much different to media or an export, and they are organised into their own wp-content subdirectory.

Great software should work with little configuration and setup. WordPress is designed to get you up and running and fully functional in no longer than five minutes. You shouldn’t have to battle to use the standard functionality of WordPress.

While this is obviously a crucial aspect of the WordPress philosophy and one I very much agree with, "battle" is a rather subjective term. For example, I don't consider using a filter to be a battle. While the ideal is that no configuration is necessary to use new functionality, it's sometimes needed because we have multiple factors to consider.

Whether a filter is sufficient for scalable architecture to handle it, is another matter. I know VIP filters it, as does Pantheon. Altis, for example, filter upload_dir to serve an s3:// location. Would doing similar for the font_dir filter work?

@peterwilsoncc
Copy link
Contributor

Elsewhere I've been using Altis, WP VIP and Pantheon as examples for this issue. Part of the reason I feel comfortable using these hosts as an example is because I know these companies follow WP developments and will work to handle it. It's the long tail of hosts that don't follow development that this issue needs to be decided for.

The Font Library makes use of assets from two locations:

  • theme.json within themes and stored in the theme's folder
  • Fonts uploaded by the user, either directly as files or via Google Fonts as a directory service

It's unlikely the font directory will need to be kept under version control on large sites as simply storing the files there will have no effect. The file isn't known to the font library without an entry in the post_content table. Sites and agencies are likey to store commitable fonts in the theme folder and theme.json

It's only files that are uploaded that will end up in the fonts directory, be that wp-content/fonts, wp-content/uploads/wp-fonts or wp-content/uploads/2024/02.

This argument is marked as in wp-content/uploads/fonts folder. I'd say that for me, it's more an argument for wp-content/fonts. Technical challenges can be overcome, philosophical aspects shape WordPress forever.

The big issue I see here is that the burden for overcoming these technical challenges is been moved from something managed by a few dozen contributors to WordPress to the entire plugin, hosting and site owner communities.

WordPress makes allowances for the practical over the technically pure frequently. A good example being privacy exports as discussed elsewhere in both issues and Slack.

Accepting for a moment that WordPress needs a directory structure for uploaded content that isn't an attachment, then design decision to call that folder fonts isn't great as it limits it to a particular type of media. At the risk of showing my age, in years gone by the folder might have been called htcs for the use of HTML Components.


I think we (those of us riding these various tickets) need to switch font uploads to the uploads directory alongside other user generated file data. In short:

  • the files are uploaded
  • they require associated post content entries & meta data
  • it will just work for sites running bucket offloaders, CDNs and file structures in which uploads is on a different disk

Whether the destination is uploads/yyyy/mm/file, uploads/(wp-)fonts/file or something else doesn't really bother me as much.

@youknowriad
Copy link
Contributor

youknowriad commented Feb 28, 2024

If tomorrow, we are to introduce a way for users a way to upload "patterns" or "blocks", "an ai model" or something else, I don't think it would make sense to put these in uploads. In fact it's the "content" of WP in general, so I think wp-content is still the right place for it. Same goes for fonts.

So we need to ensure that we can make the right decisions for WordPress in general.

I think a small amount of adaptation is required for any big feature, especially for environments that are very specific. It was the case for the REST API addition, for the customizer, for the block editor, and it will be the case for fonts regardless of the chosen folder. Yes, it's our job to make the adaptation very minimal (and inexistent for regular users just downloading the zip).

I believe that regardless of the chosen folder, we need:

  • A way to filter the location (which we already have and I believe will be improve upon to make it simpler). Special environments can and have already adapted to the feature for some of them.
  • A lot of communication (we have some time, one month, to overshare here so make sure everyone is ready).
  • Ensure that the UI is disabled in environments that didn't adapt.

It's unlikely the font directory will need to be kept under version control on large sites as simply storing the files there will have no effect. The file isn't known to the font library without an entry in the post_content table. Sites and agencies are likey to store commitable fonts in the theme folder and theme.json
It's only files that are uploaded that will end up in the fonts directory, be that wp-content/fonts, wp-content/uploads/wp-fonts or wp-content/uploads/2024/02.

I think that's a good point, but maybe it's only temporary. I don't see why we can't follow-up with an enhancement to show these fonts as inactive if people think it's useful.


Now, I'm not yet convinced that we need to change the font directory to uploads.

So I still think we need to answer this question: How do we decide which part of wp-content is the right place for fonts? Also I do believe that this discussion is important for future similar additions? Will we always clutter uploads with any user files or should we have a more organized wp-content folder?

@mcsf
Copy link
Contributor

mcsf commented Feb 28, 2024

I've been sitting on this for the better part of today, going back and forth on my position.

Conceptually, I'm certain the right place is wp-content/fonts/, but I understand the headaches that come with anything other than wp-content/uploads/*. That said, I don't think that choosing this path necessarily means, as @aaronjorbin warned, letting "WordPress bear the weight of this decision and the potential for this valuable functionality not working out of the box". That is the danger for sure, but we why not consider it and work to reduce it?

Riad's last comment resonates with me: don't we have the means to get the word out, put some conservative heuristics in place (e.g. to fall back to wp-content/uploads/ if conditions aren't met, etc.), obviously add the appropriate filters, and ultimately come out of this with a framework that is more future-proof even with large enterprise setups?

@jazzsequence
Copy link

As the author of one of the workarounds @swissspidy referenced earlier for Pantheon, I will start by saying that I don't have a strong opinion about where, as long as where is configurable, which it is today (it wasn't when we had this discussion for 6.4 and I'm happy that we've taken the time to resolve that).

Pantheon, as mentioned above, is one of the hosts that uses an immutable file system.

Frankly I'm a bit unsure what "immutable file systems" means here (no files are ever updated?).

Yes, precisely. The entirety of a sites files are an immutable artifact -- they cannot be modified or changed -- with the exception of specific, named and excluded paths (that point to specific container mounts). This ensures that malicious code can't simply be installed (or executed via the admin and bad actors) that writes changes to the file system.

seems it prevents the WP (security related) auto-updates from being applied.

This is true. Pantheon maintains a mirror of WordPress core that is automatically updated when new releases are published and pushed to customer sites. Customers update WordPress via their Pantheon dashboard using our internal git version control system. Automatic updates are disabled. Plugin and theme updates are committed to version control. Customers need to apply these updates and push them through our development and test environments before finally pushing those updates live. On the one hand, this means that customers may be behind on their updates. On the other hand, the immutable file system saves them from many security vulnerabilities that would otherwise use privilege escallation and the like to write to the filesystem.

I'd like to know about the roadblocks to them and others making the new wp-content/fonts directory writable instead of filtering to wp-content/uploads/fonts.

when there is enough personnel for manual auto-updates or to maintain a proprietary auto-updates system, it won't be such a big burden to adjust the server settings to allow use of wp-content/fonts?

Changing the infrastructure to allow an additional writeable path is far more burdensome than simply writing a function in a mu-plugin and adding it to our maintained version of WordPress. Carving out that exclusion specifically for site frameworks of this CMS type and not that CMS type also adds overhead. There are fewer components (namely one) involved in making the change in the WordPress application than in making the change at the infrastructure level. Everything @johnbillion and @peterwilsoncc noted in this regard is true for us as well.

Also seems that in some of these cases the site owners may decide to have the /fonts directory under version control thus preventing any WP users from installing any fonts through the UI (same as plugins and themes).

This was also noted above, but, today, this is not possible either. Or rather, you could version control your fonts, but it's not going to do anything for you. As noted in #59102 if I have fonts in the wp-content/fonts directory committed to version control and moved across environments, the Font Manager has no knowledge of them. They are simply files that exist. The only way for WordPress to have knowledge of those files is for them to be uploaded through the Font Manager explicitly (e.g., in the same way that media files are uploaded and managed). If I want to move my uploaded fonts between environments, this is made possible by importing the database from the environment where the font files exist into the environment where the files exist but the references do not (e.g. from production to staging or develop).

I agree that the distinction between wp-content/fonts and wp-content/uploads/fonts is philosophical. However, I tend to think that, based on the way that the Font Manager works, the fact that I can't just drop font files into /fonts and have WordPress recognize them (as it does with plugins in /plugins, themes in /themes, or even mu-plugins in /mu-plugins) and the fact that the only way for WordPress to "see" my fonts is by uploading them through the Font Manager (or installing via the Font Manager) that functionally they behave far more like media files/attachments than they resemble plugins, themes, mu-plugins or other core components of a WordPress site.

If a WordPress site has no theme, it would not display. If a WordPress site has no plugins, it would lack some functionality. If a WordPress site has no media files, nothing happens. Similarly, if a WordPress site has no installed fonts, it has no functional effect on the site.

@joemcgill
Copy link
Member

@jazzsequence makes a very good point here. I'm curious how challenging it would be to have fonts uploaded along with some sort of .json manifest that WP can use to load the files? This could work similarly to block patterns, where WP can also store this data in a post type, but themes can just include the files and WP will pick them up, without a need for the info to be in the DB.

@justlevine
Copy link

I think we are perhaps glossing over the fact that a large motivation behind the font library is to introduce fonts as a first-class object within WordPress—stored, described, and managed in a way that allows us to convey this distinct nature alongside other objects.
[... ]
Hence, I don't think placing it in uploads due to the immediate convenience outweighs these considerations

@mtias could you perhaps clarify what you mean by "distinct nature", and how fonts more closely resemble plugins, themes, languages, etc versus those currently in uploads (be them managed by the Media Library or a plugin)?

Of the several non-"convenience" arguments made above, the one that changed my mind over to camp uploads was the delineation between database-managed and file-managed registration (I believe first brought up on this ticket by @jazzsequence ).

It's a solid through-line to guide future architecture and abstraction decisions (ai models vs uploadable patterns , generated caches or static assets vs backups, data liberation exports, etc) that clears up the catch-all vibes, and just-so-happens to play well with what good ecosystem actors have been doing. It also helps differentiate these database-managed fonts vs those file-managed by the theme.

@Zealth57
Copy link

Zealth57 commented Mar 1, 2024

Thank you all for the nuanced and considered discussion here. I think we are perhaps glossing over the fact that a large motivation behind the font library is to introduce fonts as a first-class object within WordPress—stored, described, and managed in a way that allows us to convey this distinct nature alongside other objects. So this is fundamentally an architectural decision we are making and not exclusively one of convenience, which aims to position fonts as a concept alongside themes, plugins, languages, and media uploads within wp-content.

Keeping things in uploads may provide a quicker path for adoption but I think at significant expense towards clarity and longer term brittleness of the architecture. It would further project "uploads" as a catch all, which degrades the abstraction with conceptual dilution, and introduces a higher chance for mistakes like the ones @azaozz describes. There's already countless articles describing how to change the location of the media folder in WP and it's not intuitive that this will include fonts but not things like themes and patterns.

Hence, I don't think placing it in uploads due to the immediate convenience outweighs these considerations. We are introducing a new concept and we should be able to navigate the adjustments required across the ecosystem to support it in a positive way. I believe this ultimately serves the project, its users, and the ecosystem better.

Apologies if I'm wrong because I haven't followed this feature closely; but I believe I read that fonts can't be dropped into the folder and automatically recognized, they must be uploaded because they live in the database. This is counter to plugins, themes, etc so the insinuation that they should be treated as this sort of this new property or like those doesn't hold merit. If you can put fonts into the folder like the other things I'd be more open to having /wp-content/fonts. As the feature is written now (or as I understand it) and the fact that these are really more like frontend assets that themes and plugins I am against making a new special folder outside of uploads. For every CDN that moves things from the uploads folder or that offloads it from there this wouldn't work without extra work. I understand you want to introduce a new concept but I respectfully disagree that it's better for the project, ecosystem or users. In fact there are many voices above who insist this is not better.

@azaozz
Copy link
Contributor

azaozz commented Mar 2, 2024

how fonts more closely resemble plugins, themes, languages, etc versus those currently in uploads

Good question.

Thinking it would be good to try to help "bring everybody on the same page" about how fonts are being considered and the (current) logic around installing and managing them. Please correct me if something seems wrong :)

Fonts are:

  • Media files (not "executable" and not "scripts").
  • Considered to be part of a website's visual design (the WP theme and/or template), not something that is used "per post" like images for example.
  • The expectation is that a typical website would use about a dozen fonts at most. Technically it is possible to add a lot more, but chances are that would not make sense from a design point of view. Having a lot of different fonts on the same page would also degrade front-end performance.

Font management:

Fonts are managed from the Font Library. It is available at: Site editor->Design->Styles->Edit Styles(the small pencil icon next to the "Styles" label)->Typography->Manage Fonts(the small "settings" icon next to the "Fonts" label).
font-lib

The expectation (afaik) is that fonts will generally be installed by using the "Install Fonts" tab. This works pretty much the same as installing WP plugins and themes: the users select which fonts they want and WP (the server) downloads and installs them. The only difference is that font files are (currently) downloaded from the Google CDN, not from wordpress.org (for privacy reasons there is a confirmation dialog where the user has to give permission to their server to actually connect to the Google CDN and download the fonts).
install-fonts-1

As an alternative install method there is also an "Upload" tab (like for plugins and themes) where the users can upload font files they have purchased from a type foundry (for example), or if they prefer to download the font files themselves from other sources, for example Fontsource.

To answer the question directly: Font files resemble plugins, themes, and language files because:

  • They are optional "visual website design components".
  • The expectation is that there will not be a large number of them per site, if any. For comparison there usually are tons of images, PDFs, and other media files that are uploaded by the users.
  • They are installed (in most cases) by WP/the server downloading the file(s) form a third-party CDN.

Thinking @mtias said it really well above:

a large motivation behind the font library is to introduce fonts as a first-class object within WordPress—stored, described, and managed in a way that allows us to convey this distinct nature alongside other objects.

@Zealth57
Copy link

Zealth57 commented Mar 2, 2024

Thanks for the additional context and summary @azaozz that was very helpful. A few initial reactions:

  • Media files (not "executable" and not "scripts").
  • Exactly why I believe putting them into uploads makes the most sense. These at the end of the day are media files not scripts and code like plugins/themes
  • Considered to be part of a website's visual design (the WP theme and/or template), not something that is used "per post" like images for example.
  • Not just images in a post go into the media library. Many developers create places to upload favicons and logos which also go into the media library and are part of a websites visual design
  • This works pretty much the same as installing WP plugins and themes
  • Not really, plugins and themes can be added to file systems and pushed up, in this fonts can't. That's a HUGE distinction and critically important that cannot be glossed over. Themes and plugins definitions are not stored in the database, they're only stored as active/inactive. Fonts are in the database just like media library files.
  • The only difference is that
  • This is not the only difference as noted above
  • The expectation is that there will not be a large number of them per site
  • The number of files is not a strong argument to me to relate them to being as themes, plugins, etc. The relation to them is meant to be relation to how they interact with the filesystem, database, and user interaction - not sheer number.
  • They are installed (in most cases) by WP/the server downloading the file(s) form a third-party CDN.
  • This has been out in beta in some places, do we have real data to support this? I don't know if this is a true statement or opinion. Also is this because the current implementation doesn't support any other way? If it was a folder that picked things up (like themes and plugins actually do) it might make more sense.
  • a large motivation behind the font library is to introduce fonts as a first-class object within WordPress—stored, described, and managed in a way that allows us to convey this distinct nature alongside other objects.
  • I don't think putting them into wp-content/fonts instead of wp-content/uploads introduces them as a first-class object. If it did then it would allow me to put files in there just like plugins and themes and 'just know' about them. As you describe the majority are installed by WP or downloading the files from a third-party. Users will not know or care where the files are stored on their servers. Users WILL care deeply if we release something that's broken on hosting and require significant effort to update. Putting a folder so it's conveyed as something distinct and works like some things and not others while potentially breaking things doesn't seem like a great idea.

@azaozz
Copy link
Contributor

azaozz commented Mar 2, 2024

@Zealth57 Sure, lets talk a bit more about this :)

Exactly why I believe putting them into uploads makes the most sense.

Don't think /uploads is the canonical location for specific file types. It is the canonical location for all files that are uploaded by the users to WP's Media Library.

plugins and themes can be added to file systems and pushed up

Sorry but I'm unsure what "pushed up" means here. Assuming it refers to the files being able to be under version control. If that's the case I don't think all WP plugins and themes can be under version control. Afaik some (many?) "commercial" themes and plugins do not have publicly accessible repositories and are available as .zip downloads only. Updates are also applied by downloading the zip file and then uploading it to WP.

Themes and plugins definitions are not stored in the database, they're only stored as active/inactive.

Sorry but don't see big difference here. Some metadata is stored for themes and plugins in the DB. Some more metadata is stored for images and fonts, and probably will be stored for other distinct components like for example AI models. True, the plugins and themes directories are also "traversed" as backwards compatibility with how themes and plugins were installed 15 years ago: by uploading them with FTP. Frankly I'd really like to remove that if possible, just slows WP down pointlessly :)

The number of files is not a strong argument

Perhaps. But that still makes (the requirements for) handling of font files quite different than images for example.

The relation to them is meant to be relation to how they interact with the filesystem, database, and user interaction

Not quite imho. The most important part is how users would perceive, manage, and interact with fonts. Filesystem and metadata in the DB are by far secondary. I.e any UI and UX solution can be "coded" and made to work. The important part is to be "logical", to "make sense" to the users, to be "easy to understand and use", etc.

do we have real data to support this

No "real data" afaik. The assumption is based on how themes and plugins (that are hosted on wordpress.org) are installed and managed. It seems very rare for modern WP sites to have plugins and themes (that are in the WP repositories) installed by not using WP (the UI or wp-cli). Seems these are only higher-end sites that have support personnel, etc. The expectation is afaik that fonts will also be installed by using the WP UI, as that is a lot easier and straightforward than the more elaborate method of selecting form external sources, downloading and then uploading the file(s).

I don't think putting them into wp-content/fonts instead of wp-content/uploads introduces them as a first-class object.

As mentioned /uploads is primarily for files that the users are uploading to the WP Media Library. The fact that some plugins are using the same location for other files is quite unfortunate imho. See #59417 (comment), mostly the part about how "messy" the /uploads dir may get :)

@samuelsidler
Copy link

Don't think /uploads is the canonical location for specific file types. It is the canonical location for all files that are uploaded by the users to WP's Media Library.

This isn't quite right.

Taking a step back, themes and plugins are generally code and assets packaged together in a way that can be executed by WordPress. Plugins can include assets within them—these assets are defined by the plugin author and are meant to be utilized only when the plugin is activated on the site. If a plugin is not active, it cannot be used by the site. (Barring edge cases, of course.)

Themes, including block themes, can include assets within them—these assets are defined by the theme author and are connected to the given theme. If a person modifies the visual display of a theme using the Site Editor or other built-in tools within the dashboard (excluding the code editor), these modifications are stored at the "site" level—the code and associated assets for the theme are not modified within the theme directory itself, on the filesystem. When a user deactivates a theme (by switching to another), any assets including with the theme files are immediately unavailable to the site, including icons, fonts, images, and (naturally) code.

This is an important point, because, historically, there are assets that are not stored within an active theme, which may be used by themes, in support of the visual display of the site. A solid example here is the site logo and icon, which are not stored alongside the theme—these are available site-wide. If a user modifies their site logo or icon, those assets remain in use, even when the theme is changed.

The site logo and icon are stored in... /uploads/, along with other assets that persist after deactivating plugins and themes.

More recently, block themes can be modified within the Site Editor. These modification are stored in the database and do not modify the code of the theme file itself. Except... if a user adds (e.g.) an image to a template within the Site Editor (perhaps a header or background graphic, to be used atop all templates), the image is uploaded to /uploads/, just as other site-level assets are. They are not stored within the theme directory.

Going back to your statement:

Don't think /uploads is the canonical location for specific file types. It is the canonical location for all files that are uploaded by the users to WP's Media Library.

I would counter that /uploads/ is the canonical location for assets which should be available site-wide, regardless of whether a theme or plugin is activated. This is the current and legacy behaviour of WordPress. In that regard, fonts, like images and logos and icons, are also assets that should persist. Once installed, every theme should have access to these fonts, and disabling a theme should not disable fonts that were uploaded to the Font Library.

Circling back to an earlier, related comment:

Fonts are:

  • Media files (not "executable" and not "scripts").
  • Considered to be part of a website's visual design (the WP theme and/or template), not something that is used "per post" like images for example.
  • The expectation is that a typical website would use about a dozen fonts at most. Technically it is possible to add a lot more, but chances are that would not make sense from a design point of view. Having a lot of different fonts on the same page would also degrade front-end performance.

I agree with virtually all of these points, but there's a nuance to the second bullet point. Images can also be part of the website's visual design, and not per-post. As described above, that is not only a common behaviour with block themes (a relatively new behaviour of WordPress), but also a legacy behaviour of WordPress since the introduction of site icons in WordPress 4.3.

To answer the question directly: Font files resemble plugins, themes, and language files because:

  • They are optional "visual website design components".
  • The expectation is that there will not be a large number of them per site, if any. For comparison there usually are tons of images, PDFs, and other media files that are uploaded by the users.
  • They are installed (in most cases) by WP/the server downloading the file(s) form a third-party CDN.

Fonts files resemble other, optional visual website design assets used on sites, like site logos and icons, and images used in templates. Just like those other visual website design assets, there may not be a large number of them, but it's important that they are treated like the assets that they are—e.g. if all assets of the site are offloaded to a CDN, font files should also be offloaded to the CDN. How they get installed, and what that interface looks like, isn't relevant to how these assets should be treated and used within a WordPress installation.

Sorry but I'm unsure what "pushed up" means here. Assuming it refers to the files being able to be under version control. If that's the case I don't think all WP plugins and themes can be under version control. Afaik some (many?) "commercial" themes and plugins do not have publicly accessible repositories and are available as .zip downloads only.

A very, very common behaviour with enterprise WordPress sites is that commercial plugins (and private ones) are downloaded from their respective locations (as a ZIP, if need be), and committed to version control. On enterprise WordPress environments, the file system is locked, with the exception of the uploads directory, which is usually offloaded (e.g. to a CDN). While I'm certain there are a few WordPress plugins which are behaving badly (writing to their plugin directory), virtually every WordPress plugin and theme can be committed to and managed within version control.

Updates are also applied by downloading the zip file and then uploading it to WP.

To be clear, this is not how plugin updates work within enterprise WordPress environments (including WPVIP). Updates are committed to version control and deployed to the production environments.

@peterwilsoncc
Copy link
Contributor

peterwilsoncc commented Mar 3, 2024

I think what comments that the ecosystem will adapt are missing a fundamental point. The ecosystem IS adapting and is generally doing so by reversing the architectural decision to place font uploads outside the uploads directory.

The ecosystem is respecting the argument that fonts are slightly different to other attachments by using a font directory in uploads rather than simply removing the filter on the uploads hook, ie:

add_filter( 'upload_dir', function ( $ul ) {
	remove_filter( 'upload_dir', 'wp_get_font_dir' );
	return $ul;
}, 5 );

Note: The above is broken following df5dc24 but I'm working on restoring it WordPress/wordpress-develop#6211

Generally this reversal of the decision is being made by some of the most influential players in the ecosystem so will become the default: very large hosts, large agencies, even official WordPress sites in WordPress/wordcamp.org#1075. The very folks who shared case studies to promote the use of the block-editor when it was only available in the plugin prior to WordPress 5.0.

There is talk of including a workaround in the dev note announcing the library. If it's accepted that the dev note requires documenting a fix, then the fix should be unnecessary and implemented at a Core level.

@youknowriad
Copy link
Contributor

If it's accepted that the dev note requires documenting a fix, then the fix should be unnecessary and implemented at a Core level.

The dev note is not about documenting a fix, it's about documenting a filter that may or may not be used.

@priethor
Copy link
Contributor

priethor commented Mar 5, 2024

Thanks for the extensive discussion, everybody.

At this point, a decision needs to be made. Since there isn't an agreement on the best path forward, project leadership should make the call. For transparency, in the last few hours, I've been filling @josephahaden in on this topic so that an informed decision can be made in the next few days and before RC2.

@azaozz
Copy link
Contributor

azaozz commented Mar 5, 2024

Sorry for the late response, just wanted to clear couple of things.

@samuelsidler

This isn't quite right.

You mean the /uploads directory being (the canonical location) for files uploaded by the users by using the WP Media Library? Thinking this is and has always been the intended use of wp-content/uploads.

I agree, seems these have been some assumptions over the years:

  1. DISALLOW_FILE_MODS actually means that every WP file and directory regardless of its location in the filesystem is not writable (by WP) except wp-content/uploads.
  2. There also seems to be an assumption that "anything can be dumped in /uploads regardless of what it is and where it comes from".

The first is more of a coincidence as far as I see. There are several files and sub-directories that WP may create or use in wp-content. Some like /upgrade are temporary, others like /mu-plugins, /languages, database and caching "drop-ins", debug.log, etc. are permanent. When DISALLOW_FILE_MODS is set to true on a standard install WP will not attempt to create these extra dirs as it will not allow the users to install or upgrade anything through the UI. However note that the return value from wp_is_file_mod_allowed() is filterable meaning that a plugin can change/disregard DISALLOW_FILE_MODS any time/in different contexts.

Unfortunately the docs for wp_is_file_mod_allowed() are quite unclear. None of this is explained there. I hope that as a result of this discussion (at least) these docs will be updated :)

The second seems to be a result from the first. As wp-content/uploads seem to "always" be writable (by WP or directly by PHP), some plugins are using it as a "dumping ground" :)

@samuelsidler
Copy link

The first is more of a coincidence as far as I see. There are several files and sub-directories that WP may create or use in wp-content. Some like /upgrade are temporary, others like /mu-plugins, /languages, database and caching "drop-ins", debug.log, etc. are permanent. When DISALLOW_FILE_MODS is set to true on a standard install WP will not attempt to create these extra dirs as it will not allow the users to install or upgrade anything through the UI.

I don't think that's a coincidence—that was the intended behaviour when DISALLOW_FILE_MODS was introduced in... April 2010, almost 14 years ago, which was further reinforced over the years as new directories were introduced. You may feel that this behaviour is not correct, that WordPress should have not "doubled-down" on this behaviour and instead opened up other directories to being writable, even when DISALLOW_FILE_MODS is set to true.

I think that ship has sailed. This has been core behaviour for almost 14 years, and companies, hosts, and vendors rely on this behaviour. "Breaking" compatibility now does not feel like the correct path for WordPress-as-a-CMS, even if it may feel like the correct path for WordPress-as-a-consumer-product. Unfortunately, we have to balance both, and there is no harm to using /uploads/ for font assets for WordPress-as-a-consumer-product. Further, for the reasons described (a few times) in this issue around fonts requiring a database entry to work, in a way that other assets (like images) do, they are explicitly not treated by WordPress and by this new feature as a peer to /plugins/, /themes/, /mu-plugins/, and /languages/.

@justlevine
Copy link

justlevine commented Mar 6, 2024

This talk about DISALLOW_FILE_MODS has me wondering... If we are discussing from the philosophical-side-of-things, what actually is the default assumption of behavior for fonts when DISALLOW_FILE_MODS is enabled?

Whatever that assumption is, we can adapt the behavior if necessary (as @azaozz noted earlier), but there's something to be said for having our code checks and directory tree follow the same logical structure.

(alternatively, allowing fonts to be autodiscoverable if they're added directly to the fonts folder would also seem to clear all this up. I'm sure that's been discussed in a previous ticket, but I cant seem to find it).

@jazzsequence
Copy link

@justlevine

(alternatively, allowing fonts to be autodiscoverable if they're added directly to the fonts folder would also seem to clear all this up. I'm sure that's been discussed in a previous ticket, but I cant seem to find it).

You mean like this one? 😁 #59102

@azaozz
Copy link
Contributor

azaozz commented Mar 6, 2024

I don't think that's a coincidence—that was the intended behaviour when DISALLOW_FILE_MODS was introduced in... April 2010, almost 14 years ago

Hmm, doesn't seem to be the case? Several of the "other" files and/or dirs in /wp-content that WP may use were introduced before that. Also, imho the commit message puts it quite nicely:

(In [14088]) Introduce DISALLOW_FILE_MOD for disabling all ops that modify core, theme, or plugins files. see #13000

In any case, as far as I see it this is more of a "future proof" architecture decision, coincidence or not.

@johnbillion
Copy link
Member Author

Josepha has published a post on make/core about the location of the fonts directory (and synced pattern overrides).

@peterwilsoncc
Copy link
Contributor

I've closed this issue as unplanned. As a reminder, discussion can continue on a closed issue and participants will continue to be notified.

#59699 has been opened for considering the architectural decisions that are required for the fallback.

@peterwilsoncc peterwilsoncc closed this as not planned Won't fix, can't repro, duplicate, stale Mar 7, 2024
@peterwilsoncc
Copy link
Contributor

After learning a little more about including a fallback within Core by default, its' been decided not to include the feature in 6.5. An announcement post has posted on the Make Core blog.

The original feature announcement dev note has been updated to reflect this change.

@justlevine
Copy link

justlevine commented Mar 22, 2024

After learning a little more about including a fallback within Core by default, its' been decided not to include the feature in 6.5. An announcement post has posted on the Make Core blog.

The original feature announcement dev note has been updated to reflect this change.

Wasn't Josepha's decision for 6.5 based on the ability to make this compromise?

@peterwilsoncc
Copy link
Contributor

@peterwilsoncc peterwilsoncc reopened this Mar 26, 2024
@github-actions github-actions bot added the [Status] In Progress Tracking issues with work in progress label Apr 2, 2024
@creativecoder
Copy link
Contributor

I created a PR to bring WP changes from https://core.trac.wordpress.org/changeset/57878 to Gutenberg: #60354

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature] Font Library Needs Decision Needs a decision to be actionable or relevant [Priority] High Used to indicate top priority items that need quick attention [Status] In Progress Tracking issues with work in progress
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.