Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render dynamic blocks on the front end in JavaScript instead of PHP #14446

Open
iandunn opened this issue Mar 14, 2019 · 29 comments
Open

Render dynamic blocks on the front end in JavaScript instead of PHP #14446

iandunn opened this issue Mar 14, 2019 · 29 comments
Labels
Developer Experience Ideas about improving block and theme developer experience [Feature] Blocks Overall functionality of blocks [Type] Discussion For issues that are high-level and not yet ready to implement. [Type] Enhancement A suggestion for improvement.

Comments

@iandunn
Copy link
Member

iandunn commented Mar 14, 2019

Is your feature request related to a problem? Please describe.

When creating a dynamic block, developers have to duplicate the rendering logic of the edit JS component in a PHP render_callback, which is time-consuming, error prone, and (in theory) unnecessary.

Describe the solution you'd like

The front end should be rendered by JavaScript, reusing the same components that are used to render the block in the editor. In my mind, that would look like this:

  1. PHP could render the initial front end page using the_content() like it does now, and the block's metadata comment would be left in tact -- e.g., <!-- wp:wordcamp/speakers {...} -->
  2. A JS function would scan the page content looking for block comments, and asynchronously renders them, injecting the rendered block into the DOM in place of the comment. In this context, the editing controls/toolbar would be removed.

To clarify, that would be the opposite of using ServerSideRender; both the front- and back-ends would be written as components and rendered by JavaScript.

I realize that we can't have the entire front-end page served from JavaScript, because most servers won't have Node.js available, but it seems like we could have a client-side script render the blocks asynchronously after the the PHP page has loaded.

What that rejected for performance reasons? Are there use cases where the editing and viewing experiences are so disparate that they can't reuse the same components? Am I missing some other fundamental flaw with that approach?

I'm guessing this has already been discussed and rejected for one reason or another, but I searched for a long time and couldn't find any previous discussions. If that's the case, can you provide links to those discussions, to make them more discoverable?

@iandunn iandunn added [Type] Enhancement A suggestion for improvement. [Type] Question Questions about the design or development of the editor. Needs Technical Feedback Needs testing from a developer perspective. labels Mar 14, 2019
@iandunn
Copy link
Member Author

iandunn commented Apr 2, 2019

I discussed this a bit with @aduth, and want to summarize that so we can continue the discussion here.

Andrew explained that Gutenberg was designed with a fundamental distinction between the editing and saving modes, which is also reflected in how static blocks have separate edit and save functions.

The ideal editing experience will not always look like a preview of the final content. They often look similar in simple cases, but can diverge greatly in more complex cases, and are fundamentally different concepts. He referred to #104 for some background on the early formation of that distinction, and felt like reusing the edit function for preview would quickly become messy in practice.

After considering that, and playing around with an example, I agree that trying to reuse edit would not be practical. I do still feel like there's a way to vastly improve the developer experience, though.

For static blocks, the separation between edit and save is usually not that painful, because the developer is working in the exact same environment, and can reuse the vast majority of the code they wrote for edit.

For dynamic blocks, though, that's not true, and the developer is forced to re-write everything from scratch, in a different language, using different APIs, taking into account different esoteric security considerations, etc. It's an awful developer experience, and has a huge opportunity cost because the developer has to waste a ton of time reinventing the wheel, manually keeping the two sides in sync when changes are made, and fixing bugs that inevitable arise from the error-prone nature of the separation. Instead of doing all that, developers should be able to spend that time building new things that make an impact for their organizations.

The primary source of that pain is the fact that the rendering for dynamic blocks is done in PHP instead of JavaScript. If we could do it in JavaScript instead, then we could reuse components like we do for static blocks. There'd be some overhead and duplication, similar to save, but it'd be minimal, and a huge improvement from the current situation.

One way to do that would be to add a render.js file inside the block folder to define the rendering markup. The block's save function would render a spinner as a placeholder, and that would appear when the page first loads. The build would generate a separate render.min.js file, which would be enqueued via wp_enqueue_script, and once the DOM is ready, a function in render.min.js would be called, and it'd replace the spinner with the real content.

render.js could import only what it needs from the rest of the block, in order to make itself as lean as possible.

That could eventually be taken a step further, and optional support for Node could be added, so that in professional environments the content could be pre-rendered and bundled into the initial page that gets sent to the visitor. It could even integrate WP Cron, to periodically prime a cache of the markup asynchronously, and save that in post_content (along with the meta-data), so that the pre-rendering doesn't slow down the TTFB.

@iandunn iandunn changed the title Reuse editing components to render dynamic blocks on front end, instead of duplicating logic in PHP Render dynamic blocks on the front end in JavaScript instead of PHP Apr 4, 2019
@felixarntz
Copy link
Member

Andrew explained that Gutenberg was designed with a fundamental distinction between the editing and saving modes, which is also reflected in how static blocks have separate edit and save functions.

Agreed, preferably the variant used in edit should provide interactive controls right where the final content would appear, to provide a more intuitive in-content editing experience.
However, for several dynamic blocks, as mentioned this is not viable.

The primary source of that pain is the fact that the rendering for dynamic blocks is done in PHP instead of JavaScript. If we could do it in JavaScript instead, then we could reuse components like we do for static blocks. There'd be some overhead and duplication, similar to save, but it'd be minimal, and a huge improvement from the current situation.

I would like to point out the possible role Web Components could play here. I think they are a viable alternative as they would follow a component-based model close to the one that React uses, but at the same time would be agnostic of the environment so that they could as easily work in the frontend as they would in the editor. Some remarks:

  • Web Components can be used inside React, especially on the leaf component level.
  • They are browser-native, so would easily work in the frontend (in a few cases still requires a polyfill, but support is improving steadily).
  • They can include additional markup, JavaScript and styles so dynamic functionality is covered.
  • They could even be printed in PHP if the server wanted to print markup with the same dynamic functionality directly.
  • They are context-agnostic and reusable so other projects could benefit from the ones Gutenberg would create and vice-versa.

I explored usage of Web Components in Gutenberg block types in a POC, where I re-created the Latest Posts block type in an alternate variant: https://github.com/felixarntz/web-components-in-gutenberg/blob/master/assets/block-types/latest-posts.js is the block type definition, while https://github.com/felixarntz/web-components-in-gutenberg/blob/master/assets/custom-elements/wordpress-post-list.js is the Web Component definition used by the block type.

@gziolo
Copy link
Member

gziolo commented Apr 8, 2019

I also want to emphasize that blocks like Latest Posts with edit written in JavaScript and save rewritten in PHP is extremely confusing for new Gutenberg contributors. I had chats with @AmartyaU and @draganescu on this topic in the last two weeks, and it seems like they both were surprised learning that you still need to code the same logic in PHP to make it work.

@felixarntz
Copy link
Member

A considerable downside to most likely any approach that solves this in a pure-JS way is that the frontend would need to make API requests to get the data rather and dynamically create the markup which decreases performance and provides a worse user experience than when seeing the content immediately.

The DUX would totally be improved by getting rid of PHP here, but I think we need to carefully weigh the advantages/disadvantages against each other - after all the users come first, and for them the requirement we have for PHP doesn't matter.

@iandunn
Copy link
Member Author

iandunn commented Apr 9, 2019

That's a really great point. I wonder if there's a way that the block metadata which gets stored in post_content could include some query arguments, and then the server could include the JSON results of those queries in the initial HTML response?

@iandunn
Copy link
Member Author

iandunn commented Apr 10, 2019

Fetching any data that a block needs and bundling it with the initial HTML response would also benefit the editing experience, since it would make the initial render a lot faster by avoiding the HTTP requests that would otherwise be necessary.

@gziolo
Copy link
Member

gziolo commented Apr 10, 2019

Gutenberg does prefetching for the most important REST API endpoints and stores results in cache of wp.apiFetch middleware. This makes it possible to avoid triggering additional network requests on initial page load.

@coreymckrill
Copy link
Contributor

Gutenberg does prefetching for the most important REST API endpoints

Is it possible to augment the list of endpoints that gets prefetched?

@gziolo
Copy link
Member

gziolo commented Apr 11, 2019

Is it possible to augment the list of endpoints that gets prefetched?

Yes, it looks like there are ways to do so already:

https://github.com/WordPress/wordpress-develop/blob/17146f7192cd1c63ccde76be93d75465ccd32900/src/wp-admin/edit-form-blocks.php#L42-L64

It's probably targeting only admin page which renders Gutenberg.

@aduth
Copy link
Member

aduth commented Apr 11, 2019

A considerable downside to most likely any approach that solves this in a pure-JS way is that the frontend would need to make API requests to get the data rather and dynamically create the markup which decreases performance and provides a worse user experience than when seeing the content immediately.

Important to note the SEO implications here as well, as even with Google's crawling of JavaScript content having improved over the years, it remains imperfect from my most recent understanding, and is a reason why services like prerender.io exist.

@draganescu
Copy link
Contributor

Because of "circumstances" I have some good experience with server side rendered SPAs. Since we don't enjoy the luxury of running JS on the server, then, in my opinion, I don't think we'll be able to escape using PHP to render content and serve its initial state in the response.

It is both a matter of SEO and one of UX: having content in the page is much better than spinners and flickers or page blocks jumping because they get content later. Of course, everything can be solved client side only, but, in my experience, in a much more complicated way than simply rendering on the server.

In our case, the problem is not that we want to "get rid of PHP", which is a non starter, but to make it such that we don't need to write the same logic twice in two different programming languages, better put: to be able to skip this step as often as possible.

And that's what meta data is for: to explain between environments what's happening so that a generic agent can do its thing without all the implementation details.

I think @iandunn 's idea with having some more meta data ("query args") in the post_content for dynamic blocks could be a good direction. However a solution would require access to the template of the content, and since there is no such notion of template anymore, b/c components have the markup in the render function, we need to do more work.

Imagine the system would somehow have used Handlebars templates or similar, then PHP would only need one generic filterable render routine, for which the editor would embed arguments and the template location in the post_content.

We don't have Handlebars or similar, however, most likely a solution would be on the same line: somehow having blocks providing a template which is server render-able and the pointer to data. At least for what can be done this way, while still supporting server side render callbacks for extremely advanced situations which I assume are rare and worth the effort.

In other words: we shouldn't aim to "Render dynamic blocks on the front end in JavaScript instead of PHP" but instead aim to "Improve dynamic blocks' PHP rendering to avoid code duplication".

For example, and this is a ballpark solution, the latest posts block could provide as block meta data two things: the loop's template (in a parseable form) and the rest endpoint where the latest posts are. It's trivial to render server side then, without any matching, just by using the keys of whatever REST returns as template variables. The key here is to have a way to extract this template automagically from the save / edit renders :) which is something I have no idea if it can be done.

Of course, this is not a universal solution but it is another step into having less and less need to do the duplication. This kind of approaches incrementally move closer and closer to not needing to do "custom" SSR, but still having that server render callback feature in place as it still is a very good feature, in any environment where GTB is used but there is no JS on the server.

@felixarntz
Copy link
Member

For example, and this is a ballpark solution, the latest posts block could provide as block meta data two things: the loop's template (in a parseable form) and the rest endpoint where the latest posts are. It's trivial to render server side then, without any matching, just by using the keys of whatever REST returns as template variables. The key here is to have a way to extract this template automagically from the save / edit renders :) which is something I have no idea if it can be done.

I like this approach a lot, as it's a solid compromise between UX and DUX here. We could define a schema for how dynamic data should be handled in Gutenberg which (as said above) would need to follow three key principles:

  • The block type must specify a REST endpoint that returns the data in exactly the right format (so that no further JS/PHP processing is needed).
  • The block type must define mappings between block attributes and whether/how to send them to the REST API request.
  • The block type must define a template that will be used both server-side and client-side (in a templating language we will decide on that works in both JS and PHP). The REST API response data is directly passed through to that template.

This should cover almost everything - a consideration would be things like date formatting for example: We can theoretically return dates in a formatted way from the REST API, but doing that would really take us away from the structured approach REST APIs should typically have. Although, to be fair, having block type-specific REST endpoints already does that.

@gziolo gziolo added [Type] Tracking Issue Tactical breakdown of efforts across the codebase and/or tied to Overview issues. [Feature] Blocks Overall functionality of blocks and removed Needs Technical Feedback Needs testing from a developer perspective. [Type] Question Questions about the design or development of the editor. labels May 24, 2019
@ddluc
Copy link

ddluc commented Sep 2, 2019

Hi folks,

I'm a long time react developer working with Wordpress and Gutenberg for the first time and find that having to resave my page/post that uses my custom Gutenberg block every time I make an update to be incredibly painful. My "block" is fairly complex and needs to be broken down into multiple sub-components. But, any time the markup for any component in my tree is updated, my block is invalidated.

I've bypassed this pain by setting up a local dev server within my plugin directory, but even that doesn't solve the problem that once I publish the block, if I need to make any changes, every instance of the block on the site needs to be manually updated.

Why not leave it up to the developer to choose whether the block is prerendered? So, instead of two attributes (i.e. save and edit) you could have a third method, say ,render, that allows the block to be rendered on the front end and bypass the block validation?

import './components/MyComponents`. 

registerBlockType( 'example/block', {

  ...

  edit: ( props ) => {
    return (
	<div>Editor Block View</div>
    );
  },

  save: ( props ) => {
    return null
  },

  render: (props) => {
    return (
      <MyComponent />
    )
  }

} );

I don't know enough about what is happening under the hood, but presumably, there is some way to run this render method on the front end and bind it to an html root element for the block:

ReactDOM.render(
 // psuedo code where my block's render method is saved as a reference to be 
 // used by wordpress 
  wp.block.referenceToMyBlock.render(); 
  document.getElementById('')
);

@achatainga
Copy link

I have to second what @ddluc is saying. I have been working with React and React Native for a while and trying out Gutenberg for the first time... It turns out my plugin is designed to show a custom React Form and do API calls to login/register/recover_pass a user before displaying the content. Everything works so well until I get to the validation part. WordPress just is not disabling it.

Using a third render as stated above would be the perfect solution for my use case since I could avoid this validation errors and proceed...

The data I saved from the block is encrypted and many more complex things are happening and I am wondering if I have to through away the idea of using wordpress altogether because of this

@mintplugins
Copy link

Here's an approach I have taken for my blocks thus far to avoid building in both PHP and JS, and for me it has worked pretty well.

  • All of my blocks are dynamic (rendered via PHP)
  • But the only things I render are:
      1. A div with a unique ID
      1. Within that div, I output the variables for my block settings as JSON, wrapped in a div set to display:none.
  • Upon page render, I use React-Dom to find-and-replace that div with a react component, and I pass that hidden JSON as a prop for the initial state of my react component.
  • My react component has an "edit_mode" flag as part of the state, which conditionally outputs editing elements, and passes the values back up to Gutenberg for saving.

@ddluc
Copy link

ddluc commented Mar 17, 2020

@mintplugins That is not a bad temporary solution until a more permanent solution is in place—I'd love to see how you've configured your plugin and webpack build to make this happen.

Do you have any open-source plugins or examples I could refer to? I think this solution would make a really good blog post. As I mentioned above, I'm more of React developer than a WP developer, but a couple of projects I work on have CMS requirements and this is the first problem I have run into, so I imagine there might be a lot more people out there that are looking for a more passable solution to this other than rendering blocks in PHP.

@dgwyer
Copy link
Contributor

dgwyer commented Mar 17, 2020

@ddluc Another consideration, if you're rendering blocks on the frontend via React, is SEO. I initially thought it would be a great idea to be able to reuse React components on the frontend for ALL my blocks.

However, depending on the block type this might not be a good idea. For block types like galleries, sliders, and so on where content is mainly visual it's a good fit. But if you're outputting text then I don't know how this affects SEO performance?

The solution @mintplugins posted is an interesting one as the content is rendered as attributes on a hidden element. So it is there on page load in some form, albeit not the final semantic structure. So, again the SEO impact needs taking into account.

@skorasaurus
Copy link
Member

For what it's worth, I felt completely in the same boat as @gziolo mentioned... I had been wondering whether was some rule that I didn't find documented anywhere (why where some blocks in JS, part in php?)

@iandunn thank you so much for sharing your chat.

@youknowriad youknowriad added [Type] Discussion For issues that are high-level and not yet ready to implement. and removed [Type] Tracking Issue Tactical breakdown of efforts across the codebase and/or tied to Overview issues. labels Jun 4, 2020
@gziolo
Copy link
Member

gziolo commented Sep 10, 2021

@ivanjeremic opened an issue that I closed as duplicate: #34718.

Make React work in the front-end with blocks out of the box
I personally think the reason why React developers struggle with this is React is an amazing tool for building interactive UI and the fact that Blocks are not interactive in the front-end is somehow a missed opportunity considering how manny react developers are out there, something like this build in would not only boost the block development since more react devs would come in, it will also help create amazing new things with Gutenberg and blocks.

@fabiankaegy
Copy link
Member

I wonder whether the new SSR features in React 18 could be the way that we could use to start getting dynamic blocks to also be written in JS.

reactwg/react-18#37

One of the main downsides if we just switch to rendering the dynamic content is the additional server requests and the initial render without the data. So the current approach of having it in PHP actually is kind of nice because it essentially is a Server Side Rendering of the component. The downside is that we need to replicate all of the logic in PHP again.

Of course, just executing js on the server is not really feasible because the typical WordPress hosting environment is not set up for that. And the way that integrates into core would probably be very clunky. And I myself am not familiar enough with the underlying architecture of the pipeToNodeWritable function in react 18 would be something we can use to jump-start this effort of having "Server Components" that don't rely on API calls and are server-side rendered within a react application on the frontend that is fully hydrated and interactive.

(Maybe / probably not feasible at all. This is very much half-knowledge and not deep understanding of the issues here :) )

@landwire
Copy link

I really don't care whether a block is rendered with JS/React or PHP or whatever, but what I care about is that the block's markup is written in a well structured templating language. The blocks data is passed to the template and the template consumes it.

Therefore until now I mainly use ACF to build my blocks and PHP/Twig to render them. This gives me the freedom to override a block's markup in every child theme I create and output a totally different design based on the same data. Being able to override a block's markup easily and dynamically (without the need of re-saving it) is crucial for my development workflow with ever changing designs for the same data.

@gziolo gziolo added the Developer Experience Ideas about improving block and theme developer experience label Oct 14, 2021
@gziolo
Copy link
Member

gziolo commented Oct 14, 2021

@landwire, thank you for sharing your feedback. I think it very much aligns with what some other developers struggle with and that's why dynamic block rendering with PHP has been so popular in the community. In fact, it's also what many core blocks from the widgets or theme category use internally to account for their dynamic nature.

I can also see how blocks saved as static HTML might cause troubles when the requirements for the HTML structure change. It's impossible with the à la carte WordPress instance to update previously published posts/pages to use the most recent version of the block. Those blocks will technically work but that might not meet the expectations of the site owner. We definitely should seek ways to address that.

Of course, just executing js on the server is not really feasible because the typical WordPress hosting environment is not set up for that. And the way that integrates into core would probably be very clunky. And I myself am not familiar enough with the underlying architecture of the pipeToNodeWritable function in react 18 would be something we can use to jump-start this effort of having "Server Components" that don't rely on API calls and are server-side rendered within a react application on the frontend that is fully hydrated and interactive.

@fabiankaegy, some good thinking on your side. It's definitely something that has been discussed briefly also here on GitHub in some random places. It's a very difficult challenge to tackle because WordPress uses PHP on the server. Anyway, the plan is to follow closely how the new APIs planned for React 18 shape and how it integrates with Node.js-based frameworks.

@Aljullu
Copy link
Contributor

Aljullu commented Nov 15, 2021

WooCommerce has a system to render React-based (or JS-based) blocks in the frontend. There is an explanation of how it works in this post: 'How does WooCommerce Blocks render interactive blocks in the frontend?'.

I'm sharing it here in case it's helpful for other projects until there is a Gutenberg-supported way of achieving this. Happy to answer any questions.

@ivanjeremic
Copy link

Is there any work done to tackle this issue? I think react server components come with react18 from what I heard and I think a node server is needed(also not sure about that), but if yes I think PHP should start to ship Node in newer versions of PHP :D

@aristath
Copy link
Member

While rendering blocks on the frontend using JS/React would significantly improve the developer experience, at the end of the day server-side rendered blocks are faster and they don't require anything extra to be loaded on the browser.
From a performance and sustainability perspective, SSR is a lot better.
If a developer wants to use client-side rendered components, there's nothing stopping them from doing that. It's very possible to implement it, it's just not a method that is available out of the box because improved performance and user experience come first in WordPress.

@ivanjeremic
Copy link

ivanjeremic commented Jan 18, 2022

While rendering blocks on the frontend using JS/React would significantly improve the developer experience, at the end of the day server-side rendered blocks are faster and they don't require anything extra to be loaded on the browser. From a performance and sustainability perspective, SSR is a lot better. If a developer wants to use client-side rendered components, there's nothing stopping them from doing that. It's very possible to implement it, it's just not a method that is available out of the box because improved performance and user experience come first in WordPress.

Important to note is that SSR is only faster on slow networks on fast network CSR is faster. But regarding SEO sure SSR wins for now but it is also a lot of work done by search engines to work better with client code, they have improved a lot.

@michalczaplinski
Copy link
Contributor

I have created a discussion about blocks that can be rendered both on the client and server, which I have called "universal blocks". It summarizes most of the ideas mentioned in this thread.

There, I also show a proof of concept of a block that is authored using Mitosis and compiles to both JS (for frontend) and a Liquid template (for PHP backend). This demo is, of course, just an interesting demo to stimulate more research and discussion and not a serious attempt at something that Gutenberg should adopt.

@jc365
Copy link

jc365 commented Feb 11, 2023

i'm a rookie programmer but i'm interested in this discussion
it's important for me to find a solution

@iandunn
Copy link
Member Author

iandunn commented Oct 25, 2023

The proposed Interactivity API could be a way to solve this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Developer Experience Ideas about improving block and theme developer experience [Feature] Blocks Overall functionality of blocks [Type] Discussion For issues that are high-level and not yet ready to implement. [Type] Enhancement A suggestion for improvement.
Projects
None yet
Development

No branches or pull requests