💬 RFC: Frontend System Evolution

[Rugvip]

🔖 Need

This RFC outlines our proposal for evolving the Backstage frontend system towards what we have been calling “Declarative Integration”. There are a number of reasons for us to focus on this work.

Integration Without TypeScript

Removing the need for writing TypeScript to integrate plugins has been the single requirement driving all of this work. Since the start of the Backstage project you have needed to write TypeScript in order to integrate installed plugins into an app. One clear downside of this is that you need to know TypeScript if you want to manage a Backstage installation, and there are a significant number of APIs that you need to learn even for trivial customization. The existing frontend system is designed in a way where we hoped that TypeScript knowledge was not required, but in practice that hasn’t worked out. What we want to move towards is a system where this initial API surface is much more slim and centered around configuration, and TypeScript knowledge is only needed when you want to start making deeper customizations to an app or to build new plugins and features.

Simplify Installation and Upgrades

One of the most important outcomes of this work is to standardize and simplify the installation of plugins. Installation instructions in plugins can get quite verbose, especially when there are extensions that need to be installed on the entity pages. While we hoped to arrive on a small set of standardized extension types with centralized installation instructions, this is not what happens in practice. We have learned that it needs to be possible for installation instructions to be self-contained, users do not want to bounce between sites to figure out how to install a plugin. It should be clear how to install and configure a plugin from the plugin README alone, and those instructions should not need to be overly verbose, complex, or change significantly over time.

Furthermore, since an app is all built up and integrated with code, upgrades can become quite complicated. It is also hard to provide upgrade automation tools, since the codemods required for many upgrades end up being very complex. By moving to a declarative system, it is both more likely that we can embed more behavior in the plugins themselves, but also much easier to create codemods and other tools to help out with migrations.

Sane Defaults

Our current plugin installation procedure is very explicit. It’s designed this way to make it entirely clear what the structure of the app is and where each piece of content is placed. One thing it does particularly badly however, is to allow for plugins to provide sane defaults.

We would like for plugins to be able to provide defaults to the furthest possible extent. For example, for any plugin that installs a page on a root route you can assume that users almost always want that. It is of course important to be able to override these defaults too, allowing for things like disabling a route that’s added by default, or exposing the route on a different path.

More Flexibility than JSX

The fact that extensions that build up a Backstage app need to be React components can be quite awkward. It has led to situations where extensions contain a placeholder component that doesn’t render anything, just so that they can be included in the app. While this doesn’t look terrible, it gets more awkward and difficult to understand the further you venture away from React. Structures such as the ScaffolderFieldExtensions and TechDocsAddons where plugins need a way to integrate a distinct category of extensions start getting a bit strange and also quite tricky to implement correctly. We have found that it is desirable to have this built into the framework itself, rather than something that plugins need to implement themselves.

While the JSX structure might look familiar to those that know React, it’s also quite limited in many ways and can be quite verbose.

Remove React Element Confusion

A very common point of confusion when building a Backstage frontend is the “app element tree”, and in particular that you shouldn’t render Backstage plugin extensions in a React component. When an extension is used outside of the app element tree it means that the Backstage app is not able to discover the extension or anything else that it provides, for example routes or a plugin. This can lead to breakages in the app, such as a Utility API not being available. Because Backstage plugin extensions are also React components, they do sometimes function without issues even outside of the app element tree, which leads to even more confusion.

Enable Dynamic Plugins

While this RFC does not in itself propose a solution for how to dynamically install and manage plugins in Backstage, we want to build towards that goal. Being able to dynamically install plugins unlocks new ways of deploying and managing Backstage, and has the potential to hugely improve adoption by lowering the barrier of entry. A Backstage installation currently requires quite a lot of care to maintain, meaning it may not be worth the investment for smaller organizations. This is something that we hope that both an evolved frontend framework as well as future dynamic plugins can help solve.

Larger organizations that still wish to manage a Backstage installation through code can still greatly benefit from a more dynamic plugin integration. In particular, the ability to load plugins from multiple different sources at runtime can help scale the ownership of a Backstage installation, with separate teams managing the lifecycle of their own plugins independently.

Quick reminder that this RFC does not propose a solution for dynamic plugins, that is something for the future.

🎉 Proposal

Our proposed solution is a system where every app is built up of a set extension instances declared in a flat configuration structure. For example, the extension instance for the TechRadar page might look like this:

- tech-radar.page:
    at: core.router/routes
    extension: '@backstage/plugin-tech-radar#TechRadarPage'
    config:
      path: /tech-radar
      width: 1500
      height: 800

The 'tech-radar.page' extension instance is "attached" to the 'routes' input of the 'core.router' extension instance. It uses the specified extension as its implementation, which also gets passed the provided configuration. The declared structure is flat, but using the attachments as edges it ends up forming a tree that is quite similar to the React tree used in an existing Backstage app.

It would be a lot of work for everyone to declare all of these extensions manually in each app, so that is not what our proposed solution looks like. Instead, a minimal app declaration would look something like what we have below. Don’t worry, we’ll dive into all the details of this configuration later.

app:
  packages: 'all'

  routes:
    catalog:
      createComponent: scaffolder/root
    scaffolder:
      registerComponent: catalog-import/importPage

  extensions:
    - core.nav:
        config:
          layout:
            - label: Search
              icon: search
              to: /search
              items:
              - input: search
            - type: divider
            - label: Menu
              icon: menu
              items:
                # ... more items
            - type: space

    - core.nav/search: '@backstage/plugin-search#SidebarSearchModal'

    - core.router/routes:
        extension: '@backstage/core-app-api#Redirect'
        config:
          path: /
          to: /catalog

    - techdocs.page/addons: '@backstage/plugin-techdocs-module-addons-contrib#ReportIssue'

    - search.page/content: app#customSearchPage

    - entity.card.orphanWarning
    - entity.card.processingErrors
    - entity.card.about
    - entity.card.catalogGraph
    - entity.card.links

    - entity.content.apiDefinition
    - entity.content.componentApiRelations
    - entity.content.dependencies
    - entity.content.systemDiagram

To understand this configuration better, let’s first have a look at the underlying architecture.

Extension Architecture

The deep dive into this system architecture will wait for eventual architecture documentation, but to help in understanding how it all is supposed to fit together we skim the surface. It should also be noted that this architecture is what we came up with as a potential implementation, but the main focus of this RFC is the app integration experience.

The system is built up of a couple of core concepts:

• Extensions: An extension is the implementation of a piece of functionality that is exported by a plugin. Each extension has a configuration schema used to describe its configuration, as well as an output and zero or more inputs.
• Extension Output: The output of an extension is the facility that an extension uses to contribute functionality to the app.
• Extension Inputs: An extension can optionally have inputs that can be used to consume the outputs of other extensions. Inputs can be constrained to require specific data to be present in any consumed outputs.
• Extension Data: This is the concrete link between extension inputs and outputs. It is analogous to component data in our existing frontend system. Some extension data types are defined by the core framework, but additional types can be defined by plugins that wish to create their own extension points.
• Extension Instances: Extensions themselves are akin to blueprints, to actually use an extension in an app you need to create an extension instance. Each extension instance has its own unique ID, and selects an extension to use as its implementation, an attachment point in the app, and its own configuration.
• Attachments: To make an extension instance part of an app you need to attach it somewhere. You do this by specifying the ID of an extension instance, as well as the name of the input on that extension that you want to attach to. Once an extension is attached to an in input, it will share its output extension data to the parent extension.

Below is an architecture diagram that shows the shape of extensions, as well as how extensions are used to create extension instances that integrate into the larger app hierarchy.

At some point we of course must hit the root of this tree, which is the "app root" extension instance, provided by the framework itself. Only extension instances that are directly or transitively connected to this root extension instance are considered to be part of the app.

The following diagram is a visualization of what the extension instance tree of a tiny app might look like. Note the different types of inputs and outputs of each extension instance, and the fact that many extensions don’t have any inputs at all.

One layer above this architecture, the core framework would also provide a set of built-in extension instances. For example the ’core.router’ referenced many times in this RFC would be provided by the framework itself, along with other instances. The exact set or shape of these built-in extensions is something that we plan to explore later, assuming we move forward with this design.

Similarly to the built-in extension instances, we also plan to have a set of built-in extension data types. In the diagrams we use "React Component", "Route Path", and "Display Title". These are representative examples of what we imagine the initial set might look like. This set of built-in extension data types both help create standardization across the Backstage ecosystem, but it also allows us to discover this data throughout the app. We envision that this will allow us to both build powerful app-wide features, as well as tools that help visualize or otherwise interact with the entire app structure.

If you’re curious and want to further explore how we imagine that this all fits together, you can check out or experimentation branch. The app-experiments package contains a few implementation experiments that led us to this architecture, while the app-next package contains the beginning of a real-world implementation.

Default Extension Instances

A critical part of this new system is the ability for plugins to provide default extension instances to the app. When defining a plugin, you can also provide a list of default extension instances, for example the GraphiQL plugin might look as follows:

const graphiqlPlugin = createPlugin({
  id: 'graphiql',
  defaultExtensionInstances: [{
      id: 'graphiql.page',
      at: 'core.router/routes',
      extension: GraphiqlPageExtension, // direct reference in code
      config: { path: '/graphiql' },
  }],
});

This means that the 'graphiql.page' extension instance is automatically installed in an app as soon as you install the GraphiQL plugin. If that’s all you want there is no further configuration required, but there are also many ways that you can customize this instance in the app.

For example, you could change the path that the page is mounted at:

  - graphiql.page:
      config:
        path: /graphql-explorer

If for some reason you don’t want the extension to be present, you can disable it:

  - graphiql.page: false

You can change the attachment point of the extension:

  - graphiql.page:
      at: tools.router/routes

Or replace the extension with your own customized implementation:

  - graphiql.page:
      extension: ‘@internal/custom-graphiql’

You can of course also combine any of these actions as well, for each extension instance exported by any plugin or the core framework.

Ordered Extension Instances

There is a particular category of extension instances where ordering is important. For example cards and tabs on the catalog entity pages. We tried many different approaches for how to handle this, for example additional configuration to determine order, or a default ordering with some form of overrides and opt-out.

In the end we found that in combination with default extension instances, the best approach is to always explicitly list all of these extensions and then maintain that order. To do this, we added the ability to specify that a default extension instance should be disabled by default. That way plugins can still provide the base configuration for each instance, but the app determines which instances are used, and in what order. For example, the cards on the catalog entity page might be configured like this:

  - entity.card.about
  - entity.card.catalogGraph
  - entity.card.links

Only these three cards would be present on entity pages, and they would be presented in the same order.

The benefit of this approach is that it keeps all configuration simple, for any other approach the complexity introduced by just straying off the default path even a tiny bit can be massive. The tradeoff of course is that you end up needing to have this explicit list in configuration.

Entity Pages

A particularly complex part to tackle as part of this design was the catalog entity pages. What we found in the end was that the best approach was to simply not attempt to replicate the different types of entity pages that exist in an app currently, for example the "service component page", or "user page". By flattening out the entity pages to a single list of content we found that we end up hitting a sweet spot of flexibility and sane defaults.

Every card and tab content on the entity page defines its own set of filters for where it should be shown. The exact format of how these filters are defined is not something that we explored thoroughly yet, but it may look something like this:

  extensions:
    - entity.card.userProfile:
        at: entity.page.overview/cards
        extension: '@backstage/plugin-org#EntityUserProfileCard'
        config:
          if:
            anyOf:
              - isKind: user
              - isKind: group

With this approach we found that adding additional custom entity kinds and types has the least impact, with it often being compatible with existing configurations in plugins. In the event that the filters need to be reconfigured, you would only need to apply that updated configuration to those particular extension instances.

Extension Configuration Schema

This section focuses on the app.extensions part of the configuration, which is where you define your own extension instances and overrides. While the internal data model of this architecture will have a different shape and be more explicit, it is important that the configuration defined by integrators in their app is clear and concise. This entire architecture was formed from the point of view of the app integration experience, and you can see the steps we took to reach this design in our experimentation repo.

The most verbose configuration format is the first one we saw in this RFC:

  extensions:
    - <id>:
        at: <id>/<input>
        extension: <implementation-reference>
        config: <configuration-object>

In addition to this schema, there are a number of shorthands, listed in no particular order:

You can enable an individual extension just by ID, note that the array item is a string rather than an ID in this case:

  extensions:
    - ‘<id>’

You can enable/disable individual extension by ID, in this case the value is a boolean:

  extensions:
    - <id>: <true/false>

You can override the implementation of an extension by ID, in this case the value is a string:

  extensions:
    - <id>: ‘<implementation-reference>’

You can create a new extension instance with a generated ID by including an input name in the key:

  extensions:
    - <parent-id>/<parent-input>:
        extension: <implementation-reference>
        config: <configuration-object>

This syntax is only for use in the app configuration itself, every extension provided by default from a plugin must have an explicit ID. For example, the following two configurations are equivalent, except that the former does not have an explicit instance ID:

  extensions:
    # Generated ID
    - core.router/routes:
        extension: '@backstage/plugin-tech-radar#TechRadarPage'
    # Explicit ID
    - tech-radar.page:
        at: core.router/routes
        extension: '@backstage/plugin-tech-radar#TechRadarPage'

Lastly, if you do not need to provide additional configuration, you can combine the key input format with the implementation value format as a shorthand for creating a new extension instance with a generated ID and no configuration:

  extensions:
    - <parent-id>/<parent-input>: ‘<implementation-reference>’

For example:

  extensions:
    - core.router/routes:  '@backstage/plugin-tech-radar#TechRadarPage'

Complete App Configuration

Let’s look back at the app configuration from the beginning of the RFC, this time with annotations:

app:
  # This indicates a piece of magic where you automatically detect plugin packages from package.json
  # and include them in the app. We initially avoided this because package.json has a fixed ordering,
  # but with our new approach to ordering extension we are able to add this utility.
  # We will of course also be providing explicit options here, allowing you either to list all plugin
  # packages, or exclude some packages.
  # It’s worth noting that this will need an integration with the build system to make sure plugins
  # are included in the bundle and loaded at runtime.
  packages: 'all'

  # We will also need to move over the external route bindings currently configured in createApp().
  # This is more or less a 1-to-1 mapping, but as we discuss below we may end up making the values
  # here extension instance IDs instead
  routes:
    catalog:
      createComponent: scaffolder/root
    scaffolder:
      registerComponent: catalog-import/importPage

  # And now for the tricky bit
  extensions:
    # This is one way in which we envision the sidebar taking form. It’s something that we can leave
    # for later design though, and there may be multiple options here too
    - core.nav:
        config:
          layout:
            - label: Search
              icon: search
              to: /search
              items:
              - input: search # This input is created dynamically
            - type: divider
            - label: Menu
              icon: menu
              items:
                # ... more items
            - type: space
    # Using the dynamic input we created above, provide an instance
    - core.nav/search: '@backstage/plugin-search#SidebarSearchModal'

    # The index redirect might be handled like this instead
    - core.router/routes:
        extension: '@backstage/core-app-api#Redirect'
        config:
          path: /
          to: /catalog

    # Generated ID shorthand used to install individual TechDocs extensions.
    # It remains to be seen whether plugins themselves are extension instances, it may make more
    # sense to install these kinds of extensions directly on a plugin instead.
    - techdocs.page/addons: '@backstage/plugin-techdocs-module-addons-contrib#ReportIssue'

    # The new pattern for customizing plugin pages would look something like this. In particular, we’ll
    # likely want some lightweight way to provide additional extensions directly in the app that can be
    # referenced in config.
    - search.page/content: app#customSearchPage

    # Explicitly listing all the entity page cards that we want to show
    - entity.card.orphanWarning
    - entity.card.processingErrors
    - entity.card.about
    - entity.card.catalogGraph
    - entity.card.links

    # As well as all the entity page content, although the overview page is always enabled and the
    # first tab by default. It can be listed here for a different ordering though, or disabled.
    - entity.content.apiDefinition
    - entity.content.componentApiRelations
    - entity.content.dependencies
    - entity.content.systemDiagram

〽️ Alternatives

Fixed Attachments

An option that we have considered is to make the output attachment point of extensions fixed, instead of leaving it to be configured. In particular this would mean that the ecosystem of plugins becomes more fixed, leading to less flexibility in how you can use certain extensions. For example, we might enforce that the AboutCard extension from the catalog must be placed on the CatalogEntityPage. If one wanted to re-use the AboutCard anywhere else in an app they would need to create their own extension using the underlying implementation of the AboutCard extension, which may or may not be available.

The benefit of this approach is that with a more fixed ecosystem we can make more assumptions about the structure of an app. In particular we can assume that a certain extension instance is mounted on a particular route, including specific route parents.

In the end we currently believe that this is not a significant simplification that will provide any considerable benefit, and that more flexibility in the use of extensions will allow for more and simpler re-use.

Fixed Input/Output Shapes

In our experimentation towards this new system we considered extension input and outputs with individual distinct types, rather than the collection of different extension data that we are proposing. For example, one might have an "entity page content" extension output type that would be consumed by the entity page. We opted for a more loose approach because it composes well, and in particular it allows for us to have a fixed set of data types as part of the core framework that we can use to introspect the app.

For example, we can implement a routing system by having a shared extension data type for route paths, which the app can then collect and use to build the routing tree. If the routing information is mixed into a distinct type with other properties, we would need some additional layer that helps us collect the routes from all of the different extension output types in the app. It also turned out that the extension data approach is analogous to our existing component data system, which we feel has been working well and has been easy to evolve.

❌ Risks

Migration

These deep core framework changes are always going to have an impact. We will strive to keep this impact as low as possible, and have a very large migration window. We have already identified a couple of tools that can help us achieve this.

One feature that we want to include in this new system is the ability to easily jump back and forth between code and declarative app wiring. This will both help with deeper customizations, but also allow for partial and gradual migration of apps. For example, one might keep the entity pages declared in code, as well as the sidebar, but migrate all the root level routes to the new system. Similarly, it should also be possible to use a custom root app layout, and within that render the rest of the app declaratively.

We also envision that we will be able to create adapters for our existing frontend system, similar to the legacyPlugin helper in the new Backend system. At minimum we would provide helpers that let you wrap existing plugins and extensions and use them declaratively, although it might also be that existing plugins and extensions are simply compatible with and usable in the new system.

Runtime Overhead

There is a risk that this system adds considerable runtime overhead, both in the size of the initial page load, as well as execution time. When designing our existing system we constructed benchmarks to understand the overhead of the app element tree traversal that we do. For a very large (unrealistically huge) app it was on the order of hundreds of milliseconds, which we felt was acceptable. Given the significant simplification that this proposed system should bring we hope that performance will only improve.

That said, we do anticipate the number of extensions in an app to grow significantly from our current state. We will be benchmarking the new system to make sure it performs well with the order of thousands of extensions installed in the app. We will of course also make sure that as much as possible of the content of the app is lazy-loaded, and also evaluated lazily where possible.

No Visualization of App Structure

One of the main reasons for our existing JSX-based approach in the frontend framework was a desire for the app code to also serve as a site map. For example, if you’re on the /create page in the browser, we wanted it to be trivial to look that location up in code and identify the plugin that is installed at that path. This is something that we largely lose in this proposed system, since almost all this information is moved under the hood. One could of course repeat the route paths in the app configuration, but that seems like a shame.

To counteract this loss, our plan is to also build developer tools that help you visualize the structure of the app. It would help you find all of the installed extensions and their configuration, potentially documentation, and perhaps also look that up directly based on content in the app too.

Routing System

So far we have not explored in-depth what this proposal means for the routing system. It is arguably the most complex part of the Backstage frontend system, and still has many complexities that need to be addressed in the new system. We have found what we believe is a good strategy for managing the route declarations by using a common extension data type that lets us discover all routes in the app. However, we do not yet know exactly how this system will interact with the existing route refs.

It does look like there's potential for a huge improvement, which is to route directly to extension instances by their ID. It might be that RouteRefs stick around, but will point to extension instance IDs rather than be used as mount points. This would let us route to any extension in the app, potentially also scrolling it into view or highlighting it. It’s still a big unknown how this would interact with route parameters however, especially considering local routing and routing deep into multiple layers of plugins from the outside.

Naming: Replace "Extension" naming with "Feature"?

As is often the case, naming is one of the big challenges in creating complex software. We are open to feedback on the chosen names of the moving parts in this RFC.

One suggestion that’s been floated is to name the extensions "features" instead, technically FrontendFeature under the hood. This is intended to align naming patterns with BackendFeature from the new backend system, where the pieces that contribute functionality (plugins and modules) are commonly referred to as “features”.

A potential counter argument to that is that the frontend system functions slightly differently in how things are wired up, so the analogy may seem a bit strained at a purely technical level. A change to that name might also benefit from an accompanying fitting rename of "extension point", and will it blend well still with "extension data"?

What About Declarative Integration for Backend Plugins?

When we first announced the work towards Declarative Integration, as well as in discussions in the Adoption SIG and Declarative Integration Working Group, we have always said that the scope is for it to be available for both frontend and backend plugins. This is still the case. The reason that this RFC only focuses on the frontend system is because it is by far the most complex challenge and the one that needs the most work. We want to focus the attention of this RFC on the frontend framework and the new extension architecture.

This RFC is in many ways the frontend equivalent to the Backend System Evolution RFC, and we have already done the majority of the work that takes the backend system to the point of declarative integration. It is however still important that we finish that work, and that declarative integration works well and in a coherent way across both the frontend and backend systems. To what extent we can have a common configuration across the two systems remains to be seen, but at this point it is not the priority. We will not ship declarative integration for the frontend system in a stable state until we have made similar changes to enable declarative integration in the backend system.

[JordiPolo]

I think another risk is that this looks complicated, or maybe does not look so much so right now but as corner cases and more flexibility are demanded, it may become very complicated. Which will lead to bugs and to a lot of man-hours of effort. I think those man-hours could alternatively be used in other areas.

I think the risk is the lost opportunity by an unknown amount of man-hours thrown into a completely new system that may require of a lot of work to get it bug-free and continuously maintained.

[Rugvip]

@JordiPolo Yep it's certainly complicated, especially considering migration paths from the existing system. Something I do like with the design we're proposing here is that it is actually fairly similar to what we have right now, especially the way extensions use children in the existing system to share component data is very similar to extension data shared through inputs in this proposed system. Something that we forgot to mention in the RFC is the idea that extension data would be a lower level building block that you generally don't interact with as a plugin builder. It's only as you start building plugins and core features with some level of composability that you need to care about extension data.

More towards your point though: why do this now? Overall this design direction is something that has been in our sights for a long time, but as you say it's complicated and a costly effort. So far we've been picking off lower hanging fruit, and higher priority things like evolving the backend system. What has changed now is that we're running out of simpler fixes and are hitting the point where it is worth investing the time into this work.

What we are also taking into account is how difficult it is to contribute to different areas of the project. For engineering efforts in the core maintainers group we feel we should be focusing on the areas with high impact where it is also difficult for the community to help out. That way we leave other areas of the project open for contributions (including from other teams at Spotify), and we're able to move the entire project forward faster.

We of course also believe that this evolution of the framework will bring massive benefits in the long term, for both future and existing adopters.

[Hyperkid123]

Hello. I have a few questions

Extension Inputs: An extension can optionally have inputs that can be used to consume the outputs of other extensions. Inputs can be constrained to require specific data to be present in any consumed outputs.

About can be used to consume the outputs of other extensions. Assuming an extension A has required input from extension B. What happens in extension B is not explicitly configured? (extension B is a hard dependency of extension A). Do we expect extension B to be instantiated with some default configuration? Or is this not allowed?

I think that extension might be also interested in something like global context. Things like using user object data or other say core API of the instance. Is this something that will be provided globally to all extensions or would we have to specify it as an input?

Runtime Overhead

IMO this is probably the biggest risk. I would be very interested in any future benchmarks. The performance hit will grow with the granularity of the configuration. Any declarative rendering system (in React) means that it essentially builds another VirtualDOM layer on top of the existing one.

My question is (and I realize it's extremely difficult to say right now) what granularity of the configuration do you expect? Should we expect most of the extensions to be fairly large sets of components? Entire routes/views. Or do you envision absolutely everything to be defined declaratively with a minimum of JSX?

The risk grows as the number of interactive nodes grows. If the state is managed poorly it can mean re-rendering whole subtrees on any user interactions. A nice example is conditional rendering which depends on the state of the entire UI and how often this state changes. Any time such dependency changes, all conditionally rendered nodes have to be re-validated which can be very expensive.

And as @JordiPolo mentioned, as the complexity will grow these factors will be more visible. Hope I don't sound like fun police, but as I said. I think the extra runtime can cause issues and has to be designed and developed with exceptional caution.

Conditional rendering

The RFC touches on this topic slightly. I would consider this a core feature of any rendering loop. Do you have some vision of where you want to take this? Is it expected that:

• a condition will be "static" (determined at some initialization window/browser refresh)
• a condition result can be influenced by user actions in the UI
• a condition can be changed asynchronously by the backend
• we can expect only simple conditions (if variable A has value B then the outcome) or do you envision some condition changing
• condition outcome will be node is visible/hidden or can it be used to influence for example the node input/output

Routing System

How do you expect to handle "generic" extensions that can sit on multiple different and differently nested routes? If an extension is attached to a route, is it going to receive the parent route or should we expect the routing to be relative or computed at runtime?

I know this is a long one. Sorry about that 🙃. Thanks for putting the RFC together. It is always very exciting to see these kinds of systems develop.

cc @tumido

[Rugvip]

Thank you for the comments @Hyperkid123 😁

Extension Inputs: An extension can optionally have inputs that can be used to consume the outputs of other extensions. Inputs can be constrained to require specific data to be present in any consumed outputs.

About can be used to consume the outputs of other extensions. Assuming an extension A has required input from extension B. What happens in extension B is not explicitly configured? (extension B is a hard dependency of extension A). Do we expect extension B to be instantiated with some default configuration? Or is this not allowed?

Right now I believe that we won't have any strict requirements across extensions. It would be up to A to either have a fallback, or in some cases that part of the app might simply be broken. For example if you have an entity page but don't register any cards or content then you'll just be left with an empty entity page 🤷. That's possible in the system we have today too.

I think that extension might be also interested in something like global context. Things like using user object data or other say core API of the instance. Is this something that will be provided globally to all extensions or would we have to specify it as an input?

We'll still have Utility APIs for this purpose, included the config API. It remains to be seen exactly how far we go with working those into this system.

Runtime Overhead

...

Like you said it's hard to know at this point, but I would imagine that the average app has about one to two orders of magnitude more react components than extension instances. Definitely not at the level where every component is also an extension instance. The extension instances are also quite static in nature, it's a tree that is built up once when the app is initialized. All dynamic interactions happen within the extensions themselves, which is really the same kind of structure as we have in the app today. We'll be sure to make APIs available to interact with and mutate that structure at runtime, but those would be very low frequency operations.

Conditional rendering

The RFC touches on this topic slightly. I would consider this a core feature of any rendering loop. Do you have some vision of where you want to take this? Is it expected that:

• a condition will be "static" (determined at some initialization window/browser refresh)

Most likely not, since evaluation of the conditions are owned by the extension implementation, you'd typically need to render the extension first.

• a condition result can be influenced by user actions in the UI

The entire routing system can be viewed as conditional rendering, which is all implemented by React Router. The user will navigate to different parts of the app, and depending on the location different routing extensions will render different children.

• a condition can be changed asynchronously by the backend

Extension implementations can implement any arbitrary logic for how to handle and potentially render their children, so yep you could do that too.

• we can expect only simple conditions (if variable A has value B then the outcome) or do you envision some condition changing

Conditions will not be owned by the core framework itself, but implemented by extensions, so therefore there aren't really limitations on what you can do. Of course this means that the framework itself won't be aware of any of the decisions that are made, and if that's a problem it might be something we need to revisit.

• condition outcome will be node is visible/hidden or can it be used to influence for example the node input/output

If you want to influence the decisions of conditional rendering from the outside you are likely to need to re-implement or augment the parent extension implementation to support that logic.

Routing System

How do you expect to handle "generic" extensions that can sit on multiple different and differently nested routes? If an extension is attached to a route, is it going to receive the parent route or should we expect the routing to be relative or computed at runtime?

It is computed at runtime, and we will most likely keep using RouteRefs to help facilitate routing logic in plugins. Although they are likely to receive some changes to fit this system.

[jamieklassen]

A team at my company has taken a step towards structured/automated plugin installation with a kind of alternate app runtime -- a drop-in replacement for App.tsx -- which introduces an abstraction intended to capture the snippets of diff that often appear in plugin READMEs, and a "plugin wrapper" concept which contains a BackstagePlugin along with the snippets of diff and spots in the app at which to apply them to install it.

The "diff snippet" concept in our model maps loosely to the extension instance concept in this RFC, and these snippets tend to fit into two categories:

1. Layout-based: These include things like static banners to add to the app, items to add to the nav sidebar, tabs to add on the settings page, or cards to add to the overview tab on the entity page in the catalog.
2. Data-based: These include things like routes, utility APIs and themes.

Especially the layout-based extension instances are pretty easy to understand, and well-illustrated in this RFC. I was able to build a basic plugin which registered its own nav item, route, page and banner pretty easily on my branch (based off the linked experimentation branch).

I'd vote to make sure that, as part of this effort, utility APIs and themes are refactored as extensions for two (somewhat selfish) reasons:

1. I'm pretty sure it means my sister team will be able to rewrite their alternate runtime to use extensions without changing any of their plugin wrappers.
2. This highlights a nice mechanism for packaging themes and utility APIs as plugins and wiring up/installing them pretty easily -- this isn't a pattern that usually comes to mind when I think about Backstage plugins, but it strikes me as a handy one. A marketplace of Backstage themes, distributed on NPM and "declaratively installable", seems quite cool.

If this RFC is otherwise met with agreement and we can effectively parallelize work on this initiative, my team can contribute the people-hours to achieve this.

[Rugvip]

@jamieklassen yep we didn't dive too deep into what kind of things extension instances can be, tbh we haven't explored it that much ourselves yet. It will definitely not just be layout-based things though, for example we've identified Utility APIs as something we want to make part of this system too, as well as app components, i18n modules (#17436), and plugin instances. Good call on themes, we hadn't considered that yet but definitely something we'd want to include too. Overall it sounds like we're aligned though! 👍

If this RFC is otherwise met with agreement and we can effectively parallelize work on this initiative, my team can contribute the people-hours to achieve this.

Thank you! 🙏 We're still in summer mode but we'll ramp up work on this in a just few weeks. The plan is to implement an experimental MVP of the core parts of the system asap as that seems like the least parallelizable part. Once that's available we're hoping that exploring different extensions/extension instances both in the core and in plugins is something we should be able to more easily collaborate on.

[Rugvip]

Big thank you to everyone that provided feedback here and through other channels! 🙏 ❤️

We'll close this RFC now as we have begun the implementation of this new system. For now that is taking place in two new packages, @backstage/frontend-app-api and @backstage/frontend-plugin-api. These mirror the naming of the new backend system packages and are intended to replace the existing @backstage/core-app-api and @backstage/core-plugin-api packages, although that is not set in stone. For now these new packages are marked as private and will not be published until they are more functional and we are sure we will take that route. We also have app-next serving as the example frontend app.

We still love feedback on this new system, and will find opportunities to share progress as we move along. You can post feedback either here on GitHub or in the #declarative-integration channel on Discord.

[Rugvip]

The meta issue to track progress of this work can be found here: #19545