Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EmberData | 5.0 Roadmap #8086

Closed
runspired opened this issue Jul 28, 2022 · 7 comments
Closed

EmberData | 5.0 Roadmap #8086

runspired opened this issue Jul 28, 2022 · 7 comments

Comments

@runspired
Copy link
Contributor

runspired commented Jul 28, 2022

5.0 Roadmap

Target Date: Nov/Dec 2022

Note Tasks marked [-] have been moved to the 6.0 Roadmap #8460

Overview

The EmberData Team intends to release a major version of the library around/during the fall of 2022. This issue tracks the work to be done to prepare for that major. It's a non-comprehensive list and some things will likely be added to it as unknowns are discovered over time.

5.0 will ship with an ember-data optimized for Embroider out of the box. This should significantly improve build times for all apps (even traditional ember-cli builds), allow ember-data to be chunked, and bring all of the asset compilation optimizations that ember-cli users currently receive to embroider users. Today embroider users ship 2-3x more compressed ember-data code to production than users of traditional ember-cli. This effort is tracked in #8103

This major version will focus on four critical points of evolving the data story for Ember applications: many of these will be items that will need to be individually RFC'd.

  1. removal of all support for Ember classic
  2. removal of all promise proxies
  3. introduction of a new story for managing network requests
  4. introduction of a model-optional default story for presenting data in applications

There's a lot to unpack in those four work items. Let’s dig in:

1. removal of all support for Ember classic

Some features can only work in classic mode if we build them using classic APIs. This includes features like decorators which must be built over computed, and base classes (such as Model) which must be built upon EmberObject. Building these APIs in this way introduces unintentional complications that limit what EmberData can do and prevent us from offering better solutions. For instance, the nature of reopen and reopenClass and Mixins are such that even if we were to deprecate classic class syntax, any custom decorators we write for schema information may fall victim to what these methods can do to a Model.

Historically, significant portions of internal code, typically the densest and most complex portions, were developed to defend against and manage the consequences of observers, computed chains and "mandatory setter" wreaking havoc by accessing store state before it was finalized. This did not become simpler with Octane, but harder, because EmberData now has to juggle two fundamentally opposed forms of change notification. Bug reports around change notification are the most common reports we receive. Removing support for observers and chains will significantly simplify some portions of the codebase allowing us to offer more advanced feature sets at lower performance costs and with more reliable change notification semantics. It would even allow for some portions of the library to be reimagined to use tracked properties directly, something we have to this point been unable to do due to the consequences of observers. While the observer problem was made significantly better with the deprecation and removal of sync-observers in ember 4.x, there's still much more for us to achieve.

While Ember is intending to deprecate support for classic, we feel EmberData should make the first move here due to the simple fact that we cannot support classic without building with classic APIs, and were we to not remove this support in advance then the resulting Ember deprecations would be unresolvable for our users.

What does this mean in practice?

  • Users must convert Adapter/Serializer/Model usage to native class syntax
  • We will remove EmberObject (and its APIs) from the prototype chain of all publicly extendable classes
  • Adapter Errors will no longer extend Ember Error
  • attr/belongsTo/hasMany would only work for native syntax once the deprecation lifecycle is complete
  • all array-like APIs will be removed in favor of Arrays or custom iterables. Therein, calling A() on any array or iterable returned by EmberData would be a "no-op" operation.
  • where possible the detection of a computed or observer chain interacting with an EmberData provided class will print a deprecation. In 5.0 any said interaction would either fail (with a nice assert in dev if detectable) or result in unsupported and undefined behaviors.

2. removal of all promise proxies

Several RFCs have already begun this work. All promise proxies would be eliminated in favor of awaiting the async value before usage. Promise state flags would be provided via a utility such as the awesome ember-promise-helpers library by @fivetanley. We feel such a utility should be given a home core to the ecosystem in a way that all libraries could utilize it, and would likely bring it in as an official package. Promise proxies currently make it nearly impossible to use async await within the ember-data codebase when also using typescript and properly typing a method. While we could eventually decide to sometimes provide a custom async value, simply returning promises as the default for async is better for tooling, typescript, and expectations.

3. introduction of a new story for managing network requests

Adapters and Serializers got us far, but simpler approaches with greater flexibility, maintainability, and customizability have been proved out. Watch for a RequestManager RFC. 5.0 would not itself remove support for adapters and serializers, as we can easily support them as a custom plugin for the new system for some time, but it would move their usage into a "legacy" mode and remove them from the default story.

4. introduction of a model-optional default story for presenting data in applications

ember-m3, ember-data-model-fragments, ember-data-changetracker, ember-data-storefront and so many other libraries along the way have each tried to solve how to model more complex data and interactions than what @ember-data/model has provided out of the box. ember-m3 was a proving ground that we can do better by default, and it's learnings have led to internal refactoring, new public APIs, and new RFCs to allow us to present data in a whole new approach that fits a larger number of APIs more naturally, is more adept at adapting to new situations, and scales more naturally as applications and teams grow. Critically, these new APIs offer us the chance to offer GraphQL as a default option for users of EmberData. While we are not pivoting away from a strong story for JSON:API users, we want both stories to be equally strong (and even encourage mixing the two, they complement each other well).

Action Items

This is a non-comprehensive list that we expect to grow over the next several months as we determine the exact mechanics of achieving the above.

Deprecations

Refactoring

Documentation

  • document store APIs for custom model classes
  • document store APIs for RecordData
  • document store APIs for schema
  • document RecordData
  • document store APIs for Request notifications
  • write an overview for ember-data ember-data landing page: use ember-data-overview module main if present ember-learn/ember-api-docs#761
  • README for store package
  • README for request package
  • README for adapter package
  • README for serializer package
  • README for debug package
  • README for json:api cache package
  • README for graph package
  • README for model package
  • overview for store package
  • overview for request package
  • overview for adapter package
  • overview for serializer package
  • overview for debug package
  • overview for json:api cache package
  • overview for graph package
  • overview for model package

Features

Request Manager Support

  • improve enforcement of immutability in dev
  • additional test coverage for Store.request, errors thrown during request
  • additional test coverage for cache.peek / cache.put / cache.peekRequest
  • update docs for RequestManager.useCache
  • [-] Fetch Handler | handle returned non-JSON
  • Fetch Handler | handle abort
  • implement new Collection mode to properly reflect ResourceDocument
    • Document Wrapper Class
      • Collection Support
      • Pagination Support
  • ensure single-resource-document requests are notified when data changes
  • [-] determine Cache "mutability" of CollectionResourceDocument
  • [-] add tests for enhancing context.request in RequestManager before calling next

Deprecation Guides

  • DEPRECATE_RSVP_PROMISE
  • DEPRECATE_SAVE_PROMISE_ACCESS
  • DEPRECATE_SNAPSHOT_MODEL_CLASS_ACCESS
  • DEPRECATE_STORE_FIND
  • DEPRECATE_HAS_RECORD
  • DEPRECATE_STRING_ARG_SCHEMAS
  • DEPRECATE_JSON_API_FALLBACK
  • DEPRECATE_MODEL_REOPEN
  • DEPRECATE_EARLY_STATIC
  • DEPRECATE_HELPERS
  • DEPRECATE_PROMISE_MANY_ARRAY_BEHAVIORS
  • DEPRECATE_V1CACHE_STORE_APIS
  • DEPRECATE_RELATIONSHIPS_WITHOUT_TYPE
  • DEPRECATE_RELATIONSHIPS_WITHOUT_ASYNC
  • DEPRECATE_RELATIONSHIPS_WITHOUT_INVERSE
  • DEPRECATE_V1_RECORD_DATA
  • DEPRECATE_A_USAGE
  • DEPRECATE_PROMISE_PROXIES
  • DEPRECATE_ARRAY_LIKE
  • DEPRECATE_COMPUTED_CHAINS
  • DEPRECATE_NON_EXPLICIT_POLYMORPHISM
  • DEPRECATE_INSTANTIATE_RECORD_ARGS
  • DEPRECATE_CREATE_RECORD_DATA_FOR_HOOK

Other Notes:

  • clarify abort signal handling in the RFC / handler documentation
  • clarify intended Error handling mechanisms/propagation
  • update RFC for RequestManager.useCache
  • update RFC for lifetimes.isSoftExpired/isHardExpired
  • update RFC for rename data => content

Moved to 6.0

  • [-] Model-less EmberData
    • [-] Collection Primitive
    • [-] Paginated Collection
    • [-] Support Nested Relationship Keys
    • [-] ProxyModel supports nested attributes
    • [-] schema decorators and schema model for transitioning users
@runspired
Copy link
Contributor Author

A potential thing here is to deprecate unstable identifier API signatures. As we've worked through actual implementations this overhead has turned out to be largely unnecessary, at times it even introduces potential error situations (caches doing operations in an unsupported order), and the ergonomics of working with the stable identifier are far better for both caches and the store anyway: especially with typescript.

@sandstrom
Copy link
Contributor

sandstrom commented Aug 14, 2022

Great roadmap! 🏅

Removing EmberObject, observers, etc. sound like a very good idea.

Snapshots and/or forks of Ember Data models

I don't want to pile on additional things to the roadmap. Just mentioning this as more of a background feedback and general thought, rather than something that I think should necessarily be in 5.0. But having this use-case in mind early in the planning, may make it easier to add them in the future, even if it isn't in 5.0.

The ability to make snapshots (or forks) of model state (attributes + relationships) and then roll back is something that would make working with forms easier. Basically, a lot of forms have an edit/cancel button, and if you attach a model to the form, make some changes and then press cancel it's nice to have an easy way of rolling it all back.

I know there are snapshots internally, exposed to serializers. But they aren't accessible "from the outside".

There are also some prior addons:

Today we are using Ember Changeset to handle forms, and are using a bunch of workarounds to buffer changes to relationships. I've also discussed this with @snewcomer in poteto/ember-changeset#551, as something that could also potentially be handled by that addon.

Would the ideal place for this be Ember Data or Ember Changeset? I don't know. But maybe it's a primitive that should reside in Ember Data, so I'm mentioning it here.

Happy to clarify and explain more about our use-case. There is also a small code example in the aforementioned issue.

@runspired
Copy link
Contributor Author

@sandstrom internally it's actually possible at this point to rollback relationships. This capability is not exposed to users because historically rollback did not include that.

With the V2 Cache, store-forking becomes something we can (relatively easily) add. Store forking is our intended route for handling all the various encapsulation scenarios where you want to be able to group a set of changes and/or quickly discard a set of records and/or a set of changes.

@sandstrom
Copy link
Contributor

Sounds awesome @runspired!

@runspired
Copy link
Contributor Author

Added deprecate accessing record props after destroy. This behavior was already difficult to support and partially broken by changes in 4.4 and again 4.5. The singleton cache exposed this problem again in a sever way in #8122, the result was adding a secret second cache to make access during teardown "mostly safe" for one browser lifecycle tick.

unloadRecord(identifier: StableRecordIdentifier): void {
    const cached = this.#peek(identifier);
    const storeWrapper = this.#storeWrapper;
    graphFor(storeWrapper).unload(identifier);

    // effectively clearing these is ensuring that
    // we report as `isEmpty` during teardown.
    cached.localAttrs = null;
    cached.remoteAttrs = null;
    cached.inflightAttrs = null;

    let relatedIdentifiers = _allRelatedRecordDatas(storeWrapper, identifier);
    if (areAllModelsUnloaded(storeWrapper, relatedIdentifiers)) {
      for (let i = 0; i < relatedIdentifiers.length; ++i) {
        let identifier = relatedIdentifiers[i];
        storeWrapper.disconnectRecord(identifier);
      }
    }

    this.#cache.delete(identifier);
    this.#destroyedCache.set(identifier, cached);

    /*
     * The destroy cache is a hack to prevent applications
     * from blowing up during teardown. Accessing state
     * on a destroyed record is not safe, but historically
     * was possible due to a combination of teardown timing
     * and retention of a RecordData instance directly on the
     * record itself.
     *
     * Once we have deprecated accessing state on a destroyed
     * instance we may remove this.
     */
    if (this.#destroyedCache.size === 1) {
      schedule('destroy', () => {
        setTimeout(() => {
          this.#destroyedCache.clear();
        });
      });
    }
  }

@runspired
Copy link
Contributor Author

And we're done 🥳

@runspired runspired unpinned this issue Apr 7, 2023
@sandstrom
Copy link
Contributor

@runspired Awesome! 💯

Happy Easter! 🐰🥚

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants