What does an AppView implementation entail?

[bnewbold]

An informal overview of what an AppView does. Similar to #2350, which discussed PDS implementations.

This document isn't "normative": AppView implementations are not required to do all of this. It also isn't complete: sometimes AppView need to do more. What an AppView needs to do is generally pretty application/Lexicion specific. This document is just to give a rough idea of what roles they can fill and services they can provide to the network.

NOTE: this is a wiki-style post and might get updated.

Accounts and Identities

AppViews re-distribute content from repos, which means they need to keep track of account hosting status, and stop re-distributing content from accounts which are deleted, deactivated, taken down, etc.

They usually provide account-specific views, and sometimes host private metadata (such as personal preferences or configuration), which means they need to be able to authenticate requests. This might mean verifying service auth for requests proxied via PDS instances. Or it might mean implementing client OAuth and some bespoke auth system for direct requests from client apps (eg, the "AppView as BFF" architecture, demonstrated in the Statusphere example app).

• resolve and validate identities (handle and DID), usually with a caching layer
• keep track of account state (active, deleted, deactivated, etc)
• consume account and identity events from the firehose, and update state as needed
• verify request auth

Record Indexing

AppViews consume content from the network and maintain indices and aggregations.

• backfill mechanism for bulk-loading existing data
• consume updates from firehose
• validate lexicon schemas (and ignore invalid records)
• track "most recent processed revision" for each repo/account with relevant records
• process record and account-level deletions
• may need to implement track and de-duplicate graph relationships (eg, duplicate "follows" or "likes")
• enforce any application-level constraints
• often includes "hydration" of aggregated metadata for specific content, given an AT URI
• might include implementing com.atproto.repo.getRecord and similar endpoints to fetch raw indexed records

Media CDN

If an application includes media files (images, audio, video, etc), the AppView is usually expected to play a role as a CDN. It might also help transform media, for example generation of thumbnails.

Processing un-trusted media files is very risky! Security exploits are common even for common file formats like PNG. It is strongly recommended to implement any media transformations in separate processes, running on separate un-trusted infrastructure, or hosted services.

• fetching, caching, serving blobs from upstream PDS instances
• possibly transforming blobs
• ability to takedown individual blobs, and stop serving blobs for entire accounts (eg, based on account state)

Personalization

Optionally, an AppView might track personal preferences for users. This is private metadata sent directly from client apps to a specific AppView instance, which has no visible impact to any other account (eg, does not include shared private data, like DMs or private accounts; or data with public side-effects like "block" relationships).

• might include "notifications" APIs, tracking "last seen" state
• might include facilitating push notifications to mobile devices
• might include configuration which impacts API views, such as server-side "mutes"
• might include client customization: the AppView would not process this metadata, just pass it back to clients to help persist and synchronize between devices. for example, labeler configuration, curation configuration

Labeling and Moderation

AppViews are usually aggregating user content from the public network, which may include unwanted, abusive, or even illegal content. AppView need mechanisms to filter or remove such content.

• support infrastructure-level takedowns of content and accounts
• often implement labeling: consume from labeling services, maintain an index of labels, and hydrate them in responses based on request headers

Search and Discovery

As part of record processing, AppViews often implement global search indices. This can include full-text search, type-ahead matching, and ranking. It can also include personalized recommendations and context.

Sometimes these roles can be abstracted out to other modular or replaceable network services (such as "feed generators").

[yamarten]

Based on those characteristics, are feed generators quite appview? Or can't we call it appview because it doesn't output most of the record information and makes bsky appview hydrate?

What if the server does not have an XRPC API? For example, frontpage.fyi and smokesignal.events. They retrieve records from firehose, index them, and provide their own views, but they are integrated with the interface and do not have XRPC.
Perhaps, except for not having their own lexicon, ClearSky and Bridgy Fed may be considered similar.

I'm not going to ask for a perfect definition of "appview" here, but I just want to hear your opinion as a reference for what I call it.

[snarfed]

I'm not @bnewbold or on the team, so take this with a grain of salt, but from my perspective, an appview is a pretty specific thing: the backend, network-wide implementation of an app's (and its lexicons) logic, behavioral semantics, and aggregations. As such, the app/lexicon owner is really the only entity qualified to design and build its reference appview. Other people can definitely run independent instances of that reference appview, and maybe build their own implementations that mimic the reference appview (discussion in #2885 (reply in thread)), but only the app/lexicon owner can define the app's semantics and behavior.

Given that, I don't see ClearSky or Bridgy Fed as anything close to appviews. They don't define or provide the app.bsky app's semantics or behavior, they just interact with it.

One interesting question here is how much of an app's semantics/behavior are defined in appview code vs in lexicons. Lexicons are great for defining functionality! But I don't know that they can fully define semantics or behavior.

Anyway, just one non-offiical perspective!

[yamarten]

@snarfed Thank you for providing your perspective! It's very interesting.

It seems to be out of the scope of this discussion, but the relationship between appview and lexicon and semantics is likely to be an important independent issue.
In my opinion, lexicon is only a schema, and the behavior depends on the implementation. description indicates the expected behavior informally, but cannot be forced.

[tom-sherman]

To continue the tangent on lexicons a little longer...

I can definitely see this being enforced over time. Once lexicon discovery is added I'd expect higher level relays like Jetstream to discard records that don't match the schema.