Start an "Operator's Guide" or "Administration Guide" in GitBook docs

[ches]

Is your feature request related to a problem? Please describe.

As an operator or potential one, I don't know where to find information in one place about considerations such as:

• Component architecture (esp. operational perspective e.g. core needs a relational database which is used for registry only, serving instances need to talk to core instances, core and serving APIs are mostly stateless and horizontally scalable, etc.)
  • Feast's options for storage and compute engines—especially as support for more is added—and what Feast configuration parameters apply to each of them
• How configuration is managed in general, e.g. Spring profiles (ref Helm Chart Upgrades #458)
• How to control logging (related: Standardize logging libraries for Java components #277)
• What metrics are exposed and how
• How Feast uses Kafka topics currently
• Monitoring what Feast has last seen as the state of its managed jobs (ref Add job management APIs #302)
• Upgrading Feast
  • Considerations for re-deploying ingestion with active streaming jobs

Probably more, but that's a start.

Describe the solution you'd like

A section created that we can start from with an outline. By "section" I mean to distinguish tracks of documentation for what an end user—data engineer, data scientist, or software engineer in a client of the serving API—wants to know about using Feast's features in a deployment that's managed for them (or at least they're wearing that hat at the moment and not their DevOps one), versus operators who take care of said management.

Taking Kubernetes as a (more complex) example:

• Cluster Administration as a Concepts guide
• Administer a Cluster tree of task-oriented docs

It's doubtful we'd get get all of the above exhaustively documented in one swoop. Starting with a pebble should lower friction for contributors to extend it, I know I'll be glad to help fill things in as we prep our production-readiness.

[woop]

Is your feature request related to a problem? Please describe.

As an operator or potential one, I don't know where to find information in one place about considerations such as:

• Component architecture (esp. operational perspective e.g. core needs a relational database which is used for registry only, serving instances need to talk to core instances, core and serving APIs are mostly stateless and horizontally scalable, etc.)

...

Taking Kubernetes as a (more complex) example:

• Cluster Administration as a Concepts guide
• Administer a Cluster tree of task-oriented docs

It's doubtful we'd get get all of the above exhaustively documented in one swoop. Starting with a pebble should lower friction for contributors to extend it, I know I'll be glad to help fill things in as we prep our production-readiness.

Thanks for starting this @ches!

Projects like Feast are still about trust more than functionality. There are still a lot of areas that we can improve developer and operator confidence and this is one of them.

Internally we have documentation around running and operating Feast, but it's heavily tailored towards how we have deployed it on our infrastructure, and how our engineers should support it.

We definitely need to have the equivalent out there on Gitbook.

I will see your Kubernetes documentation (which I do think is good), and raise you the CockroachDB documentation. This is definitely also worth looking at as a standard to reach.

[woop]

what an end user—data engineer, data scientist, or software engineer in a client of the serving API—wants to know about using Feast's features in a deployment that's managed for them (or at least they're wearing that hat at the moment and not their DevOps one), versus operators who take care of said management.

Yes its very clear that these are distinct, but at the same time I do want to make sure that a data scientist can easily spin up a Feast deployment and test it locally. Often the data scientists will be the ones asking the infra team to set this up for them, so its a good route to adoption.

We currently have

• Intro/Overview Section
• Getting Started
• Reference

What about something like

• Intro/Overview Section
• Installing Feast
  • Split out the various deployment/installation methods
  • First deployment can be the quickstart, which contains a basic example to validate functionality.
• Using Feast
  • This is what the end user uses, assumes an existing deployment.
  • Basically a user manual
• Examples
  • We can move all of the examples in here, they can probably just break out to notebooks.
• Reference
  • Everything that exists already
  • + Configuration options for Helm/Docker/Docker Compose/Jars
  • + Interfaces and extension points
  • + Architecture
  • + other operating considerations as you have listed above

Or we could add an Administration section?

[woop]

I've made some changes to Gitbook. Perhaps you should have a look to see if we are on the right track @ches

[woop]

Added an Administration section to Gitbook. Feel free to add "tasks" there.

[woop]

https://docs.feast.dev/

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.