Merge pull request #231 from nikomatsakis/dynamic-databases-rfc

[RFC] Dynamic databases

book/src/SUMMARY.md

@@ -8,6 +8,7 @@
   - [On-demand (Lazy) inputs](./common_patterns/on_demand_inputs.md)
 - [YouTube videos](./videos.md)
 - [Plumbing](./plumbing.md)
+  - [Diagram](./plumbing/diagram.md)
   - [Query groups](./plumbing/query_groups.md)
   - [Database](./plumbing/database.md)
 - [RFCs](./rfcs.md)
@@ -16,4 +17,5 @@
   - [RFC 0002: Intern queries](./rfcs/RFC0002-Intern-Queries.md)
   - [RFC 0003: Query dependencies](./rfcs/RFC0003-Query-Dependencies.md)
   - [RFC 0004: LRU](./rfcs/RFC0004-LRU.md)
-  - [RFC 0005: Durability](./rfcs/RFC0005-Durability.md)
+  - [RFC 0005: Durability](./rfcs/RFC0005-Durability.md)
+  - [RFC 0006: Dynamic database](./rfcs/RFC0006-Dynamic-Databases.md)

book/src/common_patterns/on_demand_inputs.md

@@ -25,7 +25,7 @@ trait FileWatcher {
     fn did_change_file(&mut self, path: &Path);
 }
 
-fn read(db: &impl salsa::Database, path: PathBuf) -> String {
+fn read(db: &dyn salsa::Database, path: PathBuf) -> String {
     db.salsa_runtime()
         .report_synthetic_read(salsa::Durability::LOW);
     db.watch(&path);

book/src/how_salsa_works.md

@@ -42,3 +42,8 @@ example][hello_world], which has a number of comments explaining how
 things work.
 
 [hello_world]: https://github.com/salsa-rs/salsa/blob/master/examples/hello_world/main.rs
+
+## Digging into the plumbing
+
+Check out the [plumbing](plumbing.md) chapter to see a deeper explanation of the
+code that salsa generates and how it connects to the salsa library.

book/src/plumbing.md

@@ -1,17 +1,20 @@
 # Plumbing
 
-**Last updated:** 2020-06-24
-
 This chapter documents the code that salsa generates and its "inner workings".
 We refer to this as the "plumbing".
 
 This page walks through the ["Hello, World!"] example and explains the code that
 it generates. Please take it with a grain of salt: while we make an effort to
 keep this documentation up to date, this sort of thing can fall out of date
-easily.
+easily. See the page history below for major updates.
 
 ["Hello, World!"]: https://github.com/salsa-rs/salsa/blob/master/examples/hello_world/main.rs
 
 If you'd like to see for yourself, you can set the environment variable
 `SALSA_DUMP` to 1 while the procedural macro runs, and it will dump the full
 output to stdout. I recommend piping the output through rustfmt.
+
+## History
+
+* 2020-07-05: Updated to take [RFC 6](rfcs/RFC0006-Dynamic-Databases.md) into account.
+* 2020-06-24: Initial version.

book/src/plumbing/database.md

@@ -15,13 +15,9 @@ structs (like `HelloWorldStorage`) and generates the following items:
   each query group. Note: these are the structs full of hashmaps etc that are
   generaetd by the query group procdural macro, not the `HelloWorldStorage`
   struct itself.
-* a struct `__SalsaDatabaseKey` that wraps an enum `__SalsaDatabaseKeyKind`. The
-  enum contains one variant per query group, and in each variant contains the
-  group key. This can be used to identify any query in the database.
 * an impl of `HasQueryGroup<G>` for each query group `G`
 * an impl of `salsa::plumbing::DatabaseStorageTypes` for the database struct
 * an impl of `salsa::plumbing::DatabaseOps` for the database struct
-* an impl of `salsa::plumbing::DatabaseKey<DB>` for the database struct `DB`
 
 ## Key constraint: we do not know the names of individual queries
 
@@ -38,36 +34,17 @@ a "global" context.
 The `__SalsaDatabaseStorage` struct concatenates all of the query group storage
 structs. In the hello world example, it looks something like:
 
-```rust
+```rust,ignore
 struct __SalsaDatabaseStorage {
     hello_world: <HelloWorldStorage as salsa::plumbing::QueryGroup<DatabaseStruct>>::GroupStorage
 }
 ```
 
-## The database key struct / enum and the `DatabaseKey` impl
-
-The `__SalsaDatabaseKey` and `__SalsaDatabaseKeyKind` types create a **database
-key**, which uniquely identifies any query in the database. It builds on the
-**group keys** created by the query groups, which uniquely identify a query
-within a given query group.
-
-```rust
-struct __SalsaDatabaseKey {
-    kind: __SalsaDatabaseKeyKind
-}
-
-enum __SalsaDatabaseKeyKind {
-    HelloWorld(
-        <HelloWorldStorage as salsa::plumbing::QueryGroup<DatabaseStruct>>::GroupKey
-    )
-}
-```
-
-We also generate an impl of `DatabaseKey`:
+We also generate a `Default` impl for `__SalsaDatabaseStorage`. It invokes
+a `new` method on each group storage with the unique index assigned to that group.
+This invokes the [inherent `new` method generated by the `#[salsa::query_group]` macro][new].
 
-```rust,ignore
-{{#include ../../../components/salsa-macros/src/database_storage.rs:DatabaseKey}}
-```
+[new]: query_groups.md#group-storage
 
 ## The `HasQueryGroup` impl
 
@@ -78,30 +55,23 @@ within the greater database. The impl is generated here:
 {{#include ../../../components/salsa-macros/src/database_storage.rs:HasQueryGroup}}
 ```
 
-and so for our example it would look something like
+The `HasQueryGroup` impl combines with [the blanket impl] from the
+`#[salsa::query_group]` macro so that the database can implement the query group
+trait (e.g., the `HelloWorld` trait) but without knowing all the names of the
+query methods and the like.
 
-```rust
-impl salsa::plumbing::HasQueryGroup<HelloWorld> for DatabaseStruct {
-    fn group_storage(&self) -> &HelloWorldStorage::GroupStorage {
-        &self.hello_world
-    }
+[the blanket impl]: query_groups.md#impl-of-the-hello-world-trait
 
-    fn database_key(group_key: HelloWorldStorage::GroupKey) -> __SalsaDatabaseKey {
-        __SalsaDatabaseKey {
-            kind: __SalsaDatabaseKeyKind::HelloWorld(group_key)
-        }
-    }
-}
-```
-
-## Other impls
+## The `DatabaseStorageTypes` impl
 
 Then there are a variety of other impls, like this one for `DatabaseStorageTypes`:
 
 ```rust,ignore
 {{#include ../../../components/salsa-macros/src/database_storage.rs:DatabaseStorageTypes}}
 ```
 
+## The `DatabaseOps` impl
+
 Or this one for `DatabaseOps`, which defines the for-each method to
 invoke an operation on every kind of query in the database. It ultimately
 delegates to the `for_each` methods for the groups:

book/src/plumbing/diagram.md

@@ -0,0 +1,58 @@
+# Diagram
+
+Based on the hello world example:
+
+```rust,ignore
+{{#include ../../../examples/hello_world/main.rs:trait}}
+```
+
+```rust,ignore
+{{#include ../../../examples/hello_world/main.rs:database}}
+```
+
+```mermaid
+graph LR
+    classDef diagramNode text-align:left;
+    subgraph query group
+        HelloWorldTrait["trait HelloWorld: Database + HasQueryGroup(HelloWorldStroage)"]
+        HelloWorldImpl["impl(DB) HelloWorld for DB<br>where DB: HasQueryGroup(HelloWorldStorage)"]
+        click HelloWorldImpl "http:query_groups.html#impl-of-the-hello-world-trait" "more info"
+        HelloWorldStorage["struct HelloWorldStorage"]
+        click HelloWorldStorage "http:query_groups.html#the-group-struct-and-querygroup-trait" "more info"
+        QueryGroupImpl["impl QueryGroup for HelloWorldStorage<br>&nbsp;&nbsp;type DynDb = dyn HelloWorld<br>&nbsp;&nbsp;type Storage = HelloWorldGroupStorage__;"]
+        click QueryGroupImpl "http:query_groups.html#the-group-struct-and-querygroup-trait" "more info"
+        HelloWorldGroupStorage["struct HelloWorldGroupStorage__"]
+        click HelloWorldGroupStorage "http:query_groups.html#group-storage" "more info"
+        subgraph for each query...
+            LengthQuery[struct LengthQuery]
+            LengthQueryImpl["impl Query for LengthQuery<br>&nbsp;&nbsp;type Key = ()<br>&nbsp;&nbsp;type Value = usize<br>&nbsp;&nbsp;type Storage = salsa::DerivedStorage(Self)<br>&nbsp;&nbsp;type QueryGroup = HelloWorldStorage"]
+            LengthQueryFunctionImpl["impl QueryFunction for LengthQuery<br>&nbsp;&nbsp;fn execute(db: &dyn HelloWorld, key: ()) -> usize"]
+            click LengthQuery "http:query_groups.html#for-each-query-a-query-struct" "more info"
+            click LengthQueryImpl "http:query_groups.html#for-each-query-a-query-struct" "more info"
+            click LengthQueryFunctionImpl "http:query_groups.html#for-each-query-a-query-struct" "more info"
+        end
+        class HelloWorldTrait,HelloWorldImpl,HelloWorldStorage,QueryGroupImpl,HelloWorldGroupStorage diagramNode;
+        class LengthQuery,LengthQueryImpl,LengthQueryFunctionImpl diagramNode;
+    end
+    subgraph database
+        DatabaseStruct["struct Database { .. storage: Storage(Self) .. }"]
+        subgraph for each group...
+            HasQueryGroup["impl plumbing::HasQueryGroup(HelloWorldStorage) for DatabaseStruct"]
+            click HasQueryGroup "http:database.html#the-hasquerygroup-impl" "more info"
+        end
+        DatabaseStorageTypes["impl plumbing::DatabaseStorageTypes for DatabaseStruct<br>&nbsp;&nbsp;type DatabaseStorage = __SalsaDatabaseStorage"]
+        click DatabaseStorageTypes "http:database.html#the-databasestoragetypes-impl" "more info"
+        DatabaseStorage["struct __SalsaDatabaseStorage"]
+        click DatabaseStorage "http:database.html#the-database-storage-struct" "more info"
+        DatabaseOps["impl plumbing::DatabaseOps for DatabaseStruct"]
+        click DatabaseOps "http:database.html#the-databaseops-impl" "more info"
+        class DatabaseStruct,DatabaseStorage,DatabaseStorageTypes,DatabaseOps,HasQueryGroup diagramNode;
+    end
+    subgraph salsa crate
+        DerivedStorage["DerivedStorage"]
+        class DerivedStorage diagramNode;
+    end
+    LengthQueryImpl --> DerivedStorage;
+    DatabaseStruct --> HelloWorldImpl
+    HasQueryGroup --> HelloWorldImpl
+```