Merge 86203ed into 46f5f4f

kensho-technologies · Feb 21, 2019 · 6d63b3b · 6d63b3b
2 parents 46f5f4f + 86203ed
commit 6d63b3b
Show file tree

Hide file tree

Showing 3 changed files with 151 additions and 5 deletions.
diff --git a/README.md b/README.md
@@ -49,6 +49,10 @@ It's modeled after Python's `json.tool`, reading from stdin and writing to stdou
      * [\_x_count](#_x_count)
   * [The GraphQL schema](#the-graphql-schema)
   * [Execution model](#execution-model)
+  * [SQL](#sql)
+     * [Configuring SQLAlchemy](#configuring-sqlalchemy)
+     * [End-To-End SQL Example](#end-to-end-sql-example)
+     * [Configuring the SQL Database to Match the GraphQL Schema](#configuring-the-sql-database-to-match-the-graphql-schema)
   * [Miscellaneous](#miscellaneous)
      * [Expanding `@optional` vertex fields](#expanding-optional-vertex-fields)
      * [Optional `type_equivalence_hints` compilation parameter](#optional-type_equivalence_hints-parameter)
@@ -80,12 +84,16 @@ A: No -- there are many existing frameworks for running a web server. We simply
 
 **Q: What databases and query languages does the compiler support?**
 
-A: We currently support a single database, OrientDB version 2.2.28+, and two query languages
+A: We currently support a single graph database, OrientDB version 2.2.28+, and two query languages
    that OrientDB supports: the OrientDB dialect of `gremlin`, and OrientDB's own custom SQL-like
    query language that we refer to as `MATCH`, after the name of its graph traversal operator.
    With OrientDB, `MATCH` should be the preferred choice for most users, since it tends to run
    faster than `gremlin`, and has other desirable properties. See the
    [Execution model](#execution-model) section for more details.
+
+   Support for relational databases including PostgreSQL, MySQL, SQLite,
+   and Microsoft SQL Server is a work in progress. A subset of compiler features are available for
+   these databases. See the [SQL](#sql) section for more details.
 
 **Q: Do you plan to support other databases / more GraphQL features in the future?**
 
@@ -1041,7 +1049,7 @@ The compiler abides by the following principles:
 - When the database is queried with a compiled query string, its response must always be in the
   form of a list of results.
 - The precise format of each such result is defined by each compilation target separately.
-  - Both `gremlin` and `MATCH` return data in a tabular format, where each result is
+  - `gremlin`, `MATCH` and `SQL` return data in a tabular format, where each result is
     a row of the table, and fields marked for output are columns.
   - However, future compilation targets may have a different format. For example, each result
     may appear in the nested tree format used by the standard GraphQL specification.
@@ -1150,6 +1158,144 @@ the opposite order:
 }
 ```
 
+## SQL
+The following table outlines GraphQL compiler features, and their support (if any) by various 
+relational database flavors:
+
+     
+| Feature/Dialect      | Required Edges | @filter                                                                                                                         | @output                                                          | @recurse | @fold | @optional | @output_source |
+|----------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------|----------|-------|-----------|----------------|
+| PostgreSQL           | No             | Limited, [intersects](#intersects), [has_edge_degree](#has_edge_degree), and [name_or_alias](#name_or_alias) filter unsupported | Limited, [\__typename](#__typename) output metafield unsupported | No       | No    | No        | No             |
+| SQLite               | No             | Limited, [intersects](#intersects), [has_edge_degree](#has_edge_degree), and [name_or_alias](#name_or_alias) filter unsupported | Limited, [\__typename](#__typename) output metafield unsupported | No       | No    | No        | No             |
+| Microsoft SQL Server | No             | Limited, [intersects](#intersects), [has_edge_degree](#has_edge_degree), and [name_or_alias](#name_or_alias) filter unsupported | Limited, [\__typename](#__typename) output metafield unsupported | No       | No    | No        | No             |
+| MySQL                | No             | Limited, [intersects](#intersects), [has_edge_degree](#has_edge_degree), and [name_or_alias](#name_or_alias) filter unsupported | Limited, [\__typename](#__typename) output metafield unsupported | No       | No    | No        | No             |
+| MariaDB              | No             | Limited, [intersects](#intersects), [has_edge_degree](#has_edge_degree), and [name_or_alias](#name_or_alias) filter unsupported | Limited, [\__typename](#__typename) output metafield unsupported | No       | No    | No        | No             |
+
+### Configuring SQLAlchemy
+Relational databases are supported by compiling to SQLAlchemy core as an intermediate
+language, and then relying on SQLAlchemy's compilation of the dialect specific SQL string to query
+the target database.
+
+For the SQL backend, GraphQL types are assumed to have a SQL table of the same name, and with the
+same properties. For example, a schema type 
+```
+type Animal {
+    name: String
+}
+```
+is expected to correspond to a SQLAlchemy table object of the same name, case insensitive. For this
+schema type this could look like:
+
+```python
+from sqlalchemy import MetaData, Table, Column, String
+# table for GraphQL type Animal
+metadata = MetaData()
+animal_table = Table(
+    'animal', # name of table matches type name from schema
+    metadata,
+    Column('name', String(length=12)), # Animal.name GraphQL field has corresponding 'name' column
+)
+```
+
+If a table of the schema type name does not exist, an exception will be raised at compile time. See
+[Configuring the SQL Database to Match the GraphQL Schema](#configuring-the-sql-database-to-match-the-graphql-schema)
+for a possible option to resolve such naming discrepancies.
+
+
+### End-To-End SQL Example
+An end-to-end example including relevant GraphQL schema and SQLAlchemy engine preparation follows.
+
+This is intended to show the setup steps for the SQL backend of the GraphQL compiler, and
+does not represent best practices for configuring and running SQLAlchemy in a production system. 
+
+```python
+from graphql import parse
+from graphql.utils.build_ast_schema import build_ast_schema
+from sqlalchemy import MetaData, Table, Column, String, create_engine
+from graphql_compiler.compiler.ir_lowering_sql.metadata import SqlMetadata
+from graphql_compiler import graphql_to_sql
+
+# Step 1: Configure a GraphQL schema (note that this can also be done programmatically)
+schema_text = '''
+schema {
+    query: RootSchemaQuery
+}
+# IMPORTANT NOTE: all compiler directives are expected here, but not shown to keep the example brief
+
+directive @filter(op_name: String!, value: [String!]!) on FIELD | INLINE_FRAGMENT
+
+# < more directives here, see the GraphQL schema section of this README for more details. >
+
+directive @output(out_name: String!) on FIELD
+
+type Animal {
+    name: String
+}
+'''
+schema = build_ast_schema(parse(schema_text))
+
+# Step 2: For all GraphQL types, bind all corresponding SQLAlchemy Tables to a single SQLAlchemy
+# metadata instance, using the expected naming detailed above.
+# See https://docs.sqlalchemy.org/en/latest/core/metadata.html for more details on this step.
+metadata = MetaData()
+animal_table = Table(
+    'animal', # name of table matches type name from schema
+    metadata,
+    # Animal.name schema field has corresponding 'name' column in animal table
+    Column('name', String(length=12)), 
+)
+
+# Step 3: Prepare a SQLAlchemy engine to query the target relational database.
+# See https://docs.sqlalchemy.org/en/latest/core/engines.html for more detail on this step.
+engine = create_engine('<connection string>')
+
+# Step 4: Wrap the SQLAlchemy metadata and dialect as a SqlMetadata GraphQL compiler object
+sql_metadata = SqlMetadata(engine.dialect, metadata)
+
+# Step 5: Prepare and compile a GraphQL query against the schema
+graphql_query = '''
+{
+    Animal {
+        name @output(out_name: "animal_name")
+             @filter(op_name: "in_collection", value: ["$names"])
+    }
+}
+'''
+parameters = {
+    'names': ['animal name 1', 'animal name 2'],
+}
+
+compilation_result = graphql_to_sql(schema, graphql_query, parameters, sql_metadata)
+
+# Step 6: Execute compiled query against a SQLAlchemy engine/connection. 
+# See https://docs.sqlalchemy.org/en/latest/core/connections.html for more details.
+query = compilation_result.query
+query_results = [dict(result_proxy) for result_proxy in engine.execute(query)]
+```
+
+### Configuring the SQL Database to Match the GraphQL Schema
+For simplicity, the SQL backend expects an exact match between SQLAlchemy Tables and GraphQL types, 
+and between SQLAlchemy Columns and GraphQL fields. What if the table name or column name in the
+database doesn't conform to these rules? Eventually the plan is to make this aspect of the
+SQL backend more configurable. In the near-term, a possible way to address this is by using
+SQL views.
+
+For example, suppose there is a table in the database called `animal_table` and it has a column
+called `animal_name`. If the desired schema has type
+```
+type Animal {
+    name: String
+}
+```
+Then this could be exposed via a view like:
+```sql
+CREATE VIEW animal AS 
+    SELECT
+        animal_name AS name
+    FROM animal_table
+```
+At this point, the `animal` view can be used in the SQLAlchemy Table for the purposes of compiling.
+
 ## Miscellaneous
 
 ### Expanding [`@optional`](#optional) vertex fields

diff --git a/graphql_compiler/__init__.py b/graphql_compiler/__init__.py
@@ -67,7 +67,7 @@ def graphql_to_sql(schema, graphql_query, parameters, compiler_metadata,
         schema: GraphQL schema object describing the schema of the graph to be queried
         graphql_query: the GraphQL query to compile to SQL, as a string
         parameters: dict, mapping argument name to its value, for every parameter the query expects.
-        compiler_metadata: CompilerMetadata object, provides SQLAlchemy specific backend
+        compiler_metadata: SqlMetadata object, provides SQLAlchemy specific backend
                            information
         type_equivalence_hints: optional dict of GraphQL interface or type -> GraphQL union.
                                 Used as a workaround for GraphQL's lack of support for

diff --git a/graphql_compiler/compiler/emit_sql.py b/graphql_compiler/compiler/emit_sql.py
@@ -30,7 +30,7 @@
     # 'query_path_to_node': Dict[Tuple[str, ...], SqlNode], mapping from each
     # query_path to the SqlNode located at that query_path.
     'query_path_to_node',
-    # 'compiler_metadata': CompilerMetadata, SQLAlchemy metadata about Table objects, and
+    # 'compiler_metadata': SqlMetadata, SQLAlchemy metadata about Table objects, and
     # further backend specific configuration.
     'compiler_metadata',
 ))
@@ -41,7 +41,7 @@ def emit_code_from_ir(sql_query_tree, compiler_metadata):
 
     Args:
         sql_query_tree: SqlQueryTree, tree representation of the query to emit.
-        compiler_metadata: CompilerMetadata, SQLAlchemy specific metadata.
+        compiler_metadata: SqlMetadata, SQLAlchemy specific metadata.
 
     Returns:
         SQLAlchemy Query