Update Airbyte Protocol Docs (#13709)

airbyte-cdk/python/README.md

@@ -8,7 +8,7 @@ The Airbyte Python CDK is a framework for rapidly developing production-grade Ai
 
 The CDK provides an improved developer experience by providing basic implementation structure and abstracting away low-level glue boilerplate.
 
-This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-specification) before proceeding.
+This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-protocol) before proceeding.
 
 ## Getting Started
 

airbyte-cdk/python/airbyte_cdk/models/airbyte_protocol.py

@@ -27,13 +27,13 @@ class AirbyteRecordMessage(BaseModel):
     class Config:
         extra = Extra.allow
 
-    stream: str = Field(..., description="the name of this record's stream")
-    data: Dict[str, Any] = Field(..., description="the record data")
+    namespace: Optional[str] = Field(None, description="namespace the data is associated with")
+    stream: str = Field(..., description="stream the data is associated with")
+    data: Dict[str, Any] = Field(..., description="record data")
     emitted_at: int = Field(
         ...,
         description="when the data was emitted from the source. epoch in millisecond.",
     )
-    namespace: Optional[str] = Field(None, description="the namespace of this record's stream")
 
 
 class AirbyteStateType(Enum):
@@ -70,8 +70,8 @@ class AirbyteLogMessage(BaseModel):
     class Config:
         extra = Extra.allow
 
-    level: Level = Field(..., description="the type of logging")
-    message: str = Field(..., description="the log message")
+    level: Level = Field(..., description="log level")
+    message: str = Field(..., description="log message")
 
 
 class TraceType(Enum):
@@ -261,7 +261,10 @@ class Config:
         ...,
         description="ConnectorDefinition specific blob. Must be a valid JSON string.",
     )
-    supportsIncremental: Optional[bool] = Field(None, description="If the connector supports incremental mode or not.")
+    supportsIncremental: Optional[bool] = Field(
+        None,
+        description="(deprecated) If the connector supports incremental mode or not.",
+    )
     supportsNormalization: Optional[bool] = Field(False, description="If the connector supports normalization or not.")
     supportsDBT: Optional[bool] = Field(False, description="If the connector supports DBT or not.")
     supported_destination_sync_modes: Optional[List[DestinationSyncMode]] = Field(

airbyte-cdk/python/airbyte_cdk/sources/abstract_source.py

@@ -68,14 +68,14 @@ def name(self) -> str:
 
     def discover(self, logger: logging.Logger, config: Mapping[str, Any]) -> AirbyteCatalog:
         """Implements the Discover operation from the Airbyte Specification.
-        See https://docs.airbyte.io/architecture/airbyte-specification.
+        See https://docs.airbyte.io/architecture/airbyte-protocol.
         """
         streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]
         return AirbyteCatalog(streams=streams)
 
     def check(self, logger: logging.Logger, config: Mapping[str, Any]) -> AirbyteConnectionStatus:
         """Implements the Check Connection operation from the Airbyte Specification.
-        See https://docs.airbyte.io/architecture/airbyte-specification.
+        See https://docs.airbyte.io/architecture/airbyte-protocol.
         """
         try:
             check_succeeded, error = self.check_connection(logger, config)
@@ -93,7 +93,7 @@ def read(
         catalog: ConfiguredAirbyteCatalog,
         state: MutableMapping[str, Any] = None,
     ) -> Iterator[AirbyteMessage]:
-        """Implements the Read operation from the Airbyte Specification. See https://docs.airbyte.io/architecture/airbyte-specification."""
+        """Implements the Read operation from the Airbyte Specification. See https://docs.airbyte.io/architecture/airbyte-protocol."""
         connector_state = copy.deepcopy(state or {})
         logger.info(f"Starting syncing {self.name}")
         config, internal_config = split_config(config)

airbyte-cdk/python/docs/concepts/README.md

@@ -1,6 +1,6 @@
 # Connector Development Kit Concepts
 
-This concepts section serves as a general introduction to the Python CDK. Readers will certainly benefit from a deeper understanding of the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-specification) before proceeding, but we do a quick overview of it in our basic concepts guide below.
+This concepts section serves as a general introduction to the Python CDK. Readers will certainly benefit from a deeper understanding of the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-protocol) before proceeding, but we do a quick overview of it in our basic concepts guide below.
 
 ### Basic Concepts
 If you want to learn more about the classes required to implement an Airbyte Source, head to our [basic concepts doc](basic-concepts.md).

airbyte-cdk/python/docs/tutorials/cdk-tutorial-python-http/3-define-inputs.md

@@ -4,7 +4,7 @@ Each connector declares the inputs it needs to read data from the underlying dat
 
 The simplest way to implement this is by creating a `.json` file in `source_<name>/spec.json` which describes your connector's inputs according to the [ConnectorSpecification](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L211) schema. This is a good place to start when developing your source. Using JsonSchema, define what the inputs are \(e.g. username and password\). Here's [an example](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/spec.json) of what the `spec.json` looks like for the Freshdesk API source.
 
-For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.io/understanding-airbyte/airbyte-specification).
+For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.io/understanding-airbyte/airbyte-protocol).
 
 The generated code that Airbyte provides, handles implementing the `spec` method for you. It assumes that there will be a file called `spec.json` in the same directory as `source.py`. If you have declared the necessary JsonSchema in `spec.json` you should be done with this step.
 

airbyte-cdk/python/docs/tutorials/http_api_source.md

@@ -119,7 +119,7 @@ Each connector declares the inputs it needs to read data from the underlying dat
 
 The simplest way to implement this is by creating a `.json` file in `source_<name>/spec.json` which describes your connector's inputs according to the [ConnectorSpecification](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L211) schema. This is a good place to start when developing your source. Using JsonSchema, define what the inputs are \(e.g. username and password\). Here's [an example](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/spec.json) of what the `spec.json` looks like for the Freshdesk API source.
 
-For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.io/understanding-airbyte/airbyte-specification#the-airbyte-protocol).
+For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.io/understanding-airbyte/airbyte-protocol#the-airbyte-protocol).
 
 The generated code that Airbyte provides, handles implementing the `spec` method for you. It assumes that there will be a file called `spec.json` in the same directory as `source.py`. If you have declared the necessary JsonSchema in `spec.json` you should be done with this step.
 

airbyte-protocol/protocol-models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml

@@ -55,19 +55,19 @@ definitions:
       - data
       - emitted_at
     properties:
+      namespace:
+        description: "namespace the data is associated with"
+        type: string
       stream:
-        description: "the name of this record's stream"
+        description: "stream the data is associated with"
         type: string
       data:
-        description: "the record data"
+        description: "record data"
         type: object
         existingJavaType: com.fasterxml.jackson.databind.JsonNode
       emitted_at:
         description: "when the data was emitted from the source. epoch in millisecond."
         type: integer
-      namespace:
-        description: "the namespace of this record's stream"
-        type: string
   AirbyteStateMessage:
     type: object
     additionalProperties: true
@@ -82,7 +82,6 @@ definitions:
         description: "(Deprecated) the state data"
         type: object
         existingJavaType: com.fasterxml.jackson.databind.JsonNode
-
   AirbyteStateType:
     type: string
     description: >
@@ -143,7 +142,7 @@ definitions:
       - message
     properties:
       level:
-        description: "the type of logging"
+        description: "log level"
         type: string
         enum:
           - FATAL
@@ -153,7 +152,7 @@ definitions:
           - DEBUG
           - TRACE
       message:
-        description: "the log message"
+        description: "log message"
         type: string
   AirbyteTraceMessage:
     type: object
@@ -380,10 +379,10 @@ definitions:
       # Connector Type Properties (Common to all connectors from same type)
       # Source Connectors Properties
       supportsIncremental:
-        description: If the connector supports incremental mode or not.
+        description: (deprecated) If the connector supports incremental mode or not.
         type: boolean
       # Destination Connectors Properties
-      # Normalization is currently implemented using dbt so it requires `supportsDBT` to be true for this to be true.
+      # Normalization is currently implemented using dbt, so it requires `supportsDBT` to be true for this to be true.
       supportsNormalization:
         description: If the connector supports normalization or not.
         type: boolean

docs/connector-development/README.md

@@ -1,6 +1,6 @@
 # Connector Development
 
-Airbyte supports two types of connectors: Sources and Destinations. A connector takes the form of a Docker image which follows the [Airbyte specification](../understanding-airbyte/airbyte-specification.md).
+Airbyte supports two types of connectors: Sources and Destinations. A connector takes the form of a Docker image which follows the [Airbyte specification](../understanding-airbyte/airbyte-protocol.md).
 
 To build a new connector in Java or Python, we provide templates so you don't need to start everything from scratch.
 
@@ -20,7 +20,7 @@ You can build a connector in TypeScript/JavaScript with the [Faros AI CDK](https
 
 ## The Airbyte specification
 
-Before building a new connector, review [Airbyte's data protocol specification](../understanding-airbyte/airbyte-specification.md).
+Before building a new connector, review [Airbyte's data protocol specification](../understanding-airbyte/airbyte-protocol.md).
 
 ## Adding a new connector
 

docs/connector-development/cdk-python/README.md

@@ -8,7 +8,7 @@ The Airbyte Python CDK is a framework for rapidly developing production-grade Ai
 
 The CDK provides an improved developer experience by providing basic implementation structure and abstracting away low-level glue boilerplate.
 
-This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-specification) before proceeding.
+This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-protocol) before proceeding.
 
 If you have any issues with troubleshooting or want to learn more about the CDK from the Airbyte team, head to the \#connector-development channel in [our Slack](https://airbytehq.slack.com/ssb/redirect) to inquire further!
 

docs/connector-development/connector-specification-reference.md

@@ -1,6 +1,6 @@
 # Connector Specification Reference
 
-The [connector specification](../understanding-airbyte/airbyte-specification.md#spec) describes what inputs can be used to configure a connector. Like the rest of the Airbyte Protocol, it uses [JsonSchema](https://json-schema.org), but with some slight modifications.
+The [connector specification](../understanding-airbyte/airbyte-protocol.md#spec) describes what inputs can be used to configure a connector. Like the rest of the Airbyte Protocol, it uses [JsonSchema](https://json-schema.org), but with some slight modifications.
 
 ## Demoing your specification
 

docs/connector-development/testing-connectors/legacy-standard-source-tests.md

@@ -3,7 +3,7 @@
 ## Airbyte's Standard Tests (v1)
 
 This document describes the old version Standard Tests, please check the latest version [here](../../connector-development/testing-connectors/README.md)
-To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../understanding-airbyte/airbyte-specification.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
+To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../understanding-airbyte/airbyte-protocol.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
 
 ### Architecture of standard tests
 
@@ -41,7 +41,7 @@ airbyteStandardSourceTestFile {
 }
 ```
 
-These inputs are all described in the [Airbyte Specification](../../understanding-airbyte/airbyte-specification.md) and will be used as follows:
+These inputs are all described in the [Airbyte Specification](../../understanding-airbyte/airbyte-protocol.md) and will be used as follows:
 
 * **Spec file** will be compared to the spec file output by the connector when the `spec` command is called. 
 * **Config file** is expected to be a valid config file. It's expected that calling `check` with this config will succeed. 

docs/connector-development/testing-connectors/source-acceptance-tests-reference.md

@@ -1,6 +1,6 @@
 # Source Acceptance Tests Reference
 
-To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../understanding-airbyte/airbyte-specification.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
+To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../understanding-airbyte/airbyte-protocol.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
 
 _Note: If you are looking for reference documentation for the deprecated first version of test suites, see_ [_Standard Tests \(Legacy\)_](https://github.com/airbytehq/airbyte/tree/e378d40236b6a34e1c1cb481c8952735ec687d88/docs/contributing-to-airbyte/building-new-connector/legacy-standard-source-tests.md)_._