Source Github: Bugfix errors while syncing Team stream from non-organ…

…ization repo

* Github #2365 - handling Teams streams syncing

* Github Source #2365 - sync no records and show warning message if Collaborators Stream sync fails.

* Github Source #2365 - reformat

* Update docs/integrations/sources/github.md

* Update docs/integrations/sources/github.md

* Github Source #2365 - change fork to airbyte

* Github Source #2365 - Bump version

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

airbyte-config/init/src/main/resources/config/STANDARD_SOURCE_DEFINITION/ef69ef6e-aa7f-4af1-a01d-ef775033524e.json

@@ -2,6 +2,6 @@
   "sourceDefinitionId": "ef69ef6e-aa7f-4af1-a01d-ef775033524e",
   "name": "GitHub",
   "dockerRepository": "airbyte/source-github-singer",
-  "dockerImageTag": "0.2.0",
+  "dockerImageTag": "0.2.1",
   "documentationUrl": "https://hub.docker.com/r/airbyte/source-github-singer"
 }

airbyte-config/init/src/main/resources/seed/source_definitions.yaml

@@ -16,7 +16,7 @@
 - sourceDefinitionId: ef69ef6e-aa7f-4af1-a01d-ef775033524e
   name: GitHub
   dockerRepository: airbyte/source-github-singer
-  dockerImageTag: 0.2.0
+  dockerImageTag: 0.2.1
   documentationUrl: https://hub.docker.com/r/airbyte/source-github-singer
 - sourceDefinitionId: b5ea17b1-f170-46dc-bc31-cc744ca984c1
   name: Microsoft SQL Server (MSSQL)

airbyte-integrations/connectors/source-github-singer/Dockerfile

@@ -8,7 +8,7 @@ ENV CODE_PATH="source_github_singer"
 ENV AIRBYTE_IMPL_MODULE="source_github_singer"
 ENV AIRBYTE_IMPL_PATH="SourceGithubSinger"
 
-LABEL io.airbyte.version=0.2.0
+LABEL io.airbyte.version=0.2.1
 LABEL io.airbyte.name=airbyte/source-github-singer
 
 WORKDIR /airbyte/integration_code

airbyte-integrations/connectors/source-github-singer/setup.py

@@ -31,7 +31,7 @@
     author_email="contact@airbyte.io",
     packages=find_packages(),
     install_requires=[
-        "tap-github @ https://github.com/airbytehq/tap-github/tarball/v1.9.3-airbyte",
+        "tap-github @ https://github.com/airbytehq/tap-github/tarball/v1.9.4-airbyte",
         "requests==2.20.0",
         "airbyte-protocol",
         "base-singer",

airbyte-integrations/connectors/source-github-singer/source_github_singer/source.py

@@ -23,39 +23,14 @@
 """
 
 import json
-import sys
 from typing import Dict, List
 
 import requests
 from airbyte_protocol import AirbyteConnectionStatus, Status, SyncMode
-from base_python import AirbyteLogger
 from base_singer import SingerSource, SyncModeInfo
 
 
 class SourceGithubSinger(SingerSource):
-    @staticmethod
-    def _check_with_catalog(logger: AirbyteLogger, streams: List, config: json):
-        repositories = config["repository"].split(" ")
-        for repository in repositories:
-            org = repository.split("/")[0]
-            # requests for checking streams permissions
-            # first is for checking if user has access to Collaborators API
-            # if user is not one of the collaborators, request will return 403 error
-            # third is for checking access and permission to Teams API
-            # Teams API is only available to authenticated members of the team's organization,
-            # in another case it will return 404 error
-            # if user doesn't have permission, it will return 401 error
-            check_streams = {
-                "collaborators": f"https://api.github.com/repos/{repository}/collaborators",
-                "teams": f"https://api.github.com/orgs/{org}/teams?sort=created_at&direction=desc",
-            }
-            for stream in streams:
-                if stream in check_streams:
-                    response = requests.get(check_streams[stream], auth=(config["access_token"], ""))
-                    if response.status_code != requests.codes.ok:
-                        logger.log_by_prefix(f"{repository} {response.text}", "ERROR")
-                        sys.exit(1)
-
     def check_config(self, logger, config_path: str, config: json) -> AirbyteConnectionStatus:
         try:
             repositories = config["repository"].split(" ")
@@ -120,8 +95,4 @@ def read_cmd(self, logger, config_path, catalog_path, state_path=None) -> str:
         config_option = f"--config {config_path}"
         properties_option = f"--properties {catalog_path}"
         state_option = f"--state {state_path}" if state_path else ""
-        streams = [
-            stream["stream"] for stream in self.read_config(catalog_path).get("streams", []) if stream["schema"].get("selected", False)
-        ]
-        self._check_with_catalog(logger, streams, self.read_config(config_path))
         return f"tap-github {config_option} {properties_option} {state_option}"

docs/integrations/sources/github.md

@@ -52,7 +52,6 @@ Log into Github and then generate a [personal access token](https://github.com/s
 
 Your token should have at least the `repo` scope. Depending on which streams you want to sync, the user generating the token needs more permissions:
 
-* For syncing Collaborators, the user which generates the personal access token must be a collaborator. To become a collaborator, they must be invited by an owner. Read more about access permissions [here](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/access-permissions-on-github).
-* Syncing [Teams](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-organizations-and-teams/about-teams) is only available to authenticated members of a team's [organization](https://docs.github.com/en/free-pro-team@latest/rest/reference/orgs). [Personal user accounts](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/types-of-github-accounts) and repositories belonging to them don't have access to Teams features.
+* For syncing Collaborators, the user which generates the personal access token must be a collaborator. To become a collaborator, they must be invited by an owner. If there are no collaborators, no records will be synced. Read more about access permissions [here](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/access-permissions-on-github).
+* Syncing [Teams](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-organizations-and-teams/about-teams) is only available to authenticated members of a team's [organization](https://docs.github.com/en/free-pro-team@latest/rest/reference/orgs). [Personal user accounts](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/types-of-github-accounts) and repositories belonging to them don't have access to Teams features. In this case no records will be synced.
 * To sync the Projects stream, the repository must have the Projects feature enabled.
-