Add Google leveldb hook (#13109)

[radim94]

Add LevelDB hook and operator

closes: #13109

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (flake8, pylint and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[github-actions]

The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.

[ephraimbuddy]

Can you also add Write Batches, and option to use custom comparator. Custom comparator is particularly important considering that leveldb stores all keys in sorted order

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[RosterIn]

this should make the test green

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[RosterIn]

I don't really know what is the current problem. the error is:

reading sources... [ 65%] concepts
  Module "airflow.example_dags.example_subdag_operator" could not be loaded. Full source will not be available. "error importing 'airflow.example_dags.example_subdag_operator' (exception was: OperationalError('(sqlite3.OperationalError) no such table: slot_pool',))"
  reading sources... [ 65%] configurations-ref
  
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/sphinx/events.py", line 111, in emit
      results.append(listener.handler(self.app, *args))
    File "/usr/local/lib/python3.6/site-packages/sphinx/ext/viewcode.py", line 155, in env_purge_doc
      for modname, (code, tags, used, refname) in list(modules.items()):
  TypeError: 'bool' object is not iterable
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/sphinx/cmd/build.py", line 280, in build_main
      app.build(args.force_all, filenames)
    File "/usr/local/lib/python3.6/site-packages/sphinx/application.py", line 352, in build
      self.builder.build_update()
    File "/usr/local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 298, in build_update
      len(to_build))
    File "/usr/local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 310, in build
      updated_docnames = set(self.read())
    File "/usr/local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 417, in read
      self._read_serial(docnames)
    File "/usr/local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 436, in _read_serial
      self.events.emit('env-purge-doc', self.env, docname)
    File "/usr/local/lib/python3.6/site-packages/sphinx/events.py", line 120, in emit
      (listener.handler, name), exc, modname=modname) from exc
  sphinx.errors.ExtensionError: Handler <function env_purge_doc at 0x7fe0ce08eae8> for event 'env-purge-doc' threw an exception (exception: 'bool' object is not iterable)
  
  Extension error (sphinx.ext.viewcode):
  Handler <function env_purge_doc at 0x7fe0ce08eae8> for event 'env-purge-doc' threw an exception (exception: 'bool' object is not iterable)

doesn't really look related to the changes in this PR.

[radim94]

I see comments in slack in documentation channel and try to rebase on latest master

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[github-actions]

The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[ephraimbuddy]

Please rebase

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[radim94]

Rebased

[ephraimbuddy]

Please you have a static check error. Notice that the word comparator is not placed correctly in alphabetical order in the spelling wordlist file

[radim94]

fixed place of comparator in spelling wordlist

[boring-cyborg]

Awesome work, congrats on your first merged pull request!

[dstandish]

with this change it seems leveldb is a required system dependency for the google provider.

it doesn't seem right to require users to install leveldb on their machines just to use GCP hooks. anyone have any thoughts on this?

i noticed cus i got a build error when trying to install this provider (in virtualenv on mac) , which installed pyvel which depends on leveldb headers

[mik-laj]

@dstandish can you create a issue?