example_twitter_dag.py cannot import name HiveOperator

[dennisobrien]

Dear Airflow Maintainers,

Environment

Before I tell you about my issue, let me describe my Airflow environment:

Please fill out any appropriate fields:

• Airflow version: (release number or git log --oneline | head -n 3)
  airbnb_1.7.1rc3
• Airflow components: (e.g. "webserver and scheduler with a postgres database and LocalExecutor")
  Found in example_twitter_dag.py
• Relevant airflow.cfg settings:
• Python Version: (python --version)
  Python 2.7.6
• Operating System: (Windows version or uname -a)
  Linux ip-172-31-46-134 3.13.0-74-generic Render python_callable source code in Task Details for PythonOperator #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
  Running on Amazon EC2 Ubuntu 14.04 AMI
• Python packages: (pip freeze or conda list)
  $ pip freeze
  -e git://github.com/airbnb/airflow.git@f1ff65c#egg=airflow
  alembic==0.8.6
  amqp==1.4.9
  anyjson==0.3.3
  Babel==1.3
  backports.ssl-match-hostname==3.5.0.1
  billiard==3.3.0.23
  celery==3.1.23
  certifi==2016.2.28
  cffi==1.6.0
  chartkick==0.4.2
  croniter==0.3.12
  cryptography==1.3.1
  dill==0.2.5
  docutils==0.12
  enum34==1.1.4
  Flask==0.10.1
  Flask-Admin==1.4.0
  Flask-Cache==0.13.1
  Flask-Login==0.2.11
  Flask-WTF==0.12
  flower==0.8.4
  funcsigs==0.4
  future==0.15.2
  futures==3.0.5
  gunicorn==19.3.0
  idna==2.1
  ipaddress==1.0.16
  itsdangerous==0.24
  Jinja2==2.8
  kombu==3.0.35
  lockfile==0.12.2
  Mako==1.0.4
  Markdown==2.6.6
  MarkupSafe==0.23
  numpy==1.11.0
  pandas==0.18.0
  psycopg2==2.6.1
  pyasn1==0.1.9
  pycparser==2.14
  Pygments==2.1.3
  python-daemon==2.1.1
  python-dateutil==2.5.3
  python-editor==1.0
  pytz==2016.4
  redis==2.10.5
  requests==2.9.1
  setproctitle==1.1.9
  six==1.10.0
  SQLAlchemy==1.0.12
  thrift==0.9.3
  tornado==4.2
  vertica-python==0.5.6
  Werkzeug==0.11.9
  WTForms==2.1

Description of Issue

After installing airflow from the tag 'airbnb_1.7.1rc3' I see this error on the admin/ page:

Broken DAG: [/home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py] cannot import name HiveOperator

Following the imports, I think it is due to this line in airflow/hooks/hive_hooks.py trying to import unicodecsv:
https://github.com/airbnb/airflow/blame/master/airflow/hooks/hive_hooks.py#L19

My requirements.txt does not include airflow[hive] so it did not pull in these extra dependencies.

• What did you expect to happen?
  Running the examples out of the box should not result in an error.
• What happened instead?
  An error message on /admin/.
• Screen shots of your DAG's graph and tree views, if appropriate:
• Stack trace, if appropriate:

[2016-04-27 23:24:01,410] {models.py:250} ERROR - Failed to import: /home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py
Traceback (most recent call last):
  File "/home/airflow/venv/src/airflow/airflow/models.py", line 247, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/home/airflow/venv/src/airflow/airflow/example_dags/example_twitter_dag.py", line 26, in <module>
    from airflow.operators import BashOperator, HiveOperator, PythonOperator
ImportError: cannot import name HiveOperator

Reproducing the Issue

Here is how you can reproduce this issue on your machine.

• Example code that reproduces the issue, including a minimally illustrative DAG if necessary:
• Reproduction steps:

1. Minimal airflow install from airbnb_1.7.1rc3
2. Start the webserver.
3. Notice the error message on the DAGs section.

[bolkedebruin]

Please use Apache Jira (https://issues.apache.org/jira/browse/AIRFLOW). GitHub issues are closed.

For your particular issue make sure you have installed the dependencies for Hive.