Skip to content

Commit

Permalink
[AIRFLOW-1319] Fix misleading SparkSubmitOperator and SparkSubmitHook…
Browse files Browse the repository at this point in the history
… docstring

- Reword docstring to reduce ambigiuity in the
usage of `--files` option.
- See JIRA issue
https://issues.apache.org/jira/browse/AIRFLOW-1319
for more info.

Closes #2377 from chrissng/airflow-1319
  • Loading branch information
chrissng authored and Fokko Driesprong committed Feb 10, 2018
1 parent 1df6c24 commit fe73f22
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
5 changes: 3 additions & 2 deletions airflow/contrib/hooks/spark_submit_hook.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ class SparkSubmitHook(BaseHook, LoggingMixin):
:param conn_id: The connection id as configured in Airflow administration. When an
invalid connection_id is supplied, it will default to yarn.
:type conn_id: str
:param files: Upload additional files to the container running the job, separated by a
comma. For example hive-site.xml.
:param files: Upload additional files to the executor running the job, separated by a
comma. Files will be placed in the working directory of each executor.
For example, serialized objects.
:type files: str
:param py_files: Additional python files used by the job, can be .zip, .egg or .py.
:type py_files: str
Expand Down
5 changes: 3 additions & 2 deletions airflow/contrib/operators/spark_submit_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ class SparkSubmitOperator(BaseOperator):
:param conn_id: The connection id as configured in Airflow administration. When an
invalid connection_id is supplied, it will default to yarn.
:type conn_id: str
:param files: Upload additional files to the container running the job, separated by a
comma. For example hive-site.xml.
:param files: Upload additional files to the executor running the job, separated by a
comma. Files will be placed in the working directory of each executor.
For example, serialized objects.
:type files: str
:param py_files: Additional python files used by the job, can be .zip, .egg or .py.
:type py_files: str
Expand Down

0 comments on commit fe73f22

Please sign in to comment.