The Hadoop GP Toolbox provides tools to exchange features between a Geodatabase and Hadoop and run Hadoop workflow jobs.
Python

README.md

geoprocessing-tools-for-hadoop

The Geoprocessing Tools for Hadoop provides tools to help integrate ArcGIS with Hadoop. More specifically, tools are provided that:

  • Enable the exchange of data between an ArcGIS Geodatabase and a Hadoop system, and
  • Allow ArcGIS users to run Hadoop workflow jobs.

See these tools in action as part of the samples in GIS Tools for Hadoop.

Features

  • Tools to convert between Feature Classes in a Geodatabase and JSON formatted files.
  • Tools that copy data files from ArcGIS to Hadoop, and copy files from Hadoop to ArcGIS.
  • Tools to run an Oozie workflow in Hadoop, and to check the status of a submitted workflow.

Wiki and Tutorials

  • Wiki of available geoprocessing tools.
  • Tutorials on how to run the geoprocessing tools.

Instructions

  1. Download this repository as a .zip file and unzip to a suitable location or clone the repository with a git tool.
  2. WebHDFS and Requests libraries in the tool folder are provided for convenience. If you know you will using libraries installed in /site-packages folder then remove 'webhdfs' or 'requests' folders - otherwise do not.
  3. In the ‘ArcToolbox’ pane of ArcGIS Desktop, use the ‘Add Toolbox…’ command to add the Hadoop Tools toolbox (the HadoopTools.pyt file you saved in step 1) file into ArcGIS Desktop.
  4. Use the tools individually, or use them in models and scripts, such as the examples in: GIS Tools for Hadoop.

Requirements

  • ArcGIS 10.1 or later.
  • A Hadoop system with WebHDFS support.

Dependencies

Resources

Issues

Find a bug or want to request a new feature? Please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing

Licensing

Copyright 2013-2016 Esri

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's license.txt file.