CyberGIS Scripts (cybergis-scripts)
This repository contains scripts used in the CyberGIS. The scripts contained herein are written in BASH and python. They have been developed on and for Ubuntu; however, they may work on other systems. The bin folder includes all the scripts that will be added to the path. The lib folder includes scripts that are not designed to be called directly off the command line.
See the guides in the cybergis-guides repo at https://github.com/state-hiu/cybergis-guides/, for examples on using the scripts.
The Humanitarian Information Unit has been developing a sophisticated geographic computing infrastructure referred to as the CyberGIS. The CyberGIS provides highly available, scalable, reliable, and timely geospatial services capable of supporting multiple concurrent projects. The CyberGIS relies on primarily open source projects, such as PostGIS, GeoServer, GDAL, GeoGit, OGR, and OpenLayers. The name CyberGIS is dervied from the term geospatial cyberinfrastructure.
The Rapid Opensource Geospatial User-Driven Enterprise (ROGUE) Joint Capabilities Technology Demonstration (JCTD) is a two-year research & development project developing the technology for distributed geographic data creation and synchronization in a disconnected environement. This new technology taken altogether is referred to as GeoSHAPE. See http://geoshape.org for more information. HIU is leveraging the technology developed through ROGUE to build out the CyberGIS into a robust globally distributed infrastruture.
These installation instructions are subject to change. Right now, since there are non-debian package dependencies, you can really extract the scripts to whatever directory you want. The instructions below are suggested as they mirror Linux best practices for external packages. Please be careful when installing gdal-bin and python-gdal packages as they may require different version of some packages than other programs, such as the OpenGeo Suite. It is recommended to test this and other GDAL scripts within a vagrant environment first.
As root (
sudo su -), for basic install execute the following:
apt-get update apt-get install -y curl vim git #==# cd /opt git clone https://github.com/state-hiu/cybergis-scripts.git cybergis-scripts.git cp cybergis-scripts.git/profile/cybergis-scripts.sh /etc/profile.d/
After installing, you can log out and log back in or add the scripts to the path immediately with:
If you are doing an ITTC install, you need to install python bindings. Execute the following instead:
apt-get update apt-get install -y curl vim git unzip apt-get install -y libgeos-dev libproj-dev apt-get install -y gdal-bin python-gdal python-numpy #==# cd /opt git clone https://github.com/state-hiu/cybergis-scripts.git cybergis-scripts.git cp cybergis-scripts.git/profile/cybergis-scripts.sh /etc/profile.d/
For an Amazon RDS install, execute the following:
apt-get update apt-get install -y curl vim git unzip apt-get install -y postgresql-client-common postgresql-client-9.1 #==# cd /opt git clone https://github.com/state-hiu/cybergis-scripts.git cybergis-scripts.git cp cybergis-scripts.git/profile/cybergis-scripts.sh /etc/profile.d/
For a full install, execute the following:
apt-get update apt-get install -y curl vim git unzip apt-get install -y postgresql-client-common postgresql-client-9.1 apt-get install -y libgeos-dev libproj-dev apt-get install -y gdal-bin python-gdal python-numpy #==# cd /opt git clone https://github.com/state-hiu/cybergis-scripts.git cybergis-scripts.git cp cybergis-scripts.git/profile/cybergis-scripts.sh /etc/profile.d/
Logout and Login
cybergis-script-backups.sh [repos] cybergis-script-backup.sh repos manifest output
cybergis-script-env.sh [geonode|ittc] cybergis-script-env.sh geonode [install|reset] cybergis-script-env.sh rogue [install|reset] cybergis-script-env.sh maploom install cybergis-script-env.sh geogig [install|clear] cybergis-script-env.sh ittc install
cybergis-script-ec2.sh [provision_volume|resize|swap|delete_swap] cybergis-script-ec2.sh provision_volume <device> <mount_point> cybergis-script-ec2.sh resize <dev> cybergis-script-ec2.sh swap <size> <file> cybergis-script-ec2.sh delete_swap <file>
cybergis-script-geoserver.sh [prod|dev] [tune] cybergis-script-geoserver.sh prod tune <repo> <Xmx> cybergis-script-geoserver-import-styles.py --path PATH \ --prefix PREFIX --geoserver GEOSEVER --username USERNAME \ --password PASSWORD --verbose
cybergis-script-ittc-stretch.py <input_file> <breakpoints_file> \ <output_file> <bands> --rows ROWS --threads THREADS --verbose
cybergis-script-burn-alpha.py <input_file> <alpha_file> <alpha_band_index> <output_file> <rows> <threads>
cybergis-script-hide-no-data.py <input_file> <output_file>
cybergis-script-init-rogue.sh [prod|dev] [use|rvm|gems|geonode|server|remote|remote2|aws|sns] cybergis-script-init-rogue.sh prod server [tms|geonode] <name> <url> cybergis-script-init-rogue.sh prod remote <user:password> <localRepoName> <localGeonodeURL> <remoteName> <remoteRepoName> <remoteGeoNodeURL> <remoteUser> <remotePassword> cybergis-script-init-rogue.sh prod remote2 <user:password> <repoURL> <remoteName> <remoteURL> <remoteUser> <remotePassword> cybergis-script-init-rogue.sh prod cron <direction> <user> <password> <localRepoName> <remoteName> <authorname> <authoremail> [hourly|daily|weekly|monthly] cybergis-script-init-rogue.sh prod cron2 <direction> <user> <password> <localRepoName> <remoteName> <authorname> <authoremail> <frequency>
To manually execute the post_commit_hook.py script that automates notification of GeoGit commits, you need to use GeoNode's pyton interpreter and not the system default. This script requires that the AWS tools are installed and SNS is configured. As root (
sudo su -) run:
export DJANGO_SETTINGS_MODULE=rogue_geonode.settings /var/lib/geonode/bin/python /opt/cybergis-scripts.git/lib/rogue/post_commit_hook.py "test now"
cybergis-script-postgis.sh [prod|dev] [install] cybergis-script-postgis.sh prod install [rds|local] <host> <port> <user> <password> <database> <template>
cybergis-script-pull-wfs.sh <wfs> <namespace> <featuretype> <dbname> <dbuser> <dbpass> <table>
cybergis-script-pull-arcgis.sh <service> <field> <dbname> <dbuser> <dbpass> <table>
cybergis-script-pull-shapefile.sh <url> <shapefile> <dbname> <dbuser> <dbpass> <table>
#!/bin/bash DATE=$(date "+%Y%m%d") mkdir -p backup cybergis-script-backup.sh repos /opt/cybergis-scripts.git/resources/state-hiu-repos-manifest.txt "backup/$DATE" tar -czvf "state-hiu-repos-backup-$DATE.tar.gz" "backup/$DATE"
HIU is currently not accepting pull requests for this repository.
This project constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105.
However, because the project utilizes code licensed from contributors and other third parties, it therefore is licensed under the MIT License. http://opensource.org/licenses/mit-license.php. Under that license, permission is granted free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the conditions that any appropriate copyright notices and this permission notice are included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.