Skip to content
Distributed Scalable Geospatial Data Server
C++ Smarty Go PLpgSQL Shell HTML Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
acceptance_tests Run goimports. (#62) Jun 12, 2018
crawl added matching full path for crawler (#337) Sep 4, 2019
docker updated docker readme (#326) Aug 24, 2019
gdal-process fixed go vet (#265) May 7, 2019
grpc-server added non-blocking resp channel for worker processes (#343) Sep 11, 2019
libs/gdal/frmts/gsky_netcdf added support for CF SRS precedence (#333) Sep 2, 2019
mas added support for applying filters in mas ingestion pipeline (#338) Sep 5, 2019
metrics added support for filtering out unwanted URL parameters for metrics l… Oct 3, 2019
processor added support for raw data pass-through for fusion layers (#349) Oct 9, 2019
static fixed incorrect information of the gsky website (#259) Apr 4, 2019
templates added support for wps pixel fractions (#347) Sep 27, 2019
testsuite added tests for max width and height Sep 6, 2018
utils fixed empty namespace for fusion layers (#345) Sep 11, 2019
worker added support for wps pixel fractions (#347) Sep 27, 2019
.travis.yml fixed travis-ci build regression (#324) Aug 23, 2019
CITATION.md Update CITATION.md May 17, 2018
LICENSE-2.0.txt Import from internal repository. Mar 26, 2018
Makefile.in priotise prerequisites for make-all (#301) Aug 6, 2019
README.md GSKY netCDF driver (#294) Jul 30, 2019
config_json.md Fix typo in config_json.md (Geosaptial -> Geospatial). Aug 12, 2018
configure Use the #cgo pkg-config directive to find compiler and linker flags (#72 Jun 17, 2018
configure.ac Use the #cgo pkg-config directive to find compiler and linker flags (#72 Jun 17, 2018
dap.go added support for logging metrics (#310) Aug 15, 2019
data_unavailable.png Import from internal repository. Mar 26, 2018
ows.go added support for raw data pass-through for fusion layers (#349) Oct 9, 2019
ows_test.go Add support for Go testing (#60) Jun 12, 2018
zoom.png Import from internal repository. Mar 26, 2018

README.md

GSKY: Distributed Scalable Geospatial Data Server

What Is This?

GSKY was developed at NCI and is a scalable, distributed server which presents a new approach for geospatial data discovery and delivery using OGC standards. The most recent release is here.

License

Copyright 2016 - 2019 Australian National University

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this package except in compliance with the License. A copy of the License may be found in this source distribution in LICENSE-2.0.txt.

Contributions

Suggestions, enhancement requests, bug reports and patches to GSKY are welcome via this GitHub page. Please submit patches as a GitHub pull request. Authors retain copyright over their contributions.

Build Status

Citing GSKY in publications

When referring to GSKY in publications please use the citation in CITATION.md. A ready-to-use BibTeX entry for LaTeX users can also be found in this file.

Configuration Files

  1. config.json: Contains the list of WMS, WCS and WPS layers exposed by the server. It also contains the IP address of the index API used in the workflow. In addtion, it contains the list of worker nodes specifying the IP address and list of ports per worker. Several workers can be specified on a single machine by adding several entries using the same IP address and different ports. These services have to be locally started at the specified machines.

  2. Serveral config.json files can be organized into directories to form namespaces to group logical collection of datasets together. For example, the server serves two science projects with the following URLs:

    1) http://<server address>/ows/project1
    2) http://<server address>/ows/project2
    

    The directory structure of the config files will be as follows:

    <config root directory>
    
        project1
           config.json
    
        project2
           config.json
    

How To Compile the Source

Dependencies:

  • Go > 1.11.0
  • GDAL > 3.0.0
  • Various Go packages (handled by the build system)
$ export GOPATH=~/go
$ go get github.com/nci/gsky
$ cd $GOPATH/src/github.com/nci/gsky
$ ./configure
$ make all install

The configure script takes all of the standard GNU configure flags such as --prefix (to specify where to install GSKY).

Overview of the Servers

GSKY mainly consists of three servers working together to deliver services. The main server (ows.go) is the front-end server that takes WMS/WCS/WPS HTTP requests as inputs. The main server talks to the MAS Restful API server (mas/api/api.go) for the data files that intersect with the polygon bounding box in the WMS/WCS/WPS requests. With those data files, the main server talks to the RPC worker nodes (grpc-server/main.go) for compute and IO intensive tasks and then sends the results back to the client side.

How To Start the Servers

  • Start the MAS Restful API server: /opt/gsky/sbin/masapi -port 8888

    The -port option sets the API server listening port. The default is port 8080.

  • Start all the RPC worker nodes: /opt/gsky/sbin/gsky-rpc -p 6000

    The -p option sets the gRPC listening port. The default is port 6000.

  • Start the main server: /opt/gsky/sbin/gsky-ows -p 8080

    The -p option sets the main server listening port. The default is port 8080.

You can’t perform that action at this time.