Skip to content

Commit

Permalink
Merge pull request #46 from mathuin/revert-45-no-more-downloading
Browse files Browse the repository at this point in the history 
Revert "No more downloading" -- was supposed to merge this with DEVELOP
  • Loading branch information
@mathuin
mathuin committed Jul 18, 2014
2 parents 38c3c8d + 367573e commit ea5cc2d
Show file tree
Hide file tree
Showing 3 changed files with 265 additions and 105 deletions.
65 changes: 29 additions & 36 deletions README.md
View file
Expand Up @@ -14,26 +14,45 @@ I welcome any assistance with these issues.

The TopoMC project facilitates the construction of superficially realistic Minecraft worlds leveraging USGS, specifically the NED and NLCD datasets.

## New changes
## Major changes:

* TopoMC no longer downloads data from the USGS due to changes in their web services. The current workaround is somewhat clumsy and awkward, but it does work.
* TopoMC downloads tiled data from the USGS. These files can be quite large so they are now cached for reuse.

* TopoMC generates Anvil worlds with full 256-block heights thanks to @codewarrior0 and his changes to mcedit/pymclevel! Thank you!

* TopoMC also runs on Windows! Kinda. I think. It works for me, anyway! See [this](https://github.com/mathuin/TopoMC/wiki/RunningOnWindows) for more details!

* The array and world code has been replaced with region-based code which improve accuracy while saving CPU and memory. The new commands to download, prepare, and build regions are documented below.

* GPGPU support has been added. Yes, your video card can help you build Minecraft worlds! This latest feature relies on [PyOpenCL](http://mathema.tician.de/software/pyopencl) and its associated dependencies, and is not supported on all systems.

* The landcover code has been rewritten to support the usage of MCEdit schematics as templates for certain areas, specifically croplands (farms) and developed areas. More information about this feature can be found [here](https://github.com/mathuin/TopoMC/wiki/UsingSchematics).

* pymclevel has been included as a submodule. This release of pymclevel includes an accelerated NBT module which must be compiled before use, as seen below. This module can have a significant impact on performance.

* The test dataset has been removed.

* The safehouse has been removed, but the default spawn point is still at the highest point in the dataset.

## Before running TopoMC

You will need some additional software installed on your system before TopoMC can run. On Ubuntu (precise pangolin), the following packages need to be installed:

`git gdal-bin python-scipy python-gdal python-yaml`
`git gdal-bin python-scipy python-gdal python-suds python-yaml python-progressbar`

Other operating systems use other packaging systems so you're on your own -- the error messages will tell you what's missing, but it's up to you to find it and install it!

## Installation
## How to use TopoMC

The best way to get latitude and longitude is through Google Maps. Choose your chunk of the planet (still limited to the United States and its possessions, alas) and right-click the upper left corner of the region you wish to model. Select 'What's here', and the tooltip should provide the latitude and longitude in decimal degrees. Do the same for the lower right corner of the region.

Next, here's what to do!

1. Import the pymclevel submodule. Must be done once before anything else!
1. Import the pymclevel submodule. Must be done once before anything else! There are two commands here, init and update:

```
jmt@belle:~/git/TopoMC$ git submodule update --init
jmt@belle:~/git/TopoMC$ git submodule init
jmt@belle:~/git/TopoMC$ git submodule update
```

2. (Optional) Compile the accelerated NBT module in pymclevel.
Expand All @@ -42,45 +61,19 @@ Next, here's what to do!
jmt@belle:~/git/TopoMC$ (cd pymclevel && python setup_nbt.py build)
```

## Retrieving the map data

Due to changes in the USGS web services, TopoMC no longer downloads files directly from the USGS. Instead, the files must be downloaded using the [National Map viewer](http://viewer.nationalmap.gov/viewer/).

1. Open the viewer, and find your region of interest. Then click the "Download Data" button on the top of the browser frame to the right.

2. Select the region with your mouse, and in the next window, select "Land Cover" and "Elevation", then click "Next".

3. For "Land Cover", select any files marked "National Land Cover Database 2011 - Land Cover - 3x3 Degree Extents".
* As the page says, clicking on the products will reveal their footprints on the map. More than one product may be required to cover your entire region of interest. Be sure to select all relevant products required to cover your region. For the majority of cases, only one file will be required for land cover.

4. Click the "Elevation" bar to bring up the elevation files. These choices are more complex. Tips to keep in mind:
* More regions will require multiple elevation files than multiple land cover files, so it is even more important to select all relevant products for elevation.
* The highest resolution will give the best results. 1/9 arc-second is better than 1/3 arc-second which is better than 1 arc second. Not all regions have 1/9 arc-second coverage -- in those cases, select 1/3 arc-second data.
* The only format currently supported for elevation data is IMG.

5. After you click the "Next" button again, you will have one more opportunity to review your order before checkout. When you have reviewed your order and confirmed that you indeed have all the files necessary to cover your region of interest, check out and give your email address to the USGS so they can email you when your files are ready. SAVE THAT EMAIL! Not only will it contain links for downloading the data you requested, it will also contain the coordinates of your selection. These will be needed for building the Minecraft world. The coordinates will look like this:
3. Retrieve the region from the USGS. The upper left latitude is the "ymax" value as seen below, and the lower right latitude is the "ymin". For longitude, the upper left is the "xmin" while the lower right is the "xmax".

```
(-70.261, 42.009), (-70.11, 42.09)
jmt@belle:~/git/TopoMC$ ./getregion.py --name Provincetown --ymax 42.0901 --xmin -70.2611 --ymin 42.0091 --xmax -70.1100
```

## Building the Minecraft world

1. Build the region based on the datasets retrieved from the USGS. The values above directly translate into the command line coordinates, and don't forget the files!

```
jmt@belle:~/git/TopoMC$ ./getregion.py --name Provincetown --xmin -70.261 --ymin 42.009 --xmax -70.11 --ymax 42.09 --lcfile ~/Downloads/NLCD2011_LC_N42W069.zip,~/Downloads/NLCD2011_LC_N39W069.zip --elfile ~/Downloads/n43w071.zip
```

Note that multiple files are separated by commas.

2. Prepare the region for processing.
4. Prepare the region for processing.

```
jmt@belle:~/git/TopoMC$ ./prepregion.py --name Provincetown
```

3. Construct the Minecraft world based on the region.
5. Construct the Minecraft world based on the region.

```
jmt@belle:~/git/TopoMC$ ./buildregion.py --name Provincetown
Expand Down
35 changes: 29 additions & 6 deletions getregion.py
View file
Expand Up @@ -7,11 +7,35 @@
import argparse


def checkElevationIDs(string):
"""Checks to see if the given product IDs are valid."""
givenIDs = string.split(',')
validIDs = [ID for ID in givenIDs if ID in Region.productIDs['elevation']]
if validIDs == givenIDs:
return givenIDs
else:
raise argparse.ArgumentTypeError, 'elevation IDs invalid: %s' % string


def checkLandcoverIDs(string):
"""Checks to see if the given product IDs are valid."""
givenIDs = string.split(',')
validIDs = [ID for ID in givenIDs if ID in Region.productIDs['landcover']]
if validIDs == givenIDs:
return givenIDs
else:
raise argparse.ArgumentTypeError, 'landcover IDs invalid: %s' % string


def main():
"""Creates a specified region and downloads files from USGS."""
# example:
# ./GetRegion.py --name BlockIsland --ymax 41.2378 --ymin 41.1415 --xmin -71.6202 --xmax -71.5332

# defaults
default_elevationIDs = ','.join(Region.productIDs['elevation'])
default_landcoverIDs = ','.join(Region.productIDs['landcover'])

# parse options and get results
parser = argparse.ArgumentParser(description='Create regions and download files from USGS.')
parser.add_argument('--name', required=True, type=str, help='name of the region to be generated')
Expand All @@ -25,24 +49,23 @@ def main():
parser.add_argument('--trim', type=int, help='trim value (default %d)' % Region.trim)
parser.add_argument('--sealevel', type=int, help='sealevel value (default %d)' % Region.sealevel)
parser.add_argument('--maxdepth', type=int, help='maxdepth value (default %d)' % Region.maxdepth)
parser.add_argument('--elfiles', type=str, help='ZIP files containing elevation data retrieved from USGS')
parser.add_argument('--lcfiles', type=str, help='ZIP files containing landcover data retrieved from USGS')
parser.add_argument('--elevationIDs', default=default_elevationIDs, type=checkElevationIDs, help='ordered list of product IDs (default %s)' % default_elevationIDs)
parser.add_argument('--landcoverIDs', default=default_landcoverIDs, type=checkLandcoverIDs, help='ordered list of product IDs (default %s)' % default_landcoverIDs)
parser.add_argument('--disable-ore', action='store_false', dest='doOre', default=True, help='disable ore generation')
parser.add_argument('--enable-schematics', action='store_true', dest='doSchematics', default=False, help='enable schematic usage')
parser.add_argument('--debug', action='store_true', help='enable debug output')
args = parser.parse_args()

# enable debug
if (args.debug):
# JMT: previously debug was for suds, now we no longer download files
pass
logging.getLogger('suds.client').setLevel(logging.DEBUG)

# create the region
print "Creating new region %s..." % args.name
myRegion = Region(name=args.name, xmax=args.xmax, xmin=args.xmin, ymax=args.ymax, ymin=args.ymin, scale=args.scale, vscale=args.vscale, trim=args.trim, tilesize=args.tilesize, sealevel=args.sealevel, maxdepth=args.maxdepth, lcfiles=args.lcfiles, elfiles=args.elfiles, doOre=args.doOre, doSchematics=args.doSchematics)
myRegion = Region(name=args.name, xmax=args.xmax, xmin=args.xmin, ymax=args.ymax, ymin=args.ymin, scale=args.scale, vscale=args.vscale, trim=args.trim, tilesize=args.tilesize, sealevel=args.sealevel, maxdepth=args.maxdepth, lcIDs=args.landcoverIDs, elIDs=args.elevationIDs, doOre=args.doOre, doSchematics=args.doSchematics)

print "Retrieving files..."
myRegion.maketiffs()
myRegion.getfiles()

if __name__ == '__main__':
sys.exit(main())

0 comments on commit ea5cc2d

Please sign in to comment.