Gosmparse works already, but the API may change (Documentation).
It has been designed with performance and maximum usage convenience in mind; on an Intel Core i7-6820HQ with NVMe flash it is able to process 67 MB/s, so a planet file can be processed in less than 10 minutes. If you find possible speed-ups or other improvements, let me know.
- fast
panic
-free- tested with different files from different sources/generators
- more than 80% test coverage and has benchmarks for all hot spots
- one dependency only: Go protobuf package
- can read from any io.Reader (e.g. for parsing during download)
go get -u github.com/missinglink/gosmparse
// Implement the gosmparser.OSMReader interface here.
// Streaming data will call those functions.
type dataHandler struct{}
func (d *dataHandler) ReadNode(n gosmparse.Node) {}
func (d *dataHandler) ReadWay(w gosmparse.Way) {}
func (d *dataHandler) ReadRelation(r gosmparse.Relation) {}
func ExampleNewDecoder() {
r, err := os.Open("filename.pbf")
if err != nil {
panic(err)
}
dec := gosmparse.NewDecoder(r)
// Parse will block until it is done or an error occurs.
err = dec.Parse(&dataHandler{})
if err != nil {
panic(err)
}
}
It is possible to parse during download, so you don't have to wait for a download to finish to be able to start the parsing/processing. You can simply use the standard Go net/http
package and pass resp.Body
to the decoder.
resp, err := http.Get("http://download.geofabrik.de/europe/germany/bremen-latest.osm.pbf")
if err != nil {
panic(err)
}
defer resp.Body.Close()
dec := gosmparse.NewDecoder(resp.Body)
err = dec.Parse(&dataHandler{})
if err != nil {
panic(err)
}
If you found a case, where gosmparse broke, please report it and provide the file that caused the failure.