@joto joto released this Jul 23, 2018 · 1 commit to master since this release

Assets 2

Fixed

  • PBF reader and writer depended on byte order of system architecture.
  • Removed an unreliable test that didn't work on some architectures.

@joto joto released this Jul 23, 2018 · 5 commits to master since this release

Assets 2

Changed

  • Libosmium now needs the newest Protozero version 1.6.3.
  • Removes dependency on the utfcpp library for conversions between Unicode code points and UTF-8. We have our own functions for this now. This also gives us more control on where errors are thrown in this code.
  • Add support for using the CRC32 implementation from the zlib library in addition to the one from Boost. It is significantly faster and means we have one less dependency, because zlib is needed anyway in almost all programs using Osmium due to its use in the PBF format. Set macro OSMIUM_TEST_CRC_USE_BOOST before compiling the tests, if you want to run the tests with the boost library code, otherwise it will use the zlib code. Note that to use this you have to change your software slightly, see the documentation of the CRC_zlib class for details.
  • Add a clear_user() function to OSMObject and Changeset which allows removing the user name of an entity without re-creating it in a new buffer.
  • In Osmium the 0 value of the Timestamp is used to denote the "invalid" Timestamp, and its output using the to_iso() function is the empty string. But this is the wrong output for OSM XML files, where a timestamp that's not set should still be output as 1970-01-01T00:00:00Z. This version introduces a new to_is_all() function which will do this and uses that function in the XML writer.
  • Use protozero::byteswap_inplace instead of htonl/ntohl. Makes the code simpler and also works on Windows.
  • Marked MultipolygonCollector class as deprecated. Use the MultipolygonManager class introduced in 2.13.0 instead.
  • Lots of code cleanups especially around asserts. Libosmium checks out clean with clang-tidy now. Some documentation updates.

Fixed

  • Fix compilation error when fileno() is a macro (as in OpenBSD 6.3).
  • Make Box output consistent with the output of a single Location and avoids problems with some locales.

@joto joto released this Mar 31, 2018 · 48 commits to master since this release

Assets 2

Added

  • Add ReaderWithProgressBar class. This wraps an osmium::io::Reader and an osmium::ProgressBar into a nice little package allowing easier use in the common case.
  • Add polygon implementation for WKT and GeoJSON geometry factories. (Thanks to Horace Williams.)
  • Various tests.

Changed

  • Add git submodule with osm-testdata repository. Before this the repository had to be installed externally. Now a submodule update can be used to get the correct version of the osm-testdata repository.
  • The XML file reader was rewritten to be more strict. Cases where it could be tricked into failing badly were removed. There are now more tests for the XML parser.
  • Replaced strftime by our own implementation. Uses a specialized implementation for our use case instead the more general strftime. Benchmarked this to be faster.
  • Changed the way IDs are parsed from strings. No asserts are used any more but checks are done and an exception is thrown when IDs are out of range. This also changes the way negative values are handled. The value -1 is now always accepted for all IDs and returned as 0. This deprecates the string_to_user_id() function, use string_to_uid() instead which returns a different type.
  • It was always a bit confusing that some of the util classes and functions are directly in the osmium namespace and some are in osmium::util. The osmium::util namespace is now declared inline. which allows all util classes and functions to be addressed directly in the osmium namespace while keeping backwards compatibility.
  • An error is now thrown when the deprecated pbf_add_metadata file format option is used. Use add_metadata instead.
  • Extended the add_metadata file format option. In addition to allowing the values true, yes, false, and no, the new values all and none are now recognized. The option can also be set to a list of attributes
    separated by the + sign. Attributes are version, timestamp, changeset, uid, and user. All output formats have been updated to only output the specified attributes. This is based on the new osmium::metadata_options class which stores information about what metadata an OSMObject has or should have. (Thanks to Michael Reichert.)
  • The < (less than) operator on OSMObjects now ignores the case when one or both of the timestamps on the objects are not set at all. This allows better handling of OSM data files with reduced metadata.
  • Allow version = -1 and changeset = -1 in PBF input. This value is sometimes used by other programs to denote "no value". Osmium uses the 0 for this.
  • The example programs using the getopt_long function have been rewritten to work without it. This makes using libosmium on Windows easier, where this function is not available.
  • Removed the embedded protozero from repository. Like other dependencies you have to install protozero first. If you check out the protozero repository in the same directory where you checked out libosmium, libosmium's CMake will find it.
  • Various code cleanups, fixing of include order, etc.
  • Remove need for winsock2 library in Windows by using code from Protozero. (Thanks alex85k.)
  • Add MSYS2 build to Appveyor and fixed some Windows compile issues. (Thanks to alex85k.)
  • Use array instead of map to store input/output format creators.
  • Update included catch.hpp to version 1.12.1.

Fixed

  • Remove check for lost ways in multipolygon assembler. This rules out too many valid multipolygons, more specifically more complex ones with touching inner rings.
  • Use different macro magic for registering index maps. This allows the maps to be used for several types at the same time.
  • Lots of code was rewritten to fix warnings reported by clang-tidy making libosmium more robust.
  • Make ADL work for begin()/end() of InputIterator<Reader>.
  • Various fixes to make the code more robust, including an undefined behaviour in the debug output format and a buffer overflow in the o5m parser.
  • Range checks in o5m parser throw exceptions now instead of triggering assertions.
  • Better checking that PBF data is in range.
  • Check read and write system calls for EINTR.
  • Use tag and type from protozero to make PBF parser more robust.
  • Test testdata-multipolygon on Windows was using the wrong executable name.

@joto joto released this Aug 25, 2017 · 333 commits to master since this release

Assets 2

Added

  • New "blackhole" file format which throws away all data written into it. Used for benchmarking.

Changed

  • When reading OPL files, CRLF file endings are now handled correctly.
  • Reduce the max number of threads allowed for the Pool to 32. This should still be plenty and might help with test failures on some architectures.

Fixed

  • Tests now run correctly independent of git core.autocrlf setting.
  • Set binary mode for all files on Windows in example code.
  • Low-level file functions now set an invalid parameter handler on Windows to properly handle errors.
  • Restore earlier behaviour allowing zero-length mmap. It is important to allow zero-length memory mapping, because it is possible that such an index is empty, for instance when one type of object is missing from an input file as in osmcode/osmium-tool#65. Drawback is that files must be opened read-write for this to work, even if we only want to read from them.
  • Use Approx() to compare floating point values in tests.
  • Fix broken Item test on 32 bit platforms.

@joto joto released this Aug 15, 2017 · 356 commits to master since this release

Assets 2

Added

  • New RelationsManager class superseeds the relations::Collector class. The new class is much more modular and easier to extend. If you are using the Collector class, you are encouraged to switch.
  • New MultipolygonManager based on the RelationsManager class superseeds the MultipolygonCollector class. The examples have been changed to use the new class and all users are encouraged to switch. There is also a MultipolygonManagerLegacy class if you still need old-style multipolygon support (see below).
  • New FlexMem index class that works with input files of any size and stores the index in memory. This should now be used as the default index for node location stores. Several example programs now use this index.
  • New CallbackBuffer class, basically a convenient wrapper around the Buffer class with an additional callback function that is called whenever the buffer is full.
  • Introduce new ItemStash class for storing OSM objects in memory.
  • New osmium::geom::overlaps() function to check if two Box objects overlap.
  • Add function IdSet::used_memory() to get estimate of memory used in the set.
  • New is_defined() and is_undefined() methods on Location class.
  • Tests for all provided example programs. (Some tests currently fail on Windows for the osmium_index_lookup program.)

Changed

  • The area Assembler now doesn't work with old-style multipolygons (those are multipolygon relations with the tags on the outer ways(s) instead of on the relation) any more. Because old-style multipolygons are now (mostly) gone from the OSM database this is usually what you want. The new AssemblerLegacy class can be used if you actually need support for old-style multipolygons, for instance if you are working with historical data. (In that case you also need to use the MultipolygonManagerLegacy class instead of the MultipolygonManager class.)
  • Changes for consistent ordering of OSM data: OSM data can come in any order, but usual OSM files are ordered by type, ID, and version. These changes extend this ordering to negative IDs which are sometimes used for objects that have not been uploaded to the OSM server yet. The negative IDs are ordered now before the positive ones, both in order of their absolute value. This is the same ordering as JOSM uses.
  • Multipolygon assembler now checks for three or more overlapping segments which are always an error and can report them.
  • Enable use of user-provided thread::Pool instances in Reader and Writer for special use cases.
  • Growing a Buffer will now work with any capacity parameter, it is always rounded up for proper alignment. Buffer constructor with three arguments will now check that commmitted is not larger than capacity.
  • Updated embedded protozero to 1.5.2.
  • Update version of Catch unit test framework to 1.9.7.
  • And, as always, lots of small code cleanups and more tests.

Fixed

  • Buffers larger than 2^32 bytes do now work.
  • Output coordinate with value of -2^31 correctly.
  • Changeset comments with more than 2^16 characters are now allowed. The new maximum size is 2^32.
  • ChangesetDiscussionBuilder::add_comment_text() could fail silently instead of throwing an exception.
  • Changeset bounding boxes are now always output to OSM files (any format) if at least one of the corners is defined. This is needed to handle broken data from the main OSM database which contains such cases. The OPL reader has also been fixed to handle this case.
  • In the example osmium_location_cache_create, the index file written is always truncated first.

@joto joto released this Aug 15, 2017 · 601 commits to master since this release

Assets 2

This is a bugfix-only release for a stable branch.

Fixed

  • Output coordinate with value of -2^31 correctly.
  • Buffers larger than 2^32 bytes do now work.

@joto joto released this May 3, 2017 · 495 commits to master since this release

Assets 2

Added

  • Add two argument (key, value) overload of TagMatcher::operator().

Changed

  • Detect, report, and remove duplicate ways in multipolygon relations.
  • Change EOF behaviour of Reader: The Reader::read() function will now always return an invalid buffer exactly once to signal EOF.
  • Update QGIS multipolygon project that is part of the test suite to show more problem types.
  • Copy multipolygon QGIS file for tests to build dir in cmake step.
  • Some code cleanups and improved debug output in multipolygon code.
  • Refactor I/O code to simplify code.
  • Disable some warnings on MSVC.
  • Various small code and build script changes.

Fixed

  • Two bugs in area assembler affecting very complex multipolygons and multipolygons with overlapping or nearly overlapping lines.
  • Invalid use of iterators leading to undefined behaviour in area assembler code.
  • Area assembler stats were not correctly counting inner rings that are areas in their own right.
  • Fix a thread problem valgrind found that might or might not be real.
  • Read OPL file correctly even if trailing newline in file is missing.
  • Include order for osmium/index/map headers and osmium/index/node_locations_map.hpp (or osmium/handler/node_locations_for_ways.hpp) doesn't matter any more.

@joto joto released this May 3, 2017 · 601 commits to master since this release

Assets 2

Fixed

  • Two bugs in area assembler affecting very complex multipolygons and multipolygons with overlapping or nearly overlapping lines.
  • Invalid use of iterators leading to undefined behaviour in area assembler code.
  • Read OPL file correctly even if trailing newline in file is missing.

@joto joto released this Apr 10, 2017 · 526 commits to master since this release

Assets 2

Added

  • New TagsFilter::set_default_result() function.

Changed

  • Use larger capacity for Buffer if necessary for alignment instead of
    throwing an exception. Minimum buffer size is now 64 bytes.
  • Check order of input data in relations collector. The relations collector
    can not deal with history data or a changes file. This was documented as a
    requirement, but often lead to problems, because this was ignored by users.
    So it now checks that the input data it gets is ordered and throws an
    exception otherwise.
  • When writing an OSM file, set generator to libosmium if not set by app.

Fixed

  • Infinite loop in Buffer::reserve_space(). (Issue #202.)
  • ObjectPointerCollection::unique() now removes elements at end.
  • Tests comparing double using == operator.
  • Build on Cygwin.

@joto joto released this Apr 10, 2017 · 601 commits to master since this release

Assets 2

Fixed

  • Use minimum size of 64 bytes for buffers. This fixes an infinite loop
    when buffer size is zero.