Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Algorithm for OpenStreetMap ODbL transition.
Ruby Other
Pull request Compare This branch is 13 commits behind zerebubuth:master.

Merge pull request #4 from cobra/patch-1

use find instead of ls, fixes "/bin/ls: Argument list too long"
latest commit 2d01fab35d
@gravitystorm authored
Failed to load latest commit information.
.gitignore Ignore logs directory.
Gemfile try with python abbrev code
Gemfile.lock Update Gemfile.lock
INSTALL.Linux Fixing up install instructions
README.md Quick documentation for the check_history tool.
TESTING.md Add instructions on how to run the run_bot.rb script
abbreviations.py move final check behind pop / add docu
abbreviations.rb use old hash instead creating an new one
actions.rb Initial commit.
additional_users.xml Add more users and changesets from Poland, based on lists from balrog…
bounds.xml Add Belarus earlier in the list.
change_bot.rb Reset the diff state on delete
changeset.rb Initial commit.
changesets_blacklist.txt Add more users and changesets from Poland, based on lists from balrog…
changesets_whitelist.txt Update the whitelist from the wiki, avoiding a couple of hidden pitfa…
check_history.rb Output timing of each step when asked for help in optimizing
db.rb Add individual edit white & blacklisting.
dbimport.rb Add a tool for importing user statuses to the database
diff.rb Made way-diff operations detect moves so that they can be OT'd and re…
edits_blacklist.txt Allow the bot to read the edits lists, and put placeholder text files…
edits_whitelist.txt Add the edits whitelist to git.
example.auth.yaml Add a script to populate the tracker database with an ordered list of…
extract_loader.rb sequence_ids start at different values for way_nodes and relation_mem…
geom.rb Made way-diff operations detect moves so that they can be OT'd and re…
get_auth.rb Fix up get_auth so that it doesn't clobber the whole file.
osm.rb
osm_parse.rb Do not use floating points for node coordinates to prevent errors
osm_print.rb Use XML Nodes to write out user-modifiable strings.
pg_db.rb Deal with old relations in the db having a non-zero-based sequence_id
run_bot.rb Add pid to log file name
run_candidates.rb Remove stray debugging code.
run_mega_relation.rb Add a simple script to redact the mega-relation of doom.
run_regions.rb Refactor region detection to avoid duplicate regions in first loop.
split_logs.sh use find instead of ls, fixes "/bin/ls: Argument list too long"
tags.rb Do not redact type tags on relations
test.rb Re-added test_diff to list of all tests. Commented out debug statemen…
test_abbrev.rb Fix bad merge
test_auto.rb Remove created_by tags in tests
test_auto_fail.rb Fixed relation 19000 test, missing versions
test_diff.rb Merge branch 'master' of git://github.com/zerebubuth/openstreetmap-li…
test_exceptions.rb A test for the individual edit white/black lists.
test_geom.rb Merge branch 'master' of git://github.com/zerebubuth/openstreetmap-li…
test_needs_clarity.rb Do not redact type tags on relations
test_node.rb Amend the results of the fp_bug2 test, to expect no actions.
test_odbl_tag.rb Add a more complecated test case for odbl=clean wierdness and fix
test_references.rb Add failing test for deleting nested relations in the correct order.
test_relation.rb Add test case from relation 166487
test_tags.rb Add my Supercalifragilisticexpialidocious Stret trivial change example
test_tags_lowlevel.rb
test_util.rb Make individual test files runnable
test_way.rb Add more test cases for ways
user.rb Initial commit.
users_whitelist.txt
util.rb Swapped out home-grown LCS implementation for one from Rosetta Code, …

README.md

Introduction

This is example (but working) code for the algorithm for the transition from CC-BY-SA to ODbL data. At this stage all the methods are mocked out, but future development will add the ability to run this against an apidb format database, or possibly a live API.

Requirements

To run this, you'll need Ruby (probably >=1.9.3) and some gems, which you can install with bundle install. Then you'll be able to run:

ruby test.rb

which will run the full range of unit tests. The test files can also be individually run to concentrate on some aspects of the suite.

Tools

A simple command-line tool, check_history.rb, is provided to query an API and return information about the actions that the bot would execute, if it were running for real. If you find any results from this which are not as expected, then they would make good unit tests. Run ruby check_history.rb --help for more information on running the tool, and on the available options.

Test-Driven Development

This code is intended to be read as a test-driven development. It's very hard to read most code when it implements a complex algorithm, especially when it is expected to be read by anyone not fluent in the language of choice (i.e: Ruby). In order to improve the understandability of the code, this project is intended to be test-driven, with well-commented tests to define the functionality. Hopefully these tests are quite easy to read, without being a ruby expert.

Tests can be found in the various files prefixed 'test_'. For example tests_node.rb contains a set of tests to run just involving nodes, and this is a good place to start. You'll find tests which describe nodes being created, moved, and having tags changed by various users (license change agreers and disagreers in various combinations). A test then the gives the expected resulting actions which a bot should be deciding upon, to put the node in a clean state, and to redact versions from the editing history.

Actions

The main algorithm in the code, in change_bot.rb takes the history of an element and turns this into a set of "actions", where each action is one of:

  1. Edit[new object]. This will be turned into an edit which is pushed to the API. Note that the version number should be equal to the object that this will be applied on top of. Also, the changeset ID should be -1.
  2. Delete[class, id]. This will be turned into a delete request and pushed to the API. Note that this and Edit are pretty much mutually exclusive.
  3. Redact[class, id, version, visibility]. This will be a call to the special API call which hides a version in the history, and means that it won't be distributed any more.

The redaction visibility has values :hidden and :visible and these may have different effects eventually. The intended meanings are:

  • Hidden: This version does not contribute to the final version of the object and must be completely hidden.
  • Visible: This version contains information that cannot be distributed, but may also contain information which contributes to the final version. In future implementations of the API, some of this information (authorship, metadata, etc...) may be made visible.
Something went wrong with that request. Please try again.