LUA script engine for speedprofiles #1

Closed
DennisOSRM opened this Issue Sep 27, 2011 · 26 comments

Comments

Projects
None yet
3 participants
Owner

DennisOSRM commented Sep 27, 2011

The extractor should be able to support scriptable speed profiles.

DennisOSRM closed this Sep 27, 2011

DennisOSRM reopened this Sep 27, 2011

This was referenced Nov 25, 2011

Owner

DennisOSRM commented Dec 10, 2011

Moving to 0.4

Owner

DennisOSRM commented Jun 5, 2012

A first prototype is running here that uses a speedprofile defined by a LUA script. Will be moved to a seperate dev-branch in the near future.

Contributor

emiltin commented Jun 5, 2012

cool!

Owner

DennisOSRM commented Aug 23, 2012

I was thinking today to also experiment with a JavaScript library to see what works best.

Contributor

emiltin commented Aug 23, 2012

cool. i think js would be easier for a lot of people, since it's so widespread. plus you can use coffeescript :-)

Contributor

emiltin commented Aug 23, 2012

what's the performance of running lua vs js?

Owner

DennisOSRM commented Aug 23, 2012

I haven't run any tests yet, but I suspect that both fair well. I am more thinking of usability and maintainability.

@karme Any objections to using JavaScript?

Contributor

emiltin commented Aug 23, 2012

i agree usability and maintainability are probably more important that raw speed. but perhaps js offer all three things

karme commented Aug 24, 2012

Project OSRM notifications@github.com writes:

I haven't run any tests yet, but I suspect that both fair well. I am
more thinking of usability and maintainability.

@karme Any objections to using JavaScript?

fine with me

Owner

DennisOSRM commented Aug 29, 2012

I must conclude that Googles V8 is a pain after two days of working with it.

Contributor

emiltin commented Aug 29, 2012

can you say more? hard to compile, bad performance..?

Owner

DennisOSRM commented Aug 29, 2012

It is hard to integrate into existing code that was not built with V8 in mind. Also, the amount of additional wrapping code around it is not trivial.

On the other hand, it took half an hour and 15 lines of code to get basic functionality with LUA. I will investigate further what the performance of LUA.

Contributor

emiltin commented Aug 29, 2012

thanks for explaining.

Owner

DennisOSRM commented Aug 29, 2012

I have an internal branch running that does the node parsing through LUA now. LUA works like a charm and I didn’t notice any substantial speed-down.

Owner

DennisOSRM commented Aug 29, 2012

To give some feeling how parsing of nodes works, here is the LUA code snippet:

bollards_whitelist = { [""] = true, ["cattle_grid"] = true, ["border_control"] = true, ["toll_booth"] = true, ["no"] = true}
access_whitelist = { ["yes"] = true, ["motorcar"] = true, ["permissive"] = true }
access_blacklist = { ["no"] = true, ["private"] = true, ["agricultural"] = true, ["forestery"] = true }

function node_function (node)
  local barrier = node.tags:Find("barrier")
  local access = node.tags:Find("access")
  local traffic_signal = node.tags:Find("highway")

  if traffic_signal == "traffic_signals" then
    node.traffic_light = true;
  end

  if access_blacklist[barrier] then
    node.bollard = true;
  end

  if not bollards_whitelist[barrier] then
    node.bollard = true;
  end
  return 1
end
Contributor

emiltin commented Aug 29, 2012

nice, thanks for posting!

i have to admit i personally would much prefer javascript, instead of having to learn and remember yet another (similar, yet different) script language.

i believe js could potentially attract more developers to osrm than lua, since so many people know it, and there's a lot more js code out there than lua. for rubyist, the ability to use coffeescript (which converts to js) would also be an option with js.

but in any case i'm excited to see progress on the new tag parsing system!

Owner

DennisOSRM commented Aug 30, 2012

There is some progress to report. Now also ways get parsed through the LUA engine and it looks great. There are only few edges left to tweak to get it fully working. I will push the development branch to the repository later today.

Contributor

emiltin commented Aug 30, 2012

sweet

Owner

DennisOSRM commented Aug 30, 2012

Darn! For some reason, it landed in master branch. But anyway, feel free to give it a try.

Contributor

emiltin commented Aug 30, 2012

:-)

Contributor

emiltin commented Aug 30, 2012

looking forward to trying it

Owner

DennisOSRM commented Aug 30, 2012

I will close this issue now and would like to encourage everyone to file new issue reports when bugs occur. Of course discussion can also continue here.

DennisOSRM closed this Aug 30, 2012

Contributor

emiltin commented Aug 30, 2012

do you envision a lua script for each transport mode - bike, car, foot, etc - added to the repo?

Owner

DennisOSRM commented Aug 31, 2012

For the time being seperate profiles for each mode

Owner

DennisOSRM commented Sep 13, 2012

Luabind formula has been submitted to homebrew: mxcl/homebrew#14914

Owner

DennisOSRM commented Sep 13, 2012

For OS X users: Please note that OSRM builds with clang only and that there is still one issue left.

@Zhdanovich Zhdanovich pushed a commit to Zhdanovich/Project-OSRM that referenced this issue Jul 20, 2014

DennisOSRM Implements way parsing through LUA scripting engine, implements issue #1
31deac0

@DennisOSRM DennisOSRM pushed a commit that referenced this issue Nov 11, 2014

@AlanBell AlanBell Merge pull request #1 from AlanBell/master
test case for lift gate access
5b5f871

@daniel-j-h daniel-j-h added a commit that referenced this issue Feb 14, 2017

@daniel-j-h daniel-j-h Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
dfbe7ba

@MoKob MoKob added a commit that referenced this issue Feb 14, 2017

@daniel-j-h @MoKob daniel-j-h + MoKob Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
0b5fbaa

@MoKob MoKob added a commit that referenced this issue Feb 15, 2017

@daniel-j-h @MoKob daniel-j-h + MoKob Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
b15946b

@MoKob MoKob added a commit that referenced this issue Feb 15, 2017

@daniel-j-h @MoKob daniel-j-h + MoKob Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
4e4f38e

@MoKob MoKob added a commit that referenced this issue Feb 17, 2017

@daniel-j-h @MoKob daniel-j-h + MoKob Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
629d26f

@oxidase oxidase added a commit that referenced this issue Feb 21, 2017

@daniel-j-h @oxidase daniel-j-h + oxidase Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
aef75f1

@oxidase oxidase added a commit that referenced this issue Feb 28, 2017

@daniel-j-h @oxidase daniel-j-h + oxidase Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
2a65dd9

@TheMarex TheMarex added a commit that referenced this issue Mar 1, 2017

@daniel-j-h @TheMarex daniel-j-h + TheMarex Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
e5825a5

@TheMarex TheMarex added a commit that referenced this issue Mar 1, 2017

@daniel-j-h @TheMarex daniel-j-h + TheMarex Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
299d071
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment