Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to set per-layer connection timeout #204

Merged
merged 1 commit into from
Jun 27, 2017

Conversation

M4rtinK
Copy link
Owner

@M4rtinK M4rtinK commented Jun 25, 2017

Add the connection_timeout key for layer in the map
config file, which can be used to set the connection
timeout for tile downloads.

This is mostly needed for offline rendering localhost
tileservers, such as OSM Scout Server, which might
in some cases take some time to render a tile but will
still eventually return it.

Like this we can set a longer connection timeout for
layers provided by localhost renderers while keeping
a short connection timeout for online layers to allow
for quick recovery from connectivity issues.

@rinigus
Copy link
Contributor

rinigus commented Jun 26, 2017

Excellent, I will test it tonight.

@rinigus
Copy link
Contributor

rinigus commented Jun 26, 2017

How do I make RPM from this? make rsync-sailfish leads to

mv: cannot stat 'modrana_source/modules/gui_modules/gui_qt5/qml': No such file or directory
mv: cannot stat 'modrana_source/qml/universal_components/silica/UC': No such file or directory

@rinigus
Copy link
Contributor

rinigus commented Jun 26, 2017

Hi,

I think I stumbled on a case showing why timeouts are bad and should be considered harmful when used with OSM Scout Server. :)

I can trigger "crazy" mode with loads of jobs submitted to the server on OPX. For that I used currently released modRana and the latest server version. In principle, it should be possible to reproduce on any phone with any timeout. Namely, when we browse a map and switch off the screen while its still has 10 jobs in a queue. The phone enters sleep mode where it wakes up once in a while (300 sec sleep are not uncommon). During short time while phone is awake between sleep cycles (insufficient to render a tile by the server), modRana figures out that a timeout has passed and submits a new job. And the pattern repeats. I have seen 67 jobs in a queue when I checked the phone a bit later. Since the device could be left for rather long time without interaction, this scenario should be possible with any CPU and any timeout.

From this example, I would suggest to skip timeouts for localhost server. We are not expected to have any network issues anyway, so let's better allow modRana/OSM Scout Server misfunction if they are used incorrectly (server closed) than to penalize with the correct usage (as in my example). For example, we could set timeout for this layer to -1 and not check for it if its negative.

@M4rtinK
Copy link
Owner Author

M4rtinK commented Jun 26, 2017

How do I make RPM from this? make rsync-sailfish leads to

mv: cannot stat 'modrana_source/modules/gui_modules/gui_qt5/qml': No such file or directory
mv: cannot stat 'modrana_source/qml/universal_components/silica/UC': No such file or directo

To make RPM targeting desktop Linux distros, like Fedora:

make archive
cp modrana-*.tar.gz ~/rpmbuild/SOURCES/
rpmbuild -ba packaging/modrana.spec

Sailfish OS specific packages are created from the same spec file in a slightly different way using RPM macros so that I don't have to maintain multiple spec files. You can see them at the top of the modrana.spec file:

%{!?with_sailfish: %define with_sailfish 0}
%{!?with_harbour: %define with_harbour 0}

The RPM macros basically enable all the mangling that differentiate Sailfish OS RPM packages from properly packaged application for say Fedora. ;-) And because of Harbour there are actually two levels of them - just with_sailfish creates packages that go to OpenRepos and if also with_harbour is enabled, a more strict mode is triggered, that creates packages with limited functionality that are Harbour compatible.

As creating the actual packages for Sailfish OS - I actually don't do that manually, but use the Sailfish OS target on the Mer OBS to do it for me. These are the two prejects I use, first for Open Repos packages, the second one for Harbour packages:

https://build.merproject.org/project/show/home:MartinK:sailfish:modrana
https://build.merproject.org/project/show/home:MartinK:sailfish:modrana-harbour

The macros that make the projects create Sailfish OS & Sailfish OS Harbour specififc packages are set in the "Project Config" tab. This is how it for example looks like for the Harbour compatible packages:

Release: <CI_CNT>.<B_CNT>

Macros:
%with_sailfish 1
%with_harbour 1
%vendor %%foo

Now thinking about it, the packages are noarch & AFAIK have been working just fine on the x86 based tablet, so the armv8el target might not really be needed. Not sure if just running rpmbuild on your distro of choice would work though - IIRC some packages are named differently on Sailfish OS so the spec variant might not be buildable outside of the SDK, even if you somehow pass the macros to rpmbuild (that should be possible - somehow :) ).

So that should be it for "Introduction to the fun and joy of cross-distribution packaging" - and don't feart to ask if there is still something unclear about how the whole house of cards works. ;-)

Add the connection_timeout key for layer in the map
config file, which can be used to set the connection
timeout for tile downloads.

This is mostly needed for offline rendering localhost
tileservers, such as OSM Scout Server, which might
in some cases take some time to render a tile but will
still eventually return it.

Like this we can set a longer connection timeout for
layers provided by localhost renderers while keeping
a short connection timeout for online layers to allow
for quick recovery from connectivity issues.
@M4rtinK M4rtinK force-pushed the master-per_layer_connection_timeout branch from f742da4 to 3fb4a4d Compare June 26, 2017 22:57
@M4rtinK
Copy link
Owner Author

M4rtinK commented Jun 26, 2017

Hi,

I think I stumbled on a case showing why timeouts are bad and should be considered harmful when used with OSM Scout Server. :)

I can trigger "crazy" mode with loads of jobs submitted to the server on OPX. For that I used currently released modRana and the latest server version. In principle, it should be possible to reproduce on any phone with any timeout. Namely, when we browse a map and switch off the screen while its still has 10 jobs in a queue. The phone enters sleep mode where it wakes up once in a while (300 sec sleep are not uncommon). During short time while phone is awake between sleep cycles (insufficient to render a tile by the server), modRana figures out that a timeout has passed and submits a new job. And the pattern repeats. I have seen 67 jobs in a queue when I checked the phone a bit later. Since the device could be left for rather long time without interaction, this scenario should be possible with any CPU and any timeout.

Interesting, pretty crazy - and I agree, pretty bad.

From this example, I would suggest to skip timeouts for localhost server. We are not expected to have any network issues anyway, so let's better allow modRana/OSM Scout Server misfunction if they are used incorrectly (server closed) than to penalize with the correct usage (as in my example). For example, we could set timeout for this layer to -1 and not check for it if its negative.

Yeah, that really does sound like the way to go - I've changed the PR so that -1 can be used in the configuration file to disable the timeout & set -1 for the OSM Scout Server provided layers. This sets the timeout parameter of the connection pool to None, which should "disable timeout" according to the docs. Hopefully it really does that vs just setting a really long timeout. :) So testing definitely welcome & especially in your IMHO not so crazy scenario.

BTW, there is actually supposed to be a mechanism on the GUI side of modRana tile handling code that should be keeping track of failed download attempts of each visible tile & stop further attempt after three failures. Looks like that's apparently not working - which is probably not so surprising given that I've rewritten that part a while ago get more optimized & flicker free map display. Another thing to check I guess. :)

@rinigus
Copy link
Contributor

rinigus commented Jun 27, 2017

I have been testing it during the day and the latest version clearly fixes the issue, thank you! From brief testing of one online service I haven't seen any problems either.

I will write the warning for other developers to avoid timeouts when using the server on mobile.

@M4rtinK
Copy link
Owner Author

M4rtinK commented Jun 27, 2017

Nice! My brief testing also indicates things work correctly, so I'll merge this now. :)

BTW, I plan to do a new modRana bugfix release in a few days so the users should be able to benefit from this fix as well in a relatively short order. :)

@M4rtinK M4rtinK merged commit 1bd7253 into master Jun 27, 2017
@M4rtinK
Copy link
Owner Author

M4rtinK commented Jun 29, 2017

modRana 0.54.7 with the fix has been released :)

@rinigus
Copy link
Contributor

rinigus commented Jun 30, 2017

Great, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants