diff update takes > 6hrs. Is that normal ?

[benjamin-rty]

Europe is already in my postgres db.
I use diff
I run for europe:
imposm3 diff
-cachedir='/home/xxx/cachedir'
-diffdir='/home/xxx/diffdir'
-mapping='/home/xxx/mapping_imposm3.json'
-connection postgis://xxx:xxx@xxx/osm_europe?sslmode=disable

The "Parsing changes, updating cache and removing elements" section took about 2hrs.
But the "Writing added/modified elements" section takes about 6 to 8 hrs.

Is that normal time behavior ?
I use Debian 7 ("wheezy") with 8GB ram / Intel(R) Core(TM) i7 CPU / 920 @ 2.67GHz

[olt]

It's likely to little memory. Imposm makes random access to the diff and cache files and each diff operation will likely hit the disk if these files do not fit in memory (cached by the OS). How large are these files for a Europe import?
You should see high IO load during the diff import (watch wa% in top).

[benjamin-rty]

The dif files for europe are about 30 MB.
The database in postgis containing the information is (configurated by my mapping file):
osm_roads is about 16 GB;
osm_industrial 0,1 GB;
osm_nature 5 GB;
osm_water 1.5 GB;
osm_residential 0.8 GB;

Should I have then > 24 GB ram for faster processing ? Or should I use a SSD ?

[olt]

I was talking about the cache files in /home/xxx/cachedir and /home/xxx/diffdir, but yes you will need IO (or free RAM for caching) for the database as well.
8GB is definitely to small for Europe. With DB and caches you will have more then 32 GB. SSD will help a lot if you don't have enough RAM.

[benjamin-rty]

ok. Its about 24 GB in the cache files en diffdir.

[benjamin-rty]

Or use an other machine with 32GB for running imposm3 en keep the one with 8GB with the db on it ?

[tartakynov]

Is postgis database still optimized after installing diffs?

[olt]

@tartakynov That's not really related to this issue, but the clustering degrades over time.