[core] Implement SmallVector using std::vector #7165

M3Henry · 2019-02-03T11:44:19Z

This the starting point for removing SmallVector by making SmallVector an adapter type.
This replaces #6817.

The public and protected interface to SmallVector are unchanged
SmallVector now requires that items be default constructible
This isn't an issue since some contained items were previously created
uninitialized.

Temporary default constructors are added to the following structs

SmallPair
SmallStackItem
GRFPresence

Where vector is required, transition immediately to std::vector
to avoid returning proxy object references.

LordAro · 2019-02-04T20:16:55Z

I'd be interested to see if there's any significant differences in compile and/or run time for this method

M3Henry · 2019-02-04T23:45:07Z

Running make clean; /usr/bin/time -ao result.txt twice:

Old:

189.65user 11.06system 3:21.75elapsed 99%CPU (0avgtext+0avgdata 262996maxresident)k
0inputs+200152outputs (0major+6462169minor)pagefaults 0swaps
188.01user 11.27system 3:20.21elapsed 99%CPU (0avgtext+0avgdata 262928maxresident)k
0inputs+200136outputs (0major+6458952minor)pagefaults 0swaps

New:

209.38user 12.56system 3:42.99elapsed 99%CPU (0avgtext+0avgdata 288344maxresident)k
8176inputs+202184outputs (4major+7621408minor)pagefaults 0swaps
212.88user 12.72system 3:46.66elapsed 99%CPU (0avgtext+0avgdata 288408maxresident)k
0inputs+202184outputs (0major+7622902minor)pagefaults 0swaps

So 11% slower compilation, not surprising seeing as this is an intermediate shim. I expect this to improve once SmallVector is removed completely, as only one template will be instantiated per use, rather than two.

Run time performance would hopefully be better, owing to exponential growth of storage rather than linear growth. The loss of realloc may counter that to some degree. But if trivially relocatable types (P1144R1) are approved for C++20 then that would be resolved automatically.

Also explicit template instantiation should yield good results.

PeterN · 2019-02-04T23:47:32Z

Run-time performance is far more relevant that compile-time.

M3Henry · 2019-02-05T00:26:23Z

Is there a benchmark for that?

Eddi-z · 2019-02-05T05:51:07Z

you used to be able to run a non-gui benchmark with "-v null:ticks=1000"

time ./openttd -c path_to_config -v null:ticks=1000 -m null -s null -g path_to_savegame

M3Henry · 2019-02-05T19:36:57Z

Running for 10000 ticks I got:

Real -2.1%
User -0.7%
Sys -12.3%

So a boost in all categories!

std::vector

real	0m1.005s
user	0m1.068s
sys	0m0.085s

real	0m1.014s
user	0m1.094s
sys	0m0.056s

real	0m1.010s
user	0m1.122s
sys	0m0.024s

real	0m1.008s
user	0m1.116s
sys	0m0.036s
--------------------
real	0m1.00925s
user	0m1.10000s
sys	0m0.05025s
====================

SmallVector

real	0m1.023s
user	0m1.103s
sys	0m0.064s

real	0m1.033s
user	0m1.095s
sys	0m0.072s

real	0m1.028s
user	0m1.117s
sys	0m0.044s

real	0m1.040s
user	0m1.114s
sys	0m0.049s
--------------------
real	0m1.03100s
user	0m1.10725s
sys	0m0.05730s

TrueBrain · 2019-02-10T17:48:16Z

Possibly it might be good to run for more ticks; comparing two results based on runs of ~1 second means you are comparing heavily in the noise ;) Longer runtimes will give more comparable results :)

glx22 · 2019-02-10T18:32:56Z

It seems you forgot to replace some Clear() instances (as implied by the CI failure)

glx22 · 2019-02-10T21:27:37Z

And now it fails on Length()

Edit: missed the forced push ;)
Edit2: ok OS specific files are hard to check locally :)

PeterN · 2019-02-10T22:41:40Z

~~This is turning into a somewhat large PR. Might it make sense to start individually converting some uses of SmallVector directly to std::vector in stages instead?~~

LordAro · 2019-02-10T22:47:28Z

This was discussed in the previous version of the PR. I thought it made more sense to do the change in individual commits, but the same PR

PeterN · 2019-02-10T22:49:51Z

I didn't say mean split into separate PRs, I'm suggesting to switch individual uses of SmallVector to std::vector directly instead of changing the SmallVector type, then hopefully that can just be dropped.

~~And yes, I realise doing this means throwing away a large portion of this PR :-(~~

EDIT: Reviewing individual commits is a must for this PR!

LordAro · 2019-02-10T22:53:10Z

Maybe, maybe not. I figured that reviewing method a -> method b (lots of single line changes) was easier to review than object a -> object b (lots of bigger changes), hence the move to make SmallVector a subclass of std::vector

src/core/smallmap_type.hpp

src/core/smallvec_type.hpp

src/newgrf.cpp

…data() + n

…place]_back()

…o[Free|Delete]SmallVector

PeterN

I'm happy with this, as it stands. There may be minor tweaks here and there but they can be done in master.

…ng types. Const and non-const Find() have different return types.

Const and non-const Find() have different return types.

…et relative' button crashes the game. The 'offs_start_map' is a 'SmallMap', so its own 'Erase' function should be called instead of the underlying vector's 'erase' function. And fix a "typo". :)

…ng types. Const and non-const Find() have different return types.

…nt (OpenTTD#7548)

M3Henry force-pushed the c++11 branch from e757da5 to e1e7bf9 Feb 10, 2019

M3Henry force-pushed the c++11 branch 2 times, most recently from cf52201 to 38cf2f2 Feb 10, 2019

M3Henry force-pushed the c++11 branch 2 times, most recently from 2b3a8a8 to f319524 Feb 10, 2019

PeterN reviewed Feb 10, 2019

View changes

src/core/smallmap_type.hpp Show resolved Hide resolved

PeterN reviewed Feb 10, 2019

View changes

src/core/smallvec_type.hpp Outdated Show resolved Hide resolved

M3Henry force-pushed the c++11 branch 4 times, most recently from 1adf753 to 6c6f630 Feb 19, 2019

M3Henry force-pushed the c++11 branch from 6c6f630 to aec6277 Feb 19, 2019

LordAro reviewed Feb 20, 2019

View changes

src/newgrf.cpp Outdated Show resolved Hide resolved

M3Henry force-pushed the c++11 branch 2 times, most recently from f526d6b to 862c3a8 Feb 20, 2019

PeterN mentioned this pull request Feb 22, 2019

Change: Non-rectangular sparse station catchment area #7235

Merged

M3Henry force-pushed the c++11 branch from ca07d36 to 1e5d3f6 Mar 3, 2019

M3Henry added 12 commits Mar 26, 2019

Codechange: Replaced SmallVector::Get(n) non-const with std::vector::…

414c734

…data() + n

Codechange: Replaced SmallVector::Get() const with std alternatives

d01bcd0

Codechange: Replaced SmallVector::Erase() with std::vector::erase()

6010785

Codechange: Replaced SmallVector::Append() with std::vector::[push|em…

ef363c0

…place]_back()

Codechange: Removed SmallVector::Insert()

dde619b

Codechange: Replaced SmallVector::Find() with std::find()

faec3a4

Codechange: Replaced SmallVector::Include() with include()

4b1dba0

Codechange: Replaced SmallVector::[Begin|End]() with std alternatives

155fb1d

Codechange: Declare SmallVector as an alias for std::vector

6d5c72e

Codechange: Removed SmallVector completely

9a2e152

Cleanup: Remove unused size template parameters from SmallMap and Aut…

c1bc4d3

…o[Free|Delete]SmallVector

Codechange: Use range-based for-loop in Auto[Free|Delete]SmallVector

5afd853

M3Henry force-pushed the c++11 branch from 13952c1 to 5afd853 Mar 26, 2019

PeterN approved these changes Mar 26, 2019

View changes

LordAro approved these changes Mar 26, 2019

View changes

PeterN merged commit 03ca319 into OpenTTD:master Mar 26, 2019
8 checks passed

stormcone added a commit to stormcone/OpenTTD that referenced this issue Mar 26, 2019

Fix OpenTTD#7165: Missed 'Append() --> push_back()' replacement

cfb60bf

LordAro added a commit that referenced this issue Mar 26, 2019

Fix #7165: Missed 'Append() --> push_back()' replacement

b913c92

stormcone added a commit to stormcone/OpenTTD that referenced this issue Mar 27, 2019

Fix OpenTTD#7165: SmallMap::Erase(key) does not work correctly

5be0b41

michicc added a commit to stormcone/OpenTTD that referenced this issue Mar 27, 2019

Fix OpenTTD#7165: Const overload SmallMap::Contains(key) compared wro…

2be07ec

…ng types. Const and non-const Find() have different return types.

LordAro added a commit that referenced this issue Mar 28, 2019

Fix #7165: SmallMap::Erase(key) does not work correctly

ee260e4

LordAro added a commit that referenced this issue Mar 28, 2019

Fix #7165: Const overload SmallMap::Contains(key) compared wrong types.

21d9e87

Const and non-const Find() have different return types.

PeterN added a commit to PeterN/OpenTTD that referenced this issue Apr 27, 2019

Fix OpenTTD#7165: Missed another 'Append() --> push_back()' replacement

a4b0ddb

PeterN added a commit that referenced this issue Apr 27, 2019

Fix #7165: Missed another 'Append() --> push_back()' replacement (#7548)

0ab6244

douiwby added a commit to douiwby/OpenTTD that referenced this issue Apr 16, 2020

Fix OpenTTD#7165: Missed 'Append() --> push_back()' replacement

63748ea

douiwby added a commit to douiwby/OpenTTD that referenced this issue Apr 16, 2020

Fix OpenTTD#7165: SmallMap::Erase(key) does not work correctly

7e1de80

douiwby added a commit to douiwby/OpenTTD that referenced this issue Apr 16, 2020

Fix OpenTTD#7165: Const overload SmallMap::Contains(key) compared wro…

3bc8f78

…ng types. Const and non-const Find() have different return types.

douiwby added a commit to douiwby/OpenTTD that referenced this issue Apr 16, 2020

Fix OpenTTD#7165: Missed another 'Append() --> push_back()' replaceme…

a93b1d5

…nt (OpenTTD#7548)

spnda added a commit to spnda/OpenTTD that referenced this issue Jun 17, 2020

Fix OpenTTD#7165: Missed another 'Append() --> push_back()' replaceme…

254a839

…nt (OpenTTD#7548)

OpenTTD / OpenTTD Public

[core] Implement SmallVector using std::vector #7165

[core] Implement SmallVector using std::vector #7165

M3Henry commented Feb 3, 2019

LordAro commented Feb 4, 2019

M3Henry commented Feb 4, 2019

PeterN commented Feb 4, 2019

M3Henry commented Feb 5, 2019

Eddi-z commented Feb 5, 2019 •

edited

M3Henry commented Feb 5, 2019

TrueBrain commented Feb 10, 2019

glx22 commented Feb 10, 2019

glx22 commented Feb 10, 2019 •

edited

PeterN commented Feb 10, 2019 •

edited

LordAro commented Feb 10, 2019

PeterN commented Feb 10, 2019 •

edited

LordAro commented Feb 10, 2019

PeterN left a comment

OpenTTD / OpenTTD Public

[core] Implement SmallVector using std::vector #7165

[core] Implement SmallVector using std::vector #7165

Conversation

M3Henry commented Feb 3, 2019

LordAro commented Feb 4, 2019

M3Henry commented Feb 4, 2019

PeterN commented Feb 4, 2019

M3Henry commented Feb 5, 2019

Eddi-z commented Feb 5, 2019 • edited

M3Henry commented Feb 5, 2019

TrueBrain commented Feb 10, 2019

glx22 commented Feb 10, 2019

glx22 commented Feb 10, 2019 • edited

PeterN commented Feb 10, 2019 • edited

LordAro commented Feb 10, 2019

PeterN commented Feb 10, 2019 • edited

LordAro commented Feb 10, 2019

PeterN left a comment

Eddi-z commented Feb 5, 2019 •

edited

glx22 commented Feb 10, 2019 •

edited

PeterN commented Feb 10, 2019 •

edited

PeterN commented Feb 10, 2019 •

edited