PR for Modernizing for C++11: replacing instances of SmallVector #6817

M3Henry · 2018-06-06T15:09:24Z

After discussions on IRC about modernising the code to make it easier to work on, replacing non time-critical instances of SmallVector<T,S> with std::vector was deemed more practical than making SmallVector STL-like.

This allows use of range-based for loops and STL algorithms to improve expressiveness in the code base.

No changes to functionality have been made.

#6649

…over SmallVector

…er SmallVector

…mallVector

…ector

::utf16_str ::utf16_to_utf8

…llVector

…ver SmallVector

…mallVector

…r SmallVector

…lVector

…allVector

LordAro · 2018-06-06T16:24:30Z

Some build system changes will be needed to enforce c++11/14

nielsmh · 2018-06-23T14:55:28Z

LinkGraphJob::NodeAnnotationVector and GUIBridgeList (in bridge_gui.cpp) are both vectors of non-POD data types so those two are the real important ones to fix.

The former does get fixed by these changes, but GUIBridgeList does not, and from my cursory check it might be more complex to fix that one. The reason the contained data is non-POD is because of the inclusion of Money fields, that type has custom constructors which makes it non-POD.

TrueBrain

Sorry for the slow responses; we have not forgotten about this PR. I did a fast pass on the code, and "it looks fine". The main issue with patches like this is of course that reviewing takes a bit of time ;)

As mentioned earlier, it would also need some changes to the configure system, as it should detect if the used compiler can handle C++11, and warn if not. Otherwise we will have a bug-tracker full of the same error in no time :)

So please bare with us while we prepare for modern standards ;)

TrueBrain · 2018-07-22T19:53:01Z

src/network/core/host.cpp

@@ -102,7 +102,18 @@ static void NetworkFindBroadcastIPsInternal(NetworkAddressList *broadcast) // GE
 		if (ifa->ifa_broadaddr->sa_family != AF_INET) continue;

 		NetworkAddress addr(ifa->ifa_broadaddr, sizeof(sockaddr));
-		if (!broadcast->Contains(addr)) *broadcast->Append() = addr;
+		//if (!broadcast->Contains(addr)) *broadcast->Append() = addr;


I feel that this was by accident?

The removal was intentional, lines 108-116 replace it, but I shouldn't have left it commented out.
TBH the following would be much better than a hand rolled loop.

if (broadcast->end() == std::find(broadcast->begin(), broadcast->end(), addr) broadcast->push_back(addr);

M3Henry · 2018-07-27T22:47:18Z

@TrueBrain If the intention is to keep modernizing the code, then I can go over this patch series and do it properly. Looking back on this, it doesn't seem I put forward my best.

LordAro

First pass at an actual review, mostly codestyle related.

Firstly, great job on actually getting this done and working - it's a huge amount of effort (I tried SmallMap once, I gave up pretty quickly)

However, if I'm honest, I'm not sure this will ever get merged in its current form. There are just too many separate changes all bundled together. It would be better if it were a separate commit per change, rather than per file/type. Something like a commit to add new adapter functions, then one that actually converts types & functions (huge commit, but fundamentally quite simple to read through), then followed by a few commits that "neaten" usage (for loops, auto, etc)

Couple general comments:

Usage of and, or & not instead of &&, || & !
Lots of uses of auto here. Its desired usage in OTTD is yet to be formalised, but IMO it should only be used when the type of the variable doesn't matter, or if the typename has already been stated somewhere nearby. There's a few places in which that's not happened

LordAro · 2018-09-19T07:56:52Z

src/core/smallvec_type.hpp

@@ -7,14 +7,131 @@
 * See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with OpenTTD. If not, see <http://www.gnu.org/licenses/>.
 */

-/** @file smallvec_type.hpp Simple vector class that allows allocating an item without the need to copy this->data needlessly. */
+/** @file smallvec_type.hpp Simple vector class that allows allocating an item without the need to copy this->data needlessly.


text should be on a separate line, to preserve alignment

(applies to lots of functions)

LordAro · 2018-09-19T07:56:52Z

src/core/smallvec_type.hpp

+ * @tparam T the contained type of the vector
+ */
+template <typename T>
+typename std::vector<T>::const_iterator Find(const std::vector<T>& vec, const T& elem) {


opening brace of a function should be on a newline

(applies to various functions)

Suggested change

typename std::vector<T>::const_iterator Find(const std::vector<T>& vec, const T& elem) {

typename std::vector<T>::const_iterator Find(const std::vector<T>& vec, const T& elem)

{

LordAro · 2018-09-19T07:56:52Z

src/core/smallvec_type.hpp

+ * @param vec the vector to be searched
+ * @param elem the element to search for
+ * @tparam T the contained type of the vector
+ */


missing @return doxygen command

(applies to various functions)

Suggested change

*/

* @return an iterator to the position of the element if it was found. vec.end() otherwise.

*/

LordAro · 2018-09-19T07:56:52Z

src/economy.cpp

@@ -77,7 +77,7 @@ static inline int32 BigMulS(const int32 a, const int32 b, const uint8 shift)
 	return (int32)((int64)a * (int64)b >> shift);
 }

-typedef SmallVector<Industry *, 16> SmallIndustryList;
+using SmallIndustryList = std::vector<Industry *> ;


extra space before ;

Suggested change

using SmallIndustryList = std::vector<Industry *> ;

using SmallIndustryList = std::vector<Industry *>;

LordAro · 2018-09-19T07:56:52Z

src/fios.cpp

@@ -397,7 +397,7 @@ static void FiosGetFileList(SaveLoadOperation fop, fios_getlist_callback_proc *c
 	{
 		SortingBits order = _savegame_sort_order;
 		_savegame_sort_order = SORT_BY_NAME | SORT_ASCENDING;
-		QSortT(file_list.files.Begin(), file_list.files.Length(), CompareFiosItems);
+		QSortT(file_list.files.data(), file_list.Length(), CompareFiosItems);


It's a bit more work involved, but since you're converting some stuff in this PR other than just converting SmallVector, perhaps converting to use std::sort would be a good idea as well

Alternatively, it should be a separate PR (along with the various other changes in this PR) :)

I agree that the standard algorithms would be ideal where applicable, a seperate PR would make most sense.

Here an untested suggestion: (at least #include<algorithm> is missing)

Suggested change

QSortT(file_list.files.data(), file_list.Length(), CompareFiosItems);

std::sort(file_list.files.data(), file_list.files.data() + file_list.Length(), CompareFiosItems);

LordAro · 2018-09-19T07:56:52Z

src/network/network_content.h

 	/** Get the end of the content inf iterator. */
-	ConstContentIterator End() const { return this->infos.End(); }
+	ConstContentIterator End() const { return this->infos.end(); }


This looks like another class that could be refactored to be "better", instead of reimplementing a load of std::vector functions

LordAro · 2018-09-19T07:56:52Z

src/station_gui.cpp

 		}
+		else ++ts;


braces required here

src/texteff.cpp

LordAro · 2018-09-19T07:56:52Z

src/timetable_cmd.cpp

 			w->lateness_counter = 0;
 			ClrBit(w->vehicle_flags, VF_TIMETABLE_STARTED);
 			/* Do multiplication, then division to reduce rounding errors. */
-			w->timetable_start = start_date + idx * total_duration / num_vehs / DAY_TICKS;
+			w->timetable_start = start_date + idx++ * total_duration / num_vehs / DAY_TICKS;


idx++ in the middle is a bit scary, should probably be in a separate statement

LordAro · 2018-09-19T07:56:52Z

src/core/smallvec_type.hpp

+ * @tparam T the contained type of the vector
+ */
+template <typename T>
+void Reset(std::vector<T>& vec) {


pretty sure this function isn't necessary - can be replaced with a clear() - the capacity reduction was just due to how the allocations worked

It may be a good idea to keep the capacity reduction where it is used in case that behavior is desired.
According to cppreference: "Leaves the capacity() of the vector unchanged"

shrink_to_fit is only a request which may be ignored.
The guaranteed implementation looks like this:

template <typename T> void Reset(std::vector<T>& vec) { std::vector<T>{}.swap(vec); }

M3Henry · 2018-09-19T16:06:47Z

@LordAro
Rather than doing once large PR for a bunch of types at once, would it make more sense to submit one PR per type and do a proper conversion using standard algorithms with each one? (To avoid a PR flood I'd submit the next PR once the previous one is accepted)

TrueBrain · 2019-01-05T21:23:18Z

We recently switched from Jenkins as CI to Azure Pipelines as CI. This means you need to rebase before this Pull Request will pass its checks. Sorry for the troubles! (might not be applicable to this PR)

nielsmh · 2019-01-21T15:26:16Z

@M3Henry Yes I think several smaller PRs would be easier to integrate.

With the new build infrastructure I think usage of C++11 and possibly C++14 is good to go too.

M3Henry · 2019-01-21T21:25:46Z

That's good to hear, I've been putting off finishing this because such a large PR needs some contiguous time to go over. I may be able to put up a first batch this weekend if the small PR direction is desirable.

…

On Mon, 21 Jan 2019 at 15:26, Niels Martin Hansen ***@***.***> wrote: @M3Henry <https://github.com/M3Henry> Yes I think several smaller PRs would be easier to integrate. With the new build infrastructure I think usage of C++11 and possibly C++14 is good to go too. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6817 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGyG_kfpt1Jxlr-BDYhq4rx4z1fbbAQzks5vFdwcgaJpZM4Uc1hT> .

-- Regards, Henry.

fsimonis · 2019-01-23T12:28:16Z

src/core/smallvec_type.hpp

+ * @param vec the vector to be searched
+ * @param elem the element to search for
+ * @tparam T the contained type of the vector
+ */


Suggested change

*/

* @return true if vec contains elem. false otherwise.

*/

fsimonis · 2019-01-23T12:29:40Z

src/core/smallvec_type.hpp

+ * @param vec the vector to be searched
+ * @param elem the element to count instances of
+ * @tparam T the contained type of the vector
+ */


Suggested change

*/

* @return the number of occurrences.

*/

fsimonis · 2019-01-23T12:33:39Z

src/core/smallvec_type.hpp

+ * @param elem the element to add for and remove if missing
+ * @tparam T the contained type of the vector
+ * @tparam U the universal reference type for perfect forwarding
+ */


Suggested change

*/

* @return true if the element had to be added.

*/

fsimonis · 2019-01-23T12:41:53Z

src/core/smallvec_type.hpp

+/** Remove an element referenced in the vector and move the last element to take its place.
+ * @param vec the vector to be modified
+ * @param it the element to remove
+ * @tparam T the contained type of the vector


Suggested change

* @tparam T the contained type of the vector

* @tparam T the contained type of the vector

* @return the iterator it

fsimonis · 2019-01-23T12:43:09Z

src/core/smallvec_type.hpp

+ * @param vec the vector to be modified
+ * @param elem the element to search for and remove if found
+ * @tparam T the contained type of the vector
+ */


Suggested change

*/

* @return true if the element was removed, false if it was not found.

*/

fsimonis · 2019-01-23T12:55:02Z

src/core/smallvec_type.hpp

+/** Extend a vector by an amount and return an iterator to the new elements.
+ * @param vec the vector to be modified
+ * @param num the number of elements to add
+ * @tparam T the contained type of the vector


Suggested change

* @tparam T the contained type of the vector

* @tparam T the contained type of the vector

* @return an iterator to the first of the added elements.

fsimonis · 2019-01-23T13:12:36Z

src/fios.h

@@ -131,7 +131,7 @@ class FileList {
 	 */
 	inline const FiosItem *Begin() const
 	{
-		return this->files.Begin();
+		return this->files.data();


Make sure the vector contains files.

Suggested change

return this->files.data();

return &(this->files->at(0));

fsimonis · 2019-01-23T13:16:20Z

src/fios.h

@@ -149,7 +149,7 @@ class FileList {
 	 */
 	inline const FiosItem *Get(uint index) const
 	{
-		return this->files.Get(index);
+		return this->files.data() + index;


Suggested change

return this->files.data() + index;

return &(this->files.at(index));

fsimonis · 2019-01-23T13:17:03Z

src/fios.h

@@ -158,7 +158,7 @@ class FileList {
 	 */
 	inline FiosItem *Get(uint index)
 	{
-		return this->files.Get(index);
+		return this->files.data() + index;


Suggested change

return this->files.data() + index;

return &(this->files.at(index));

fsimonis · 2019-01-23T13:22:26Z

src/fios.h

 	}

 	/** Compact the list down to the smallest block size boundary. */
 	inline void Compact()
 	{
-		this->files.Compact();
+		this->files.shrink_to_fit();


Use the "swap-trick" to force the shrink.

Suggested change

this->files.shrink_to_fit();

decltype(files)(this->files).swap(this->files);

M3Henry added 30 commits Dec 30, 2017

Add: New adapter functions to help transition from SmallVector

6447f50

Codechange: Make Window::scheduled_invalidation_data use std::vector …

e397df9

…over SmallVector

Codechange: Make SubtypeList use std::vector over SmallVector

ec544d7

Codechange: Make GetBestFittingSubType()::subtypes use std::vector ov…

cd8d6cc

…er SmallVector

Codechange: Make TextfileWindow::lines use std::vector over SmallVector

ccf3c5c

Codechange: Make CheckSubsidised()::towns_near use std::vector over S…

fbc5e75

…mallVector

Codechange: Make StringFilter::word_index use std::vector over SmallV…

e894abe

…ector

Codechange: Make IcuStringIterator sung std::vector over SmallVector

daa7997

::utf16_str ::utf16_to_utf8

Codechange: Make SmallIndustryList use std::vector over SmallVector

2561105

Codechange: Make FileList::files use std::vector over SmallVector

c95b308

Codechange: Make VehicleGroupWindow::indents use std::vector over Sma…

bee3249

…llVector

Codechange: Make _hotkey_lists use std::vector over SmallVector

8897223

Codechange: Make Hotkey::keycodes use std::vector over SmallVector

7845eed

Codechange: Make LanguageList use std::vector over SmallVector

573b1b2

Codechange: Make LinkGraph::NodeVector use std::vector over SmallVector

27c0082

Codechange: Make LinkGraphJob::NodeAnnotationVector use std::vector o…

72d73e3

…ver SmallVector

Codechange: Make NetworkAddressList use std::vector over SmallVector

445bcdb

Codechange: Make _tcp_connectors use std::vector over SmallVector

f748fcd

Codechange: Make _http_connections use std::vector over SmallVector

633629c

Codechange: Make IndustryCargoesWindow::Fields use std::vector over S…

40dd391

…mallVector

Codechange: Make ContentVector use std::vector over SmallVector

81b7b07

Codechange: Make GetLinkGraphJobDesc()::saveloads use std::vector ove…

d6fba11

…r SmallVector

Codechange: Make _old_waypoints use std::vector over SmallVector

054063e

Codechange: Make StationList use std::vector over SmallVector

2f119f1

Codechange: Make _text_effects use std::vector over SmallVector

43b6222

Codechange: Make OutputStore::output_buffer use std::vector over Smal…

e660300

…lVector

Codechange: Make TrainList use std::vector over SmallVector

9804906

Codechange: Make CmdSetTimetableStart()::vehs use std::vector over Sm…

d4d0b5d

…allVector

Codechange: Make AfterLoadGame()::skip_frames use std::vector over Sm…

1f3845a

…allVector

Codechange: Make station_gui.cpp SmallVectors use std::vector over Sm…

Loading status checks…

072d1b9

…allVector

TrueBrain added the needs review label Jul 19, 2018

TrueBrain reviewed Jul 22, 2018

View changes

M3Henry closed this Jul 27, 2018

M3Henry reopened this Jul 27, 2018

LordAro requested changes Sep 19, 2018

View changes

LordAro added waiting on author and removed needs review labels Sep 19, 2018

LordAro added the wip label Oct 27, 2018

LordAro mentioned this pull request Jan 5, 2019

Modernizing for C++11: replacing instances of SmallVector #6649

Closed

andythenorth added question stale labels Jan 6, 2019

fsimonis reviewed Jan 23, 2019

View changes

stale bot closed this Jan 30, 2019

M3Henry mentioned this pull request Feb 3, 2019

[core] Implement SmallVector using std::vector #7165

Merged

	typename std::vector<T>::const_iterator Find(const std::vector<T>& vec, const T& elem) {
	typename std::vector<T>::const_iterator Find(const std::vector<T>& vec, const T& elem)
	{

	*/
	* @return an iterator to the position of the element if it was found. vec.end() otherwise.
	*/

	using SmallIndustryList = std::vector<Industry *> ;
	using SmallIndustryList = std::vector<Industry *>;

	QSortT(file_list.files.data(), file_list.Length(), CompareFiosItems);
	std::sort(file_list.files.data(), file_list.files.data() + file_list.Length(), CompareFiosItems);

	* @tparam T the contained type of the vector
	* @tparam T the contained type of the vector
	* @return the iterator it

	*/
	* @return true if the element was removed, false if it was not found.
	*/

	* @tparam T the contained type of the vector
	* @tparam T the contained type of the vector
	* @return an iterator to the first of the added elements.

	return this->files.data() + index;
	return &(this->files.at(index));

	this->files.shrink_to_fit();
	decltype(files)(this->files).swap(this->files);

OpenTTD / OpenTTD

Sponsor OpenTTD/OpenTTD

PR for Modernizing for C++11: replacing instances of SmallVector #6817

PR for Modernizing for C++11: replacing instances of SmallVector #6817

Conversation

M3Henry commented Jun 6, 2018 • edited

LordAro commented Jun 6, 2018

nielsmh commented Jun 23, 2018 • edited

TrueBrain left a comment

This comment has been minimized.

TrueBrain Jul 22, 2018 Member

This comment has been minimized.

M3Henry Jul 27, 2018 Author Contributor

M3Henry commented Jul 27, 2018

LordAro left a comment

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

M3Henry Sep 19, 2018 • edited Author Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 • edited Contributor

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

LordAro Sep 19, 2018 Member

This comment has been minimized.

M3Henry Sep 20, 2018 Author Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

M3Henry commented Sep 19, 2018

TrueBrain commented Jan 5, 2019

nielsmh commented Jan 21, 2019

M3Henry commented Jan 21, 2019

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 • edited Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

This comment has been minimized.

fsimonis Jan 23, 2019 Contributor

M3Henry commented Jun 6, 2018 •

edited

nielsmh commented Jun 23, 2018 •

edited

TrueBrain Jul 22, 2018
Member

M3Henry Jul 27, 2018
Author Contributor

LordAro Sep 19, 2018
Member

LordAro Sep 19, 2018
Member

fsimonis Jan 23, 2019
Contributor

LordAro Sep 19, 2018
Member

fsimonis Jan 23, 2019
Contributor

LordAro Sep 19, 2018
Member

fsimonis Jan 23, 2019
Contributor

LordAro Sep 19, 2018
Member

M3Henry Sep 19, 2018 •

edited

Author Contributor

fsimonis Jan 23, 2019 •

edited

Contributor

LordAro Sep 19, 2018
Member

LordAro Sep 19, 2018
Member

LordAro Sep 19, 2018
Member

LordAro Sep 19, 2018
Member

M3Henry Sep 20, 2018
Author Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019 •

edited

Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor

fsimonis Jan 23, 2019
Contributor