Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Player migration is inefficient with large player numbers #7701

Open
Megaf opened this issue Sep 4, 2018 · 9 comments

Comments

Projects
None yet
4 participants
@Megaf
Copy link
Contributor

commented Sep 4, 2018

Issue type
  • Bug report
Minetest version
Minetest 0.4.17.1
OS / Hardware

Operating system: Debian Linux Stretch
CPU: Intel Core2Duo 2,4GHz

Summary

IRC discussion: http://irc.minetest.net/minetest-dev/2018-09-04#i_5400654

See later comments for hints at the problem.





I ran the command for 42 hours in hour of my servers, all it did was creating a players.bak directory.
It uses 100% of a single CPU core the whole time and no IO used. That was a 4 core ARM dedicated server. (Not Raspberry based, it is enterprise grade server with dedicated SSDs).

Tested again on my x86 MacBook Pro with a fast SSD. Same result. Long time running, single core in use at 100% and no IO.

Steps to reproduce

Have a big players plain file database and try to migrate to sqlite with minetestserver --migrate-players sqlite3

no IO wait CPU is free to get and send data.
screenshot at 2018-09-04 16-19-13
no IO use migrate is not doing anything, nor anything else in the system halting it.
screenshot at 2018-09-04 16-19-37
100% use of a single core Plenty of CPU available in the system, and migrate is clearly doing, or trying to do something.
screenshot at 2018-09-04 16-20-00
Zero feedback no feedback from the server whatsoever.
screenshot at 2018-09-04 16-20-24

Related #7697

@sfan5

This comment has been minimized.

Copy link
Member

commented Sep 4, 2018

Clearly works fine, closing.
./minetest --server --migrate-players sqlite3 --worldname test 0,02s user 0,01s system 30% cpu 0,124 total


protip: provide a way to reproduce your issue

@sfan5 sfan5 closed this Sep 4, 2018

@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

Just because it works in your box with a small player base it doesn't mean it works and my bug report is invalid.
Besides, I did tell how to reproduce.
Not only that, I used a command identical to yours now, with --verbose and --trace and still not output whatsoever.
image

@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

And this is what strace has to say.
image

@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

GDB output.
image

Thread 1 (Thread 0x7ffff7fc2740 (LWP 3797)):
#0  0x00007ffff66bd973 in __dynamic_cast () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#1  0x00007ffff672ff20 in bool std::has_facet > > >(std::locale const&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#2  0x00007ffff6723b9f in std::basic_ios >::_M_cache_locale(std::locale const&) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#3  0x00007ffff6723d00 in std::basic_ios >::init(std::basic_streambuf >*) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#4  0x00005555557daf06 in InventoryList::deSerialize(std::istream&) ()
No symbol table info available.
#5  0x00005555557dc313 in Inventory::deSerialize(std::istream&) ()
No symbol table info available.
#6  0x0000555555899956 in RemotePlayer::deSerialize(std::istream&, std::__cxx11::basic_string, std::allocator > const&, PlayerSAO*) ()
No symbol table info available.
#7  0x0000555555781d6a in PlayerDatabaseFiles::listPlayers(std::vector, std::allocator >, std::allocator, std::allocator > > >&) ()
No symbol table info available.
#8  0x00005555558dddf4 in ServerEnvironment::migratePlayersDatabase(GameParams const&, Settings const&) ()
No symbol table info available.
#9  0x00005555557f7fad in run_dedicated_server(GameParams const&, Settings const&) ()
No symbol table info available.
#10 0x00005555555f4acf in main ()
No symbol table info available.
@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

After interrupting minetestserver and continuing.
image

@sfan5 sfan5 changed the title [Bug] --migrate-players Does not work. Player migration is inefficient with large player numbers Sep 4, 2018

@sfan5 sfan5 reopened this Sep 4, 2018

@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

@sfan5

This comment has been minimized.

Copy link
Member

commented Sep 4, 2018

No.

@t0ny2

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2018

Maybe one of the user files that is being loaded is corrupt. It seems to me that InventoryList::deSerialize() would become an infinite loop if the "EndInventoryList" line was missing.

@Megaf

This comment has been minimized.

Copy link
Contributor Author

commented Sep 6, 2018

Anyone working on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.