Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

std::bad_alloc in 0.4.12+ #2661

Closed
tenplus1 opened this issue May 1, 2015 · 32 comments

Comments

Projects
None yet
9 participants
@tenplus1
Copy link
Contributor

commented May 1, 2015

The same bug is happening on Xanadu server regularly and we are using 16x default textures and it hardly breaks to 500mb limit for memory use out of 4gb... from 0.4.12 upwards this std:bad_alloc error always crops up for reasons unknown...

2015-04-30 12:12:35: ACTION[ServerThread]: player G1127 crafts default:stick 4
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 1, 2015

That might mean a memory leak, something triggers it and it crashes, do you have any way to track your memory usage?

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 2, 2015

That's the thing, memory doesn't go up that much at all and it crashes randomly, one time 30 seconds into starting and connecting to server, others hours later, but memory is under 700mb...

@Zeno-

This comment has been minimized.

Copy link
Contributor

commented May 4, 2015

Is it always after player xxxxxxxxxxx crafts default:stick 4?

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 4, 2015

It's different each time, crafting item, placing block, punching mob etc. etc.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 5, 2015

If it's any help at all, while running around generating new areas of map underground the server crashed 3 times in so many minutes showing an std::bad_alloc error... Could it be related ?

@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 5, 2015

Hi @tenplus1
Please adjust your server settings according to the following:

max_simultaneous_block_sends_per_client = 3
max_block_send_distance = 3
max_block_generate_distance = 1
max_forceloaded_blocks = 0
active_object_send_range_blocks = 1
active_block_range = 5
item_entity_ttl = 60
server_unload_unused_data_timeout = 60

And tell us if the crash still happens.
I too get crashes on my server if I use number too high in these places, those suggested here are a good balance performance/stability I have found.

By the way, I'm running my server with 512 MB of RAM and no swap.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 6, 2015

Thanks megaf, I changed the settings in minetest.conf but unfortunately after a while the error still appears:

2015-05-06 00:19:21: ACTION[ServerThread]: player foxygirl5598 crafts farming:pumpkin
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
@C1ffisme

This comment has been minimized.

Copy link

commented May 6, 2015

I've noticed this error on singleplayer before. Sometimes it keeps throwing this when you try to open a world and you have to close minetest and open it back up again.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2015

Their is clearly a bug somewhere causing this behaviour and I really wish I knew what, it's causing Xanadu server to be a pain in the behind with it's constant crashing...

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 11, 2015

After speaking with devs on irc we narrowed this glitch down to a corrupt map error that when loading crashes the server and the only way to stop this was to mark the whole area in worldedit and remove it... Now this isn't the best solution to the problem since map errors can occur anywhere and finding the is a pain in itself, so maybe some error checking in the map loading functions of minetest to report corrupt location/node and continue onwards would be better...

@Zeno-

This comment has been minimized.

Copy link
Contributor

commented May 11, 2015

@tenplus1 You are certainly correct. A database corruption should not be able to bring a server to its knees because someone happens to walk into an area with a DB corruption. Hopefully one of us can reproduce this bug and squash it

@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 11, 2015

This bug is extremely old and might have been around since forever. I did actually shut down a good server I had 2 years ago because of this =/

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2015

Update on std::bad_alloc error... it seems newly generated areas also create map errors which cause this problem... I found out today as a player dug down and at a certain newly generated spot the server crashed again and continues to crash when in that same spot...

@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 13, 2015

hm, disable the mod vines if you have and see it happens again.

Or disable all mods, set everything to false in your world.mt and try to track down the mod is causing the crash,

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2015

Megaf, it's not a mod that's causing the crash but corrupt mapblocks being read by minetest and causing the server to crash... Devs know about this and will hopefully add a fix that displays corrupt node error and wont crash server...

@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 14, 2015

Yep, but you said that it happens on newly generated map, there are mods that change how the map is generated, my theory is that some mod is somehow causing the map to be incorrectly generated, causing the corrupt node.

I had problems with falling saplings from leaves trying to spawn inside stone nodes, causing a similar crash.

That happened when I used worldedit to replace air by stone in an area that had some leaves.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2015

Megaf: the only mod in use that changes the map is Ethereal and that has been thorougly tested and complies with 0.4.12...

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2015

2015-05-15 16:03:14: ACTION[ServerThread]: hippagriff places node default:cobble at (-1114,9,894)
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 25, 2015

This is getting ridiculous now:

2015-05-24 15:07:40: ACTION[ServerThread]: rft10 takes stuff from locked chest at (-86,25,261)
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
@kwolekr

This comment has been minimized.

Copy link
Contributor

commented May 25, 2015

What would be extremely helpful here is some valgrind and/or massif output around the time of the crash.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 25, 2015

What would you need enabled in settings to do that ? The logfile before the crash only ever contains normal chatter like players placing nodes, using items, hitting mobs... This error is so out of the blue...

@Megaf

This comment has been minimized.

Copy link
Contributor

commented May 25, 2015

@tenplus1 Are you on IRC?

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 26, 2015

Yes I have IRC but the server in question isn't my own and is situated overseas...

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented May 30, 2015

We ran: gdb --args minetest --server and this is what came out on crash:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffe65f2700 (LWP 2919)]
0x00007ffff5009cc9 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb)
@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented Jun 22, 2015

(21/06/15 21:19:50) 015-06-21 16:19:01: ERROR[main]: UNRECOVERABLE error occurred. Stopping server. Please fix the following error:
2015-06-21 16:19:01: ERROR[main]: error in error handling

In thread 7f1e796cf7c0:
/build/minetest-9tQZQ2/minetest-201506170131/src/server.cpp:505: void Server::step(float): A fatal error occurred: error in error handling
Debug stacks:
DEBUG STACK FOR THREAD 7f1e6587e700:
#0  virtual void* EmergeThread::Thread()
(Leftover data: #1  MapBlock* ServerMap::loadBlock(v3s16))
(Leftover data: #2  void ServerMap::loadBlock(std::string*, v3s16, MapSector*, bool))
(Leftover data: #3  void ItemStack::deSerialize(std::istream&, IItemDefManager*))

(21/06/15 21:20:43) DEBUG STACK FOR THREAD 7f1e66dfc700:
#0  virtual void* EmergeThread::Thread()
(Leftover data: #1  MapBlock* ServerMap::loadBlock(v3s16))
(Leftover data: #2  void ServerMap::loadBlock(std::string*, v3s16, MapSector*, bool))
(Leftover data: #3  void ItemStack::deSerialize(std::istream&, IItemDefManager*))

(21/06/15 21:21:12) DEBUG STACK FOR THREAD 7f1e675fd700:
#0  virtual void* CurlFetchThread::Thread()
DEBUG STACK FOR THREAD 7f1e67dfe700:
#0  virtual void* ServerThread::Thread()
#1  void Server::Receive()
(Leftover data: #2  void Server::SendBlocks(float))
(Leftover data: #3  void RemoteClient::GetNextBlocks(ServerEnvironment*, EmergeManager*, float, std::vector<PrioritySortedBlockTransfer>&))
(Leftover data: #4  void ItemStack::serialize(std::ostream&) const)
(Leftover data: #5  bool getCraftingResult(Inventory*, ItemStack&, bool, IGameDef*))
(Leftover data: #6  void ItemStack::deSerialize(std::istream&, IItemDefManager*))

(21/06/15 21:21:27) DEBUG STACK FOR THREAD 7f1e796cf7c0:
#0  int main(int, char**)
#1  Dedicated server branch
#2  void dedicated_server_loop(Server&, bool&)
#3  void Server::step(float)
@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented Jul 4, 2015

Interesting news, after running lag01's script to remove any and all errors from Xanadu's map, the std::bad_alloc error still appears in the world so at least we've ruled out a corrupt map as the reason behind it... It still happens on a fresh map from time to time also when not doing anything specific, so a bug that really has to be found...

@FreeLikeGNU

This comment has been minimized.

Copy link

commented Jul 12, 2015

I was getting crashes like this from the ppa build of 4.12 ( https://launchpad.net/~minetestdevs/+archive/ubuntu/stable ). I built from the git repo source https://github.com/minetest/minetest and have not had the crashes return.

@est31 est31 added Bug Blocker labels Aug 10, 2015

@est31 est31 added the High priority label Aug 10, 2015

@est31

This comment has been minimized.

Copy link
Contributor

commented Aug 10, 2015

This has been reported by forum users too: https://forum.minetest.net/viewtopic.php?f=6&t=12917

@ShadowNinja

This comment has been minimized.

Copy link
Member

commented Sep 14, 2015

@tenplus1: Please run bt full in GDB after getting the crash, your log only shows the error handler in the stdlib.

@0-afflatus

This comment has been minimized.

Copy link

commented Dec 10, 2015

I'm getting these frequently in Grail Test on 0.4.13
The word that keeps leaping out at me is (de)serialize
AFAICT they are triggered by mapgen, so not due to an existing corrupted map.
I would need serious education to provide any kind of backtrace. sorry.

@tenplus1

This comment has been minimized.

Copy link
Contributor Author

commented Mar 4, 2016

These errors haven't came up in a looong time with the newer 0.4.13 daily builds, am hoping whatever was causing it is gone for good.

@tenplus1 tenplus1 closed this Mar 4, 2016

@0-afflatus

This comment has been minimized.

Copy link

commented Mar 4, 2016

I'm not entirely convinced this is completely fixed. However, I now know how to run gdb, so I'll open a new issue if I can get a more precise fix on the actual problem.

OT: @tenplus1 you mentioned lag01's script - is this recover.py or something else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.