Islands get randomly deleted under specific circumstances after server restart

[baticag]

Description
After server restart, some players islands get deleted. They still appear to be on the world, but no data in database is valid for them.

Steps to reproduce the behavior:
Don't know.

Expected behavior
Data saved in database and players having islands.

Server Information:

• Database being used: Tried first YML, then MySQL - same
• OS: Debian GNU/Linux 9
• Java Version:
  java version "10.0.1" 2018-04-17
  Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
  Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)
• BentoBox version: 1.0 release
• Addons installed?
  [06:31:52 INFO]: Bentobox version: 1.0
  [06:31:52 INFO]: Loaded Game Worlds:
  [06:31:52 INFO]: skyblock (skyblock)
  [06:31:52 INFO]: Loaded Add-Ons
  [06:31:52 INFO]: BSkyBlock 1.0
  [06:31:52 INFO]: WelcomeWarps 0.1.0-SNAPSHOT
  [06:31:52 INFO]: Challenges 0.3.0-SNAPSHOT
  [06:31:52 INFO]: Level 0.2.0
  [06:31:52 INFO]: Biomes 0.3.0-SNAPSHOT
• Other plugins?
  [06:32:17 INFO]: Plugins (50): AnimatedNames*, AreaShop, ArmorStandTools, BentoBox, BungeeTabListPlus*, ChatControl, ChestCommands, ChestShop*, ClearLag*, CrateReloaded, EditableSign, Essentials, EssentialsChat*, EssentialsSpawn*, ExecuteEverywhere*, FastAsyncWorldEdit*, FeatherBoard*, HeadDatabase*, HolographicDisplays, HungerKeeperPlus*, IPWhitelist*, IslandBorder, LeaderHeads*, LibsDisguises, LuckPerms, MobManager*, Multiverse-Core*, MVdWPlaceholderAPI*, NickRemover*, NoSleepCMDs*, OpenInv*, PlayerHeads, PlayerPoints*, ProtocolLib*, PvPManager*, PvPManagerBossBar*, RedstoneClockDetector*, RPGHealthIndicator*, SilkSpawners, SkinsRestorer, Spartan, TradeMe, TrophyHeads*, UnbreakingAnvils*, Vault*, VoidGenerator, WorldEdit, WorldGuard, WorldGuardPistonFix*, Yamler*

Additional context
Can try and help you debugging on live server now...

[tastybento]

Sorry you are having a problem. There’s not much info to go on here right now so all I can do is ask questions to gather more info:

1. Firstly, how do you know the islands are deleted? What’s the user experience? What happens when they log in and do /island? What do you see when you do /bsb info player-name? What do you see if you are on the island and do just /island info?
2. When you restart the server, how are you doing that exactly?
3. Are there any errors in the console logs either when shutting down or restart? If so, share them.

The most likely reason for something like this is that the database is not being saved correctly. I would expect some kind of errors to be showing up in the console logs.

Let me know what you see.

By the way, I doubt it’s an issue but I have not tested on Java 10.

[tastybento]

Another thought: you say the data in the database is not valid. What data is there?

[baticag]

1. Players spawn on their island when they login in, they type /is level, get message that they don't have island. When they do /island new island is created somewhere else. When I do /bsbadmin info on the location on previous island I get message that on that location island doesn't exist. When I want to /bsbadmin register that island, it says that there is no island I am standing on.
2. Restart is done by simple /stop and then starting the server again. It is done every 24hours, this bug is happening second day.
3. No errors on shutdown, no errors on start.

Firstly data was being saved in YML files like default, but after first bug appear we moved to MYSQL, there is no row containing player uuid as owner or island center from their old island location.

I want to note again that this is not happening to all players, minority of them survived with their islands saved correctly.

[Poslovitch]

Switching from YAML to MySQL might have caused even more data loss, as BentoBox does not support switching databases at the moment.
Do you have some backups? This way, if things tend to get worse, you could always do some rollbacks.

We're very sorry you're experiencing this.

[Poslovitch]

@tastybento Maybe we could try providing a build containing some more debug?

[tastybento]

Thanks.

This is going to be hard to debug because by the time the bug is revealed, data has already been lost.

I still think that the issue is that the island data or player data(they are separate tables) are not being saved correctly. I know it’s a pain, but can you have a look through past zipped server logs and see if there are any errors from Bentobox?

I’ll have to set up a test server to continuously restart a server and see if I can replicate.

If you spot anything else or any pattern or type of player who is losing an island please let me know.

[tastybento]

@Poslovitch Yes, I think that will have to be done. It’s late here so I won’t be able to work on it for at least a day.

[baticag]

Okay, you got it wrong. Let me explain.

First bug happened on YAML, since we tough that some issues were on our side, we removed all the data and started everything from beginning since players have played only for 1 day and that happened.

After we deleted everything and started on new world, with same configurations, but now on different database type MYSQL, we have now experienced same thing happening again. And now that the problem, since players have played again for a whole day and now they can't access their island. We have started debugging this more after the second time bug happened and since we couldn't figured it out I am here now :/

Our SkyBlock server is currently whitelisted because of this and will stay until we find where the problem is.

Data isn't correctly saved only in islands database, players and names are done correctly.

Will update you if we find anything.

[tastybento]

How many islands do you think you have and how many have been lost so far?

[baticag]

Here's pic from MYSQL:
https://i.imgur.com/HJIRCeG.png

I don't think that 481 players got onto server and didn't create island. So we are talking about ~300 islands...

[Poslovitch]

What about duplicate UUIDs tasty?

[BONNe]

So,
Today I manage to get an exception in BentoBox 1.1-SNAPSHOT
Caused by: java.lang.NullPointerException at world.bentobox.bentobox.database.yaml.YamlDatabaseHandler.saveObject(YamlDatabaseHandler.java:306) ~[?:?]

at world.bentobox.bentobox.database.Database.saveObject(Database.java:85) ~[?:?]
at world.bentobox.level.Level.uncachePlayer(Level.java:195) ~[?:?]
at world.bentobox.level.listeners.JoinLeaveListener.onPlayerQuit(JoinLeaveListener.java:35) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_111]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_111]
at org.bukkit.plugin.java.JavaPluginLoader$1.execute(JavaPluginLoader.java:305) ~[spigot.jar:git-Spigot-518206a-a5b9c7b]
... 12 more

As it could be the same issue, I assume that it could help to solve this.

[14:54:38 INFO]: Bentobox version: 1.1-SNAPSHOT
[14:54:38 INFO]: Loaded Game Worlds:
[14:54:38 INFO]: AcidIsland_world (AcidIsland)
[14:54:38 INFO]: BSkyBlock_world (BSkyBlock)
[14:54:38 INFO]: Loaded Add-Ons
[14:54:38 INFO]: BSkyBlock 1.1-SNAPSHOT
[14:54:38 INFO]: AcidIsland 1.1-SNAPSHOT
[14:54:38 INFO]: WelcomeWarps 0.1.0-SNAPSHOT
[14:54:38 INFO]: Level 0.3.0-SNAPSHOT
[14:54:38 INFO]: Biomes 0.4.0-SNAPSHOT

[BONNe]

And on even more tries on doing just ./is restart and changing island settings...

[15:12:47 INFO]: BONNe1704 lost connection: Disconnected
[15:12:47 ERROR]: [BentoBox] Could not save object to database! Error: Method not found: isInitialDeathCount
[15:12:47 INFO]: BONNe1704 left the game

[tastybento]

@baticag Thanks. In that error report, the Level add-on is trying to save a null object to the database, and tracing that back implies that it was asked to get a level for a null island owner (null UUID). That could have been caused by a player object having a null UUID (e.g., an NPC of some kind) or that the island was unowned when the request for level occurred. I'm not sure what happened there, but either way, defensive code is required. I've done the following:

1. Added in code to prevent Level from bothering with null owners. Also, if it does for some reason, it'll log it and maybe we can find out why that is happening.
2. The BentoBox database should never save a null object, so I put in code to prevent and log that if an attempt is made.

This is not a root-cause fix but it might be a reason why the database saving is not occurring completely on shutdown.

[tastybento]

@Poslovitch Duplicate UUID's - in theory yes. There could be a duplicate UUID for the island object. It should have an extremely low chance of occurring, but I could put a check in there to see.

[tastybento]

@BONNe Sorry, I don't understand that. Was there an isInitialDeathCount() method in the code at some point? Maybe it's a mismatched version?

[tastybento]

@baticag I'd say that right now, it'd be useful if you could run this latest version from the develop branch and see if that reveals any errors in the log. That will help provide more information. Thanks.

[baticag]

We put the newest dev version on server and will check what happens now and update you.

[tastybento]

Also, you should use the latest Level build too as that will preemptively try to save null players. If you don't have it, it shouldn't be a major issue, but would be best for narrowing down where this null is coming from.

[baticag]

Okay I think we found something similar for players that lost their island.

• Player joins some other team
• Player leaves that team
• Player creates his own island
• Player losses his island after restart

Can you check if this is possible?

EDIT: Just tested this, these are the steps to reproduce this problem

[tastybento]

Okay, I'll test it now.

[tastybento]

Hmm, I can't replicate this right now with my setup. Can you duplicate this on a test server? e.g., just Spigot and BentoBox? If not, then I might need a copy of your server to test. Are you using YAML database?

[baticag]

Sorry for late reply, but need to do some tests in order to be sure what I write to you. Maybe it is not really like this up. But I caught this bug in console (https://pastebin.com/gyZMLVmd) and reproduced it and lost of island by doing this:

• Player B invites Player A to island
• Player A joins Player B island and does some stuff on it
• Player A command: /island team leave
• Player A command: /is
• Player A gets teleported to his island
• Player A command: /is reset
• Player A gets new island and can build on it
• Player A command: /is reset (bug in console gets shown)
• Player A gets new island and can build on it
• Server gets restarted by /stop (/restart)
• Player A joins the server, gets spawned on the same place on "his island" but no info about it in database, so he lost his island

This is tested couple of times and we are sure that this is a problem now. Using MySQL database.

[tastybento]

I think I see the issue. I’m at CES this week but I’ll try and fix this on the weekend.

[tastybento]

@baticag This should be fixed with the latest develop branch release. Are you able to test it and see?

[tastybento]

Closing. If this is still an issue, reopen.

[Rixafy]

Maybe the same issue, some random players don't have their island after restart, I have latest builds from yesterday (v1.1, BSkyBlock and BentoBox), it's happening since 1.0 version (I had previously 0.18.0 and I didn't have a problem).

[baticag]

I think you fixed problem we reported, but today 1 player got in touch with us that his island wasn't saved. One is much much better than the number we had, but still...

[Rixafy]

Yes, and it's not like their island was removed before, these players were on my servers after fix and we gave their islands back to them, but today it happens again, players are complaining that they join at island, but island is no longer their so we need to type /bsb register nick, it happens every day, and my server core is reporting that island.getOwner() is null (I have scoreboard with island details), @tastybento what can I do?

[tastybento]

@Rixafy can you file a new bug ticket and fill out the problem and way to replicate as best you can? I don’t have a full understanding from this thread what is happeneing.

[tastybento]

Create a new issue if there is a problem and reference this one (just use the hash and number 447.