Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Having over 2000+ items stored in storage systems causes chunk "corruption" #41

Open
negativize opened this issue Mar 12, 2018 · 36 comments

Comments

@negativize
Copy link

This issue seems to be consistent and has been reported numerous times on the thread on the Terraria forums. It also seems to be multiplayer-only, as the issued isn't present in singleplayer. The corruption appears to happen after exiting a world with 2000+ items stored.

Problem: when over 2000 or more items total are stored in a system and a server is restarted, everything in the spawn chunk will be erased upon restart (might be chunks around storage that are affected, not sure)

I don't believe this affects wired items (as our wired doors were not erased when the issue happened), but everything else is deleted. I believe this also happens when multiple system have a total storage of 2000+ items. This does not happen when there are over 2000 slots available.

Strangely, NPCs inside the deleted chunk fall are not affected and instead teleport to the position of where there homes were after falling a certain distance.

This corruption doesn't actually "delete" items: in singleplayer, any affected chunks return to their former state and any storage units work as intended. The only way to fix the issue (with my knowledge) is to start the corrupted world in single-player, remove items until less than 2000 are present, and restart the affected world. This fixes the issue and everything is returned to its prior state.

Image taken from forum in singleplayer:
image

Image taken from forum in multiplayer:
image

Some users are reporting this happens at different limits. (one user reported 8000 - 8500 items being the limit) I would only consider this issue critical in multiplayer, as it basically ruins the gimmick of storage by limiting the server to less than 2000 items. Works fine in singleplayer. Either way, it needs a fix.

@ukilop
Copy link

ukilop commented May 27, 2018

same issue here altho problem started way before 2k items as we have a mod that adds random stats to items as well,
temp work around is have the storage containers in different chunks.
in my multiplayer world we moved a bit from spawn and put 4 sets of 5 (hellstone) containers with each set being in a different chunk, fyi the problem doesn't exist for single player so simple take the multiplayer world and open it in single player and move some storage around, then launch host&play to test if it worked

rreminy added a commit to rreminy/MagicStorage that referenced this issue Jul 21, 2018
Compresses Magic Storage data before transmitting (and decompresses before receiving) them during multiplayer.

This is not a sure nor a correct fix  for blushiemagic#47 and blushiemagic#41 but delays them from occurring (if you place an extra massive amount of items into the storage it'll still happen again) while also improving network performance, true reason I'm posting. Data is compressed on a 10:2 ratio average (allowing you to store up to about 5 times the amount of items before the above bug recur). Example of some storage units, all upgraded to Luminite, during a heavily modded run (in bytes):

Magic Storage Data Compression Stats: 12014 => 2806
Magic Storage Data Compression Stats: 19412 => 3890
Magic Storage Data Compression Stats: 18932 => 3837
Magic Storage Data Compression Stats: 18476 => 3470
Magic Storage Data Compression Stats: 18896 => 3561

I left the compression stats code in but disabled.
rreminy added a commit to rreminy/MagicStorage that referenced this issue Aug 3, 2018
Compresses Magic Storage data during multiplayer with an average compression ratio of 10:2, improving network performance and delaying blushiemagic#47 and blushiemagic#41 from occurring. Example of some storage units, all upgraded to Luminite, during a heavily modded run, are:

Magic Storage Data Compression Stats: 12014 => 2806
Magic Storage Data Compression Stats: 19412 => 3890
Magic Storage Data Compression Stats: 18932 => 3837
Magic Storage Data Compression Stats: 18476 => 3470
Magic Storage Data Compression Stats: 18896 => 3561

I left the compression stats code in but disabled. (See lines 413 - 423)
blushiemagic pushed a commit that referenced this issue Aug 3, 2018
Compresses Magic Storage data during multiplayer with an average compression ratio of 10:2, improving network performance and delaying #47 and #41 from occurring. Example of some storage units, all upgraded to Luminite, during a heavily modded run, are:

Magic Storage Data Compression Stats: 12014 => 2806
Magic Storage Data Compression Stats: 19412 => 3890
Magic Storage Data Compression Stats: 18932 => 3837
Magic Storage Data Compression Stats: 18476 => 3470
Magic Storage Data Compression Stats: 18896 => 3561

I left the compression stats code in but disabled. (See lines 413 - 423)
@blushiemagic
Copy link
Owner

bcd3499 should delay this from happening. However I'll still leave this issue open since it's only a temporary fix, and it's planned for the next tModLoader update to fix this permanently.

@Tenrys
Copy link

Tenrys commented Oct 11, 2018

This still happens today, having mods like "Even more modifiers" and "Spirit Mod" (with glyphs) makes this happen even quicker. Not sure what the course of action is, I hardly doubt "take the items out in single player and split them through multiple chunks" is a viable solution at all.

@Tenrys
Copy link

Tenrys commented Oct 15, 2018

I tried to host another world with less mods and a tiny amount of MagicStorage containers, and the same deal happened. It only seems to happen on my Linux Dedicated Server, when my friends try to host it on their PC it just works without an issue.

@Psyda
Copy link

Psyda commented Oct 25, 2018

Linux VPS with 2 MS systems totaling 450-600 items
Linux VPS with 2 MS systems totaling 450-600 items

Windows server with same mods and 12,000+ items.
Windows server with same mods and 12,000+ items.

I think the issue is deeper than expected.

@Fadenfire
Copy link

Fadenfire commented Dec 27, 2018

I have been able to consistently reproduce this bug on my mac. It doesn't matter how many items are in the storage units, as long as there is at least one storage unit in a chunk then the whole chunk refuses to load in multiplayer.

After some testing I think I have figured out why this happens, at least on mac (possibly linux too). It is because System.IO.Compression (which is used by the storage units to compress data before it's sent over the network) requires a library called MonoPosixHelper to be loaded which isn't included in Terraria by default. If you copy libMonoPosixHelper.dylib (on linux it's probably called libMonoPosixHelper.so) to the osx (on linux I'm assuming it's called linux I think it's called lib or lib64, I don't know) directory then the world loads fine.

Edit: tModLoader 0.11 and newer now includes libMonoPosixHelper by default so that should fix this for most people. If upgrading to that version doesn't fix it for you then your issue is probably unrelated to this one.

@MarkusG
Copy link

MarkusG commented Mar 24, 2019

Where exactly is libMonoPosixHelper.so found? Does it come with tModLoader or do I have to download it myself?

@Fadenfire
Copy link

You need to extract them from a mono install. Here's a zip with a mac and linux lib. I'm not sure if the linux lib is 64 bit or 32 bit, I just installed mono in a linux virtual machine and grabbed it from the libs directory.

@MarkusG
Copy link

MarkusG commented Mar 25, 2019

It looks like those libraries are 64-bit. I tried both those and copying the library from my own mono installation, but to no avail. The chunk rendered, but the storage system itself still doesn't work. I can deposit items, but it displays "0/0 items" and I can't retrieve them. However, I can retrieve items in singleplayer, which is very confusing.

@drax32
Copy link

drax32 commented May 17, 2019

I have this problem as well on linux as a client, how do I fix this?

@ultrako
Copy link

ultrako commented Jun 1, 2019

Yep, those files are 64 bit. I put the library file in Terraria's lib64 folder, and can successfully connect to a server with magic storage. I'm on a linux client, and can access all of the items in the Storage Heart.

@halcyonesselion
Copy link

halcyonesselion commented Jun 19, 2019

Would anyone be able to provide a download for a 32-bit libMonoPosixHelper.dylib? I'm using tModLoader to host these mods and am having the same issue, but the lib in Silly511's zip folder didn't solve the problem. I suspect it's because the current stable version of tModLoader is 32-bit. All other issues are the same.

Edit 07/26/19: I tried loading my world again yesterday and it's... working? It can host a server just fine now. I suspect the most recent tModLoader update did something that let things work, or maybe there was a new update for Magic Storage? Either way, the lib is in the osx folder and the server is functioning. No idea what specifically changed.

@cobaltw
Copy link

cobaltw commented Jul 15, 2019

Hello, I would like to add to this problem by saying I am having this same problem on Windows. I have also had this problem after placing 3 empty storage units with no heart side by side. After about 5-10 minutes, my game crashed, I opened it back up (on multiplayer) and the chunk was gone.

@Steffo99
Copy link

I'd suggest pinning this issue, since it seems to be a common problem.

@Steffo99
Copy link

@Silly511 Thanks, your fix worked on Ubuntu 18.04LTS with tModLoader v0.11.3!

@syy1125
Copy link

syy1125 commented Aug 4, 2019

Encountered the world hole issue, @Silly511 thanks for your fix - it worked for me.
I'm going to look through the open issues and point others having similar problems to this thread.

@syy1125
Copy link

syy1125 commented Aug 4, 2019

Also, #74 (comment) mentioned that v0.4.3.1 was working for Linux but v0.4.3.2 and later were not. I found commit bcd3499 in the history of v0.4.3.2, so I'd say #41 (comment) is on point.

@ivakam
Copy link

ivakam commented Aug 5, 2019

I'm getting wildly varying results with the fix mentioned earlier in the thread. At first it seemed to be working, but now I'm either getting stuck on "Receiving tile data: Complete!" or the client connects, but then freezes and crashes. The steps I've taken is copying the .so file to both lib and lib64.

For reference, I'm on Fedora 30 with modloader 0.11.3 and Magic Storage 0.4.3.4.

@warren-fisher
Copy link

warren-fisher commented Dec 22, 2019

I am using Debian with tmodloader 0.11.5 and it is not working. Similar problem to @ivakam with the .so fix, I get stuck trying to connect at 33% or 40%. I will look into MagicStorage and see if I can find a fix.

Using MagicStorage v0.4.3.1 I got it working. Even loaded up Cheat Sheet and put 100,000+ items in some Storage Hearts. Worked even after relogging/restarting server. I tried the solution in #41 and it got stuck loading into the world. I will stick to v0.4.3.1 and hope that this resolves any issues

@Reutertu3
Copy link

I can attest that placing libMonoPosixHelper.so into the lib64 library solved the issue for me (Debian 10). Despite using the latest tmodloader (0.11.6.2) that particular library wasn't present which evidently caused the issue.

@MCwindTIM
Copy link

I got this issue too, Is anyone knows how to fix this on Windows 10 1903?
Magic Storage v0.4.3.5 with tModLoader v0.11.7.1
image

@tzhouhc
Copy link

tzhouhc commented Aug 26, 2020

tModloaderServer 0.11.7.5, magic storage 0.4.3.5, same issue as described above, system is ubuntu 18.04, can confirm that libMonoPosixHelper.so was not present in tModLoaderServer's linux release and the issue was fixed after manually including the lib file.

@TNTftw21
Copy link

Just had this issue rear its ugly head again. tModLoader v0.11.8.4, Magic Storage 0.4.3.5. I can't seem to find where tModLoader logs full errors (instead I only get the "Error on message Terraria.MessageBuffer" errors), but I believe this is the same problem. Strangely, there were only a handful of items - somewhere in the neighborhood of 250 items including the chests I hadn't merged into the system yet. Relogging without restarting the server (I run the server in a terminal to make finding and testing things like this easier) caused a chunk of the world to disappear for me, but my two friends running Windows were just fine. Restarting the server and relogging caused all of the blocks to no longer follow visual attachment rules, and getting close to where the chunk of the world went missing caused the client to crash. Unlike previous times I've encountered this problem, tML 0.11.8.4 does have the appropriate 32-bit and 64-bit versions of libMonoPosixHelper.so (I also made sure it was using the provided versions and not my native ones using pmap -p). Swapping in the file provided by Fadenfire did not fix the bug. Everything works fine in singleplayer.

@hollowharkov
Copy link

Same issue on Ubuntu 20.04, tmodloader 0.11.8.4, Magic Storage 0.4.3.5. TIles aren't connecting properly and chunks are missing around the storage system. Items are also missing from the system until I restart my game. I also tried default libMonoPosixHelper.so and the provided version to no avail. Works fine in singleplayer for me as well.

@TNTftw21
Copy link

TNTftw21 commented Jul 11, 2021

libMonoPosixHelper.zip
This is the version of the library I'm using. It's provided by mono-runtime-common version 6.12.0.122-0xamarin1+ubuntu2004b1. I literally just copied it from /lib to the tML lib64 directory. It got me through the rest of that playthrough without issue.

@hollowharkov
Copy link

I can confirm that the fix from TNTftw21 works for me as well, hopefully tmodloader uses that lib in a future update.

@mazunki
Copy link

mazunki commented Dec 31, 2021

libMonoPosixHelper.zip This is the version of the library I'm using. It's provided by mono-runtime-common version 6.12.0.122-0xamarin1+ubuntu2004b1. I literally just copied it from /lib to the tML lib64 directory. It got me through the rest of that playthrough without issue.

Both of the LD_PATHS under $STEAM_PATH/tModLoader included the libMonoPosixHelper.so for me, the x86 version being 263K and the 64-bit being 261K in size. Should I override the 64-bit with the older version? I didn't even have mono installed until recently. How was my game even working?

My issue is not about chunk corruption, per se, but rather inability to find certain items in the storage. Does anyone know if the cause the same?

@flarn2006
Copy link

Does this happen in Crimson worlds too?

@mazunki
Copy link

mazunki commented Jan 8, 2022

Does this happen in Crimson worlds too?

My worldgen has Fargowilta's Both Evils installed, but if I recall correctly the world "is" a corrupted one. I would be surprised if that's related at all, @flarn2006 .

EDIT: r/whoosh

dougbenham pushed a commit to dougbenham/Terraria-MagicStorage that referenced this issue May 30, 2022
@aMytho
Copy link

aMytho commented Oct 25, 2022

I'm also having the Error on message Terraria.MessageBuffer. I am using terraria 1.3.x with the 64bit version of tmodloader. (Linux server). The complete version of mono is installed. I've ran the exe and the script but both methods have the same problem.

I check my linux lib folder and I see the libMonoPosixHelper.so file. However, I don't seem to have a lib64 folder. I created one in the root directory and added the file to it but that doesn't seem to change anything. I've only tried it with a few items and it seems to work but the error still appears in the logs.

@SpoopyTim
Copy link

Both of the LD_PATHS under $STEAM_PATH/tModLoader included the libMonoPosixHelper.so for me, the x86 version being 263K and the 64-bit being 261K in size. Should I override the 64-bit with the older version? I didn't even have mono installed until recently. How was my game even working?

My issue is not about chunk corruption, per se, but rather inability to find certain items in the storage. Does anyone know if the cause the same?

Can confirm the same as mazunki

Not having any chunk issues but intense lag when opening storage and even more lag when opening the crafting interface.

There's also the disappearance of items at random, some items that should have been in there from a long time ago are gone and some items that i just put in disappear too. Both cases of which some are permanent and some come back after a server restart.

~5000 items in storage

Running a docker container on Ubuntu 22.04 running on a high spec machine
Dockerfile
docker-compose.prod.yml

version: "1.0"
services:
  tml:
    # The built docker image
    image: tmodloader-server_tml
    # Allow attach
    stdin_open: true
    tty: true
    ports:
      - 7189:7777
    volumes:
      - "./TModLoader-Server/Terraria:/home/tml/.local/share/Terraria"

@rafapousin
Copy link

rafapousin commented Oct 20, 2023

image
The chunks missing are the grey are that goes around in a square shape, u can see how it goes down passing the hell (completelly removed)

It just happend to me today, kinda worried tho in singleplayer its completelly fine, tried to see if it was some time of incompatibility with any biome generating mod, but the bug ate an insane ammount of chunks, mushroom biome and hell included, i even fall to the void and died.

image
(list of the mods installed along with magic storage)

@ExterminatorX99
Copy link
Contributor

ExterminatorX99 commented Oct 20, 2023

Considering how MUCH is missing I feel this isn't caused by magic storage. Could you send your server log and client log?

@rafapousin
Copy link

rafapousin commented Oct 20, 2023

Considering how MUCH is missing I feel this isn't caused by magic storage. Could you send your server log and client log?

yeah, ive been thinking that too since the storage chunk is not missing and i dont have 2000+ stacks of items stored, maybe i was a bit fast with the post, after restartign the server the chunks reapeared again, same with opening the world in singleplayer (in case the logs found as "client" and "server" in setamapps/commom/tModLoader/tModLoader-Logs are worthless)
server.log
client.log

Im gonna guess with my null knowledge about terraria logs that those that ive adjunted do not containt he error since as i mentioned early, i restarted the server and it worked good, checking the "Old" logs i found this one
server.log that im gonna asume is the log made after the "bug" since its quite larger than the other "old" logs.

@ExterminatorX99
Copy link
Contributor

Yeah, that second log is the bugged one.
I see a bunch of Calamity exceptions in there.
Best report it to them.
Glad to hear a restart fixed the issue.

@rafapousin
Copy link

will do, ty

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests