[New bug] ip2c takes 100 times longer to load

[ArranTuna]

Describe the bug

I noticed that player countries wasn't loading, debugged it for an hour and something inside loadIPGroupsWorker() is stuck in an infinite loop (turns out it's just extremely slow) and it's just too complex for me to try and figure out what's wrong.

Steps to reproduce

Get the newest nightly (23183)
Launch local server with ip2c resource running
Notice it doesn't work and performancebrowser shows it using 39% CPU.

Version

Multi Theft Auto v1.6-release-23183

Additional context

Relevant log output

Security Policy

• I have read and understood the Security Policy and this issue is not security related.

[ArranTuna]

I have narrowed it down to being introduced in 23167 the previous nightly, 23147 doesn't have the bug.

[ArranTuna]

Seems to not be an infinite loop, I got ChatGPT to add a load of debug lines to the function, it took 1 hour to go through each of the 322787 lines in the csv file, what used to take a few seconds.

[ArranTuna]

With 23167 4 lines can be processed per 50ms.
With 23147 314 lines can be processed per 50ms.

[Dutchman101]

I noticed it as well, my server's ip2c only began to fill in countries after many hours of it running. I had updated the server right before starting to see it. I will help to narrow it down as soon i got some time, otherwise, you can debug it more.

At least it's true, there is something wrong.

[ArranTuna]

I asked ChatGPT to narrow it down and it's these 4 lines in the newer versions that are the problem:

local buffer = ByteBuffer:new()
buffer:writeInt24(rstart)
buffer:writeInt24(rend)
buffer:writeBytes(rcountry, 2)

Bugged version:

[2025-05-08 10:51:48] INFO: [Line 5000] Total times: read=7ms, split=13ms, unrel=3ms, group=1ms, buffer=118ms, assign=0ms
[2025-05-08 10:51:49] INFO: [Line 10000] Total times: read=16ms, split=20ms, unrel=4ms, group=2ms, buffer=400ms, assign=3ms
[2025-05-08 10:51:51] INFO: [Line 15000] Total times: read=18ms, split=37ms, unrel=9ms, group=5ms, buffer=1266ms, assign=3ms
[2025-05-08 10:51:55] INFO: [Line 20000] Total times: read=23ms, split=44ms, unrel=10ms, group=6ms, buffer=2673ms, assign=3ms

Good version:

[2025-05-08 10:55:53] INFO: [Line 5000] Total times: read=4ms, split=6ms, unrel=5ms, group=1ms, buffer=12ms, assign=0ms
[2025-05-08 10:55:53] INFO: [Line 10000] Total times: read=11ms, split=13ms, unrel=8ms, group=2ms, buffer=21ms, assign=1ms
[2025-05-08 10:55:53] INFO: [Line 15000] Total times: read=14ms, split=22ms, unrel=10ms, group=3ms, buffer=37ms, assign=1ms
[2025-05-08 10:55:53] INFO: [Line 20000] Total times: read=16ms, split=27ms, unrel=16ms, group=4ms, buffer=47ms, assign=3ms

I then asked it to show the performance of each of those lines:

Bugged version:

[2025-05-08 11:04:31] INFO: [Line 0] Buffer cumulative: new=0ms, writeStart=0ms, writeEnd=0ms, writeCountry=0ms, totalBufferBlock=0ms
[2025-05-08 11:04:32] INFO: [Line 5000] Total times: read=4ms, split=11ms, unrel=6ms, group=2ms, buffer=0ms, assign=0ms
[2025-05-08 11:04:32] INFO: [Line 5000] Buffer cumulative: new=5ms, writeStart=51ms, writeEnd=36ms, writeCountry=47ms, totalBufferBlock=143ms
[2025-05-08 11:04:32] INFO: [Line 10000] Total times: read=7ms, split=22ms, unrel=9ms, group=4ms, buffer=0ms, assign=1ms
[2025-05-08 11:04:32] INFO: [Line 10000] Buffer cumulative: new=12ms, writeStart=145ms, writeEnd=123ms, writeCountry=123ms, totalBufferBlock=410ms
[2025-05-08 11:04:35] INFO: [Line 15000] Total times: read=15ms, split=32ms, unrel=16ms, group=5ms, buffer=0ms, assign=1ms
[2025-05-08 11:04:35] INFO: [Line 15000] Buffer cumulative: new=17ms, writeStart=476ms, writeEnd=450ms, writeCountry=424ms, totalBufferBlock=1377ms
[2025-05-08 11:04:39] INFO: [Line 20000] Total times: read=22ms, split=42ms, unrel=16ms, group=9ms, buffer=0ms, assign=2ms
[2025-05-08 11:04:39] INFO: [Line 20000] Buffer cumulative: new=22ms, writeStart=1031ms, writeEnd=1015ms, writeCountry=947ms, totalBufferBlock=3027ms

Good version:

[2025-05-08 11:03:09] INFO: [Line 0] Buffer cumulative: new=0ms, writeStart=0ms, writeEnd=0ms, writeCountry=0ms, totalBufferBlock=0ms
[2025-05-08 11:03:09] INFO: [Line 5000] Total times: read=2ms, split=5ms, unrel=6ms, group=1ms, buffer=0ms, assign=1ms
[2025-05-08 11:03:09] INFO: [Line 5000] Buffer cumulative: new=2ms, writeStart=3ms, writeEnd=6ms, writeCountry=1ms, totalBufferBlock=14ms
[2025-05-08 11:03:09] INFO: [Line 10000] Total times: read=8ms, split=14ms, unrel=8ms, group=3ms, buffer=0ms, assign=5ms
[2025-05-08 11:03:09] INFO: [Line 10000] Buffer cumulative: new=5ms, writeStart=11ms, writeEnd=11ms, writeCountry=3ms, totalBufferBlock=37ms
[2025-05-08 11:03:09] INFO: [Line 15000] Total times: read=17ms, split=33ms, unrel=14ms, group=4ms, buffer=0ms, assign=5ms
[2025-05-08 11:03:09] INFO: [Line 15000] Buffer cumulative: new=9ms, writeStart=15ms, writeEnd=15ms, writeCountry=3ms, totalBufferBlock=54ms
[2025-05-08 11:03:09] INFO: [Line 20000] Total times: read=25ms, split=45ms, unrel=21ms, group=8ms, buffer=0ms, assign=7ms

🧨 Conclusion:

The overwhelming majority of the performance loss in the bugged version is caused by:

writeInt24() being ~500× slower than normal

writeBytes() also becoming ~300× slower

✅ 1. Profile ByteBuffer implementation

Check if recent server changes introduced:

Per-byte writing with internal resizing

Memory allocation or resizing on every write

Debug logging or checks inside these functions

[ArranTuna]

ChatGPT has written a new version of ByteBuffer that is hundreds of times more efficient (from taking over an hour to 5 seconds):

Each time you do self.data = self.data .. something, you're creating a new string, because Lua strings are immutable. This means:

On line 1: data = "abc"

On line 2: data = "abc" .. "d" → a new "abcd" is allocated

On line 10,000: You're copying tens of thousands of bytes every time

This creates exponential memory copying, which explains the rapidly growing time in your logs (buffer block time at 20k lines: 3027 ms)._

	new = function(self, indata)
		local newItem = {
			chunks = {},
			data = indata or nil,
			readPos = 1
		}

		-- If indata is passed, treat it as pre-finalized data
		if indata then
			newItem.chunks = nil
		end

		return setmetatable(newItem, { __index = ByteBuffer })
	end,

	Copy = function(self)
		if self.chunks then
			-- Not finalized yet
			return ByteBuffer:new(self:getData())
		else
			-- Already finalized
			return ByteBuffer:new(self.data)
		end
	end,

	-- Write operations (efficient using chunk table)
	writeInt24 = function(self, value)
		local b0 = math.floor(value / 1) % 256
		local b1 = math.floor(value / 256) % 256
		local b2 = math.floor(value / 65536) % 256
		table.insert(self.chunks, string.char(b0, b1, b2))
	end,

	writeBytes = function(self, chars, count)
		table.insert(self.chunks, string.sub(chars, 1, count))
	end,

	-- Finalize chunks into string (if not done yet)
	getData = function(self)
		if not self.data and self.chunks then
			self.data = table.concat(self.chunks)
			self.chunks = nil
		end
		return self.data
	end,

	-- Read operations (no change, uses self.data)
	readInt24 = function(self)
		local b0, b1, b2 = string.byte(self:getData(), self.readPos, self.readPos + 2)
		self.readPos = self.readPos + 3
		return b0 + b1 * 256 + b2 * 65536
	end,

	readBytes = function(self, count)
		local result = string.sub(self:getData(), self.readPos, self.readPos + count - 1)
		self.readPos = self.readPos + count
		return result
	end,
}

However this doesn't explain why the new version of MTA has become so inefficient, when the less efficient ByteBuffer was never an issue before.

Edit: Turns out that this optimization doesn't seem to help because I hadn't changed:
aCountries[group][count] = buffer.data
into:
aCountries[group][count] = buffer:getData()
Which now seems to also be taking forever too.

[botder]

This issue is caused by these lines:

https://github.com/multitheftauto/mtasa-resources/blob/7e86616099dc07e94bf05ad60010fc6edb1ab471/%5Badmin%5D/ip2c/server.lua#L97-L99

I don't really understand why (coroutine/lambda capture fail?), but if you track the offset inside the worker function, the problem goes away.

Edit: ip2c can be greatly optimized, it's possible to reduce the loading to ~2 seconds.

[ArranTuna]

You disabled unrelRange calls and it fixed it for you? That's weird cause my tests said (and I did try them disabled) the byte calls were to blame. Weird thing is that I tried the optimized version and it worked, but then tried it on another server and it didn't work. Isn't it easiest to just revert any commits in 23167 that might be to blame and try again?

[botder]

You disabled unrelRange calls and it fixed it for you? That's weird cause my tests said (and I did try them disabled) the byte calls were to blame. Weird thing is that I tried the optimized version and it worked, but then tried it on another server and it didn't work. Isn't it easiest to just revert any commits in 23167 that might be to blame and try again?

Yesn't. I had a coding mistake, and when the unrelRange code was inside the line reading block, the whole thing become fast again.
Now, on the other hand, we must optimize that script because it's very slow. The ByteBuffer thing is completely unnecessary.

[samr46]

This issue is caused by new string check introduced in 4929a83de0413a304e5b8e5a34fcf68542c932b6 (@tederis)

mtasa-blue/vendor/lua/src/lstring.c

Lines 110 to 111 in 794ae47

	if (!str || str[0] == '\0' || l == 0)
	return 0;

It looks like this check slowed down strings for some reason on both server and client side.

Removing ByteBuffer from ip2c is still a good idea to optimize this resource. It's unnecessarily slow right now.

[tederis]

This issue is caused by new string check introduced in 4929a83de0413a304e5b8e5a34fcf68542c932b6 (@tederis)

mtasa-blue/vendor/lua/src/lstring.c

Lines 110 to 111 in 794ae47

if (!str || str[0] == '\0' || l == 0)
return 0;
It looks like this check slowed down strings for some reason on both server and client side.

Removing ByteBuffer from ip2c is still a good idea to optimize this resource. It's unnecessarily slow right now.

I find it hard to believe that this line of code can indeed slow down whatever. How was this conclusion reached?

[tederis]

As it turns out some of Lua string that ip2c uses have a null character in the beginning. This could potentially lead to the performance degradation. PR #4218 should fix it.