newString zeroes memory contrary to its documentation, significantly slowing it down

[arnetheduck]

Description

newString is supposed to return an uninitialized string, and thus presumably be fast:

Nim/lib/system.nim

Line 976 in d677ed3

## Returns a new string of length `len` but with uninitialized

However, the actual implementations zero the full string leading to significant overhead making the function unfit for its purpose:

https://github.com/nim-lang/Nim/blob/devel/lib/system/strs_v2.nim#L113

Nim Version

1.6, devel, both refc and orc

Current Output

No response

Expected Output

No response

Possible Solution

No response

Additional Information

No response

[ringabout]

Isn't it potentially a potential runtime breaking change supposing someone relies on the quirky behaviour? Yeah, let's find out whether it breaks something.

[ringabout]

It breaks jangko/nimPNG#85

[mratsim]

But aren't all Nim data structures supposed to be zero-initialized?

I think the correct least-surprising way forward is to introduce newStringUninit / newStringUninitialized.

[Araq]

Hmm good point.

[arnetheduck]

Came to a similar conclusion while thinking about it over the weekend (newStringUninitialized).

but to complete this section of nim, #19727 really needs addressing as well - the current situation is silly, ie can't newSeqUninitialized a seq[Complex64] nor array[byte, 32] and so on making it impossible to write any sane deserialization code.

[ringabout]

As you wish, I will add a new function and remove outdated documentation.

[ringabout]

ie can't newSeqUninitialized a seq[Complex64] nor array[byte, 32]

Hi, @arnetheduck, supportCopyMem excludes types containing managed memory or having destructors. Is it enough?

[arnetheduck]

Hi, @arnetheduck, supportCopyMem excludes types containing managed memory or having destructors. Is it enough?

It's a good enough to start with - I can imagine a few cases where this is too strict as well but it's easy to further relax in the future - ie consider a ref - strictly, if we had the expressivity saying that the instance is not a valid instance, we could allow uninit for ref as well - the situation is similar to what happens with an instance after you moved from it (it's "destroyed" and therefore could be reused without zero-init in theory) but expressing this is tricky - I haven't followed arc closely enough to know for sure where this stands.