Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: Release one copy of underlying data if the comparing result of two strings or two interfaces is true. #17526

Closed
yaxinlx opened this issue Oct 20, 2016 · 13 comments

Comments

Projects
None yet
7 participants
@yaxinlx
Copy link

commented Oct 20, 2016

What version of Go are you using (go version)?

go version go1.7.1 linux/amd64

What did you do?

https://play.golang.org/p/Lu9FZ9iqn6
https://play.golang.org/p/-wE2oUGce1

What did you expect to see?

much memory should be released after the comparing.

What did you see instead?

no memory released.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Oct 20, 2016

This isn't a safe optimization for the compiler to do in the presence of multiple goroutines. It would also generate much larger code. If you want this behavior, you can write it explicitly yourself.

@bradfitz bradfitz closed this Oct 20, 2016

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 20, 2016

@bradfitz

you can write it explicitly yourself.

this is not always feasible.

This isn't a safe optimization for the compiler to do in the presence of multiple goroutines

surely, it is not just to simply release one copy. The compiler should consider the safety.

It would also generate much larger code

Maybe yes, maybe not. I highly doubt the conclusion.

@Merovius

This comment has been minimized.

Copy link

commented Oct 20, 2016

surely, it is not just to simply release one copy.

The potentially unsafe thing is, that you need to modify a string, that could get modified concurrently.

Say, e.g. a = { 0x1234, 42 }, b = { 0x2345, 42 }, c = { 0x3456, 1000 } and 0x1234 and 0x2345 contain the same data. Let's say you have a goroutine A that does b, c = c, b in a loop. Another goroutine B does a == b in a loop. At some point, this will evaluate to true and the compiler will emit code to do b = a, to release 0x2345. But, it is racing with the other goroutine, so you might end up with b == { 0x1234, 1000 } because A assigned the pointer over it and B assigned the length. Now you can use that to do an out-of-bounds writeread.

Of course this isn't complete, but it illustrates the issue. There would need to be some locking around this optimization, which just makes it bonkers, because you pay synchronization cost on every string assign and comparison.

@alandonovan

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2016

Even if this could be implemented cheaply, compactly, and concurrency-safely, it's still not a sensible optimization. Consider:

var x = "."

func f(verylongstring string) {
    y = verylongstring[:1]
    if x == y { ... }
}

If the compiler compiled the if-statement as:

if x == y { x = y; ... }

then you would have stored a reference to verylongstring in a global variable, potentially pinning a large byte array in memory forever.

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 20, 2016

@alandonovan
for this case, just keep the short one (less memory alloced one).
(ok, I don't know if the current go runtime can judge which one is short now, but I think it is possible from technology view)

btw, maybe keeping the long one is better. I can't make a conclusion now.

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 20, 2016

@Merovius
about the sync problem, we should think by jumping out of the box.
For example, one possible solution is to maintain a global same-data merging list. We can defer to do the mergings at a safe time point later. For example, at the time when GC stops the world.

@alandonovan

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2016

@yaxinlx what problem are you trying to solve?

@dr2chase

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2016

Assuming that the GC stops the world, of course.

There's also issues with stack==heap, or equality between stack allocations from different frames.

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 20, 2016

@alandonovan
less memory consuming, faster comparing.

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 20, 2016

@dr2chase,
I don't think it is a good idea to just close this issue for there is no good implementations at this time.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2016

This is not remotely feasible with our current implementation. There is no stop-the-world pass that can modify local variables. There is no support for pointer chasing. I think that leaving this closed is correct. If the architecture of the system changes significantly--for example, if we implement a moving garbage collector, which we currently have no plans to do--we can revisit.

@yaxinlx

This comment has been minimized.

Copy link
Author

commented Oct 21, 2016

@ianlancetaylor
I never expect to implement it at the time when GC stops the world.
I just made an example to show there are many potential ways to implement it.

It is ridiculous to give up an improvement just for there is not a simple good implementation at current time.

Does this improvement break the compatibility?
Does it do any harm by keeping it open?
Aren't there many open issues for years?
This may be a small improvement, but isn't a language becoming better and better
with many small improvements?

Finally, if go team really think this improvement is pointless,
or you can confirm you will never get a simple good implementation for it,
I have no complains for closing this issue.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 21, 2016

There are far too many open issues already. Every open issue is a drag on the project, as we must periodically review it and see if we can move it along. I don't think it is useful to leave open issues for ideas that we don't think can be implemented.

@mikioh mikioh changed the title Release one copy of underlying data if the comparing result of two strings or two interfaces is true. runtime: Release one copy of underlying data if the comparing result of two strings or two interfaces is true. Oct 22, 2016

@golang golang locked and limited conversation to collaborators Oct 22, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.