bufio: bug in the emptyFinalToken example for Scanner

[pam4]

The SplitFunc in the emptyFinalToken example for bufio.Scanner doesn't handle the case of requesting more data if necessary.

The example only happens to work because the whole input is returned by a single call to Read (it uses a strings.Reader).

This should be enough to fix it:

@@ -21,7 +21,10 @@
        // There is one final token to be delivered, which may be the empty string.
        // Returning bufio.ErrFinalToken here tells Scan there are no more tokens after this
        // but does not trigger an error to be returned from Scan itself.
-       return 0, data, bufio.ErrFinalToken
+       if atEOF {
+           return 0, data, bufio.ErrFinalToken
+       }
+       return 0, nil, nil
    }
    scanner.Split(onComma)
    // Scan.

The problem is demonstrated here:
https://play.golang.org/p/CM-sWovg8mg

[meirf]

I think we should include logic like the OP includes since that's an important general pattern for the Scanner and none of the other examples directly include logic for requesting more data - all the other examples (transitively) have similar checks but only via their usage of the Scan* SplitFuncs.

cc: @ianlancetaylor

[gopherbot]

Change https://golang.org/cl/157758 mentions this issue: bufio: fix emptyFinalToken example to handle multiple Reads