Negative _io.BufferedReader.tell

[senyai]

Bug report

I believe tell should never be negative, even for /dev/urandom. And @benjaminp wrote ToDo that it shouldn't be negative.

>>> from pathlib import Path
>>> urandom = Path('/dev/urandom').open('rb')
>>> urandom.tell()
0
>>> urandom.read(1)
b'$'
>>> urandom.tell()
-4095

Environment

Fedora linux, python 3.10.6

Linked PRs

• gh-95782: Fix io.BufferedReader.tell etc. being able to return offsets < 0 #99709
• [3.12] gh-95782: Fix io.BufferedReader.tell() etc. being able to return offsets < 0 (GH-99709) #115599
• [3.11] gh-95782: Fix io.BufferedReader.tell() etc. being able to return offsets < 0 (GH-99709) #115600

[corona10]

Python 3.12.0a0 (heads/main:71c3d649b5, Aug 10 2022, 15:07:52) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> urandom = Path('/dev/urandom').open('rb')
>>> urandom.tell()
0
>>> urandom.read(1)
b'\x80'
>>> urandom.tell()
-4095
>>>

Reproducible

[6t8k]

This is caused by lseek() always returning 0 for character pseudo-devices like /dev/urandom (for other non-regular files, e.g. /dev/stdin, it always returns -1, to which CPython reacts by raising appropriate exceptions). They are thus technically seekable despite not having seek semantics.

When calling _io.BufferedReader.read(1) it reads ahead, filling its buffer (the size of which is usually being determined by the underlying file's block size as returned by fstat, which it inherits from its file system and is often 4096 bytes), creating a discrepancy between the number of bytes read and the internal tell() always returning 0 for an instance that wraps such a file. I pushed a PR that fixes this, and would appreciate any feedback.

---

I also bumped into further not necessarily expected behavior, of which I'm not sure if it can be a problem, and which I didn't change:

• _io.BufferedWriter.tell() and _pyio.BufferedWriter.tell() returning the number of bytes written, seek(0, (_py)io.SEEK_CUR) resetting, which is not supposed to change the offset (both noteworthy in their own right)
• io.Buffered{Reader,Random}.seek(0, io.SEEK_CUR) returning the number of bytes read, tell() resets it, which is not supposed to change the offset (both noteworthy in their own right)

Demo:

import io
import _pyio as pyio

# io.BufferedWriter and _pyio.BufferedWriter tell() returning number of bytes written, seek(0, io.SEEK_CUR) resets it, which is not supposed to change the offset.
f = open("/dev/urandom", "wb")
print(f.tell()) # 0
print(f.write(b"1")) # 1
print(f.write(b"12")) # 2
print(f.tell()) # 3
print(f.seek(0, io.SEEK_CUR)) # 0
print(f.tell()) # 0
f.close()

print("---")

f = pyio.open("/dev/urandom", "wb")
print(f.tell()) # 0
print(f.write(b"1")) # 1
print(f.write(b"12")) # 2
print(f.tell()) # 3
print(f.seek(0, io.SEEK_CUR)) # 0
print(f.tell()) # 0
f.close()

print("---")

# io.BufferedReader/Random seek(0, io.SEEK_CUR) returning the number of bytes read, tell() resets it. Only the C implementation is affected.
f = open("/dev/urandom", "rb")
print(f.tell()) # 0
print(f.read(1)) # b'\xf1'
print(f.read(2)) # b'X\x1a'
print(f.seek(0, io.SEEK_CUR)) # 3
print(f.tell()) # 0 with the fix, -4093 without
print(f.seek(0, io.SEEK_CUR)) # 0 with the fix, -4093 without
print(f.tell()) # 0 with the fix, -4093 without
f.close()

print("---")

f = pyio.open("/dev/urandom", "rb")
print(f.tell()) # 0
print(f.read(1)) # b'\xaa'
print(f.read(2)) # b'[\x96'
print(f.seek(0, io.SEEK_CUR)) # 0
print(f.tell()) # 0
print(f.seek(0, io.SEEK_CUR)) # 0
print(f.tell()) # 0
f.close()