Skip to content

Latest commit

 

History

History
182 lines (111 loc) · 5.05 KB

bytearray.md

File metadata and controls

182 lines (111 loc) · 5.05 KB

bytearrayimage title

contents

related file

  • cpython/Objects/bytearrayobject.c
  • cpython/Include/bytearrayobject.h
  • cpython/Objects/clinic/bytearrayobject.c.h

memory layout

The ob_alloc field represents the real allocated size in bytes

ob_bytes is the physical begin address, and ob_start is the logical begin address

ob_exports means how many other objects are sharing this buffer, like reference count in a way

memory layout

example

empty bytearray

>>> a = bytearray(b"")
>>> id(a)
4353755656
>>> b = bytearray(b"")
>>> id(b) # they are not shared
4353755712

empty

append

after append a charracter 'a', ob_alloc becomes 2, ob_bytes and ob_start all points to same address

a.append(ord('a'))

append_a

resize

The size grow pattern is shown in the code

    /* Need growing, decide on a strategy */
    if (size <= alloc * 1.125) {
        /* Moderate upsize; overallocate similar to list_resize() */
        alloc = size + (size >> 3) + (size < 9 ? 3 : 6);
    }
    else {
        /* Major upsize; resize up to exact size */
        alloc = size + 1;
    }

In appending, ob_alloc is 2, and request size is 2, 2 <= 2 * 1.125, so the new allocated size is 2 + (2 >> 3) + 3 ==> 5

a.append(ord('b'))

resize

slice

b = bytearray(b"abcdefghijk")

slice

After the slice operation, ob_start points to the real beginning of the content, and ob_bytes still points to the begin address of the malloced block

b[0:5] = [1,2]

after_slice

as long as the slice operation is going to shrink the bytearray, and the new_size < allocate / 2 is False, the resize operation won't shrink the real malloced size

b[2:6] = [3, 4]

after2_slice

now, in the shrink operation, the new_size < allocate / 2 is True, the resize operation will be triggered

b[0:3] = [7,8]

after3_slice

The growing pattern in slice operation is the same as the append operation

request size is 6, 6 < 6 * 1.125, so new allocated size is 6 + (6 >> 3) + 3 ==> 9

b[0:3] = [1,2,3,4]

after_grow_slice

ob_exports

what's field ob_exports mean ? If you need detail, you can refer to less-copies-in-python-with-the-buffer-protocol-and-memoryviews and PEP 3118

buf = bytearray(b"abcdefg")

exports

the bytearray implements the buffer protocol, and memoryview is able to access the internal data block via the buffer protocol, mybuf and buf are all sharing the same internal block

field ob_exports becomes 1, which indicate how many objects currently sharing the internal block via buffer protocol

mybuf = memoryview(buf)
mybuf[1] = 3

exports_1

so does mybuf2 object(ob_exports doesn't change because you need to call the c function defined by buf object via the buffer protocol, mybuf2 barely calls the slice function of mybuf)

mybuf2 = mybuf[:4]
mybuf2[0] = 1

exports_2

ob_exports becomes 2

mybuf3 = memoryview(buf)

exports_3

ob_exports becomes 0

del mybuf
del mybuf2
del mybuf3

exports_4