mmap.resize and offset

[ocean-city]

BPO	5282
Nosy	@tjguk
Files	fix_offset_and_resize.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-02-17.13:19:16.187>
created_at = <Date 2009-02-16.17:21:47.528>
labels = ['extension-modules', 'type-crash']
title = 'mmap.resize and offset'
updated_at = <Date 2009-02-17.13:19:16.186>
user = 'https://bugs.python.org/ocean-city'

bugs.python.org fields:

activity = <Date 2009-02-17.13:19:16.186>
actor = 'ocean-city'
assignee = 'none'
closed = True
closed_date = <Date 2009-02-17.13:19:16.187>
closer = 'ocean-city'
components = ['Extension Modules']
creation = <Date 2009-02-16.17:21:47.528>
creator = 'ocean-city'
dependencies = []
files = ['13117']
hgrepos = []
issue_num = 5282
keywords = ['patch']
message_count = 6.0
messages = ['82253', '82254', '82256', '82300', '82323', '82337']
nosy_count = 2.0
nosy_names = ['ocean-city', 'tim.golden']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue5282'
versions = ['Python 2.7']

[ocean-city]

This is first time to look at mmap module, so sorry if I'm saying
totally wrong thing. (I noticed this when I saw bpo-2733) I think the
behavior of mmap_resize_method is unclear when mapping object is created
with offset > 0.

From view of other functions, it seems that self->offset is offset from
beginning of the file, and self->size is the size of mapped area from
self->offset. If so,

#if SIZEOF_SIZE_T > 4
    newSizeHigh = (DWORD)((self->offset + new_size) >> 32);
    newSizeLow = (DWORD)((self->offset + new_size) & 0xFFFFFFFF);
#else
    newSizeHigh = 0;
    newSizeLow = (DWORD)new_size; /* shouldn't add self->offset? this is
not doing same thing as SIZEOF_SIZE_T > 4 */
#endif

And on unix part,
newmap = mremap(self->data, self->size, new_size, MREMAP_MAYMOVE);
self->offset is totally ignored. I think when self->offset > 0,
something wrong happens.

And comment above function definition saids,

/ Is this really necessary? This could easily be done
/ from python by just closing and re-opening with the
/ new size?

I think this function is not tested well. There is no test about
resize+offset in test_mmap.py.

[tjguk]

Have a look at bpo-2733

http://bugs.python.org/issue2733

where I've just proposed a patch in this area. I'm also
not sure exactly what's going on, but I have patched what
I believe is a linked pair of bugs in that code.

[ocean-city]

Yes, I noticed this issue while investigating bpo-2733. But sorry, I
couldn't reproduce the bug on my machine. Even so, LPDWORD arg for
SetPointer can be modified, so your patch looks correct. (There is
SetPointerEx which can make the code simpler, but this function seems
not available on WinCE)

[ocean-city]

I investigated more. Following code crashed with bus error on coLinux.

import mmap

def main():
    align = mmap.ALLOCATIONGRANULARITY
    with open("a.txt", "w") as f:
        f.write("0" * align)
        f.write("1" * align)
        f.write("2" * align)
    with open("a.txt", "r+") as f:
        m = mmap.mmap(f.fileno(), align, offset=align*2)
    m.resize(1)
    print m[0] # bus error

if __name__ == '__main__':
    main()

On windows, failed with AccessDenied. (32bit Win2000)

[ocean-city]

Here is a patch.

[ocean-city]

I hope this is obvious too. Fixed in r69718.