Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix range slicing and indexing to handle lengths > sys.maxsize #55098

ncoghlan opened this issue Jan 11, 2011 · 6 comments

Fix range slicing and indexing to handle lengths > sys.maxsize #55098

ncoghlan opened this issue Jan 11, 2011 · 6 comments


Copy link

BPO 10889
Nosy @birkenfeld, @mdickinson, @ncoghlan
  • issue10889_range_subscripts.diff: Indexing and slicing for large ranges
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = ''
    closed_at = <Date 2011-01-12.03:22:45.097>
    created_at = <Date 2011-01-11.15:59:27.770>
    labels = []
    title = 'Fix range slicing and indexing to handle lengths > sys.maxsize'
    updated_at = <Date 2011-01-12.07:53:56.689>
    user = '' fields:

    activity = <Date 2011-01-12.07:53:56.689>
    actor = 'mark.dickinson'
    assignee = 'georg.brandl'
    closed = True
    closed_date = <Date 2011-01-12.03:22:45.097>
    closer = 'ncoghlan'
    components = []
    creation = <Date 2011-01-11.15:59:27.770>
    creator = 'ncoghlan'
    dependencies = []
    files = ['20355']
    hgrepos = []
    issue_num = 10889
    keywords = ['patch']
    message_count = 6.0
    messages = ['126017', '126023', '126027', '126029', '126030', '126069']
    nosy_count = 3.0
    nosy_names = ['georg.brandl', 'mark.dickinson', 'ncoghlan']
    pr_nums = []
    priority = 'normal'
    resolution = 'accepted'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = ''
    versions = ['Python 3.2']

    Copy link
    Contributor Author

    Enhancement to range to correctly handle indexing and slicing when len(x) raises OverflowError.

    Note that this enables correct calculation of the length of such ranges via:

            def _range_len(x):
                    length = len(x)
                except OverflowError:
                    step = x[1] - x[0]
                    length = 1 + ((x[-1] - x[0]) // step)
                return length

    @ncoghlan ncoghlan self-assigned this Jan 11, 2011
    Copy link
    Contributor Author

    Having started work on this, the code changes are probably too significant to consider adding it to 3.2 at this late stage.

    Writing my own slice interpretation support which avoids the ssize_t limit is an interesting exercise :)

    Copy link
    Contributor Author

    Attached patch moves range indexing and slicing over to PyLong and updates the tests accordingly.

    Georg, I think this really makes the large range story far more usable - if you're OK with it, I would like to check it in this week so it lands in 3.2.

    @ncoghlan ncoghlan assigned birkenfeld and unassigned ncoghlan Jan 11, 2011
    Copy link
    Contributor Author

    Oh, and to explain my negative comment from earlier: that was my reaction when I realised I also needed to write PyLong versions of _PyEval_SliceIndex and PySlice_GetIndicesEx to make range slicing with large integers work properly.

    As it turned out, the end result wasn't as scary as I initially feared (while compute_slice_indices is quite long, most of that is just the verbosity of PyLong arithmetic).

    Copy link

    It's a moderate chunk of code, but lots of new tests... I'd say go for it.

    Copy link
    Contributor Author

    Committed as r87948.

    I added a few large_range tests to those in the patch. I checked that IndexError is raised when appropriate, as well as a specific test for the combination of a large range with a large negative step.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    None yet
    None yet

    No branches or pull requests

    2 participants