humanize_bytes incorrect in python2 and 3: new function proposal

[zopar]

Hi
I installed shodan-python with python 3
The program crash on the initialization on init key
Traceback (most recent call last):
File "/usr/local/bin/shodan", line 30, in
import shodan
File "/usr/local/lib/python3.5/dist-packages/shodan/init.py", line 2, in
from shodan.client import Shodan
File "/usr/local/lib/python3.5/dist-packages/shodan/client.py", line 16, in
from .helpers import api_request
File "/usr/local/lib/python3.5/dist-packages/shodan/helpers.py", line 155
(1<<50L, 'PB'),
^
SyntaxError: invalid syntax

This happen because in python 3 long integer are now parts of int and the L is not need anymore:
https://docs.python.org/3/whatsnew/3.0.html

I removed the L from
abbrevs = (
(1<<50L, 'PB'),
(1<<40L, 'TB'),
(1<<30L, 'GB'),
(1<<20L, 'MB'),
(1<<10L, 'kB'),
(1, 'bytes')
)
and works in python 3

But you also have a problem with python 2 because the division round the result
If you try with this:
humanize_bytes(102412341111,1)
You will obtain 1.0 and not 1.3 as expected

The problem is in
bytes / factor
you need to define factor as long to force python 2 to use long and have correct results
bytes / long(factor)

This is a proposal function that works correctly with python 2 and 3

def humanize_bytes(bytes, precision=1):
    """Return a humanized string representation of a number of bytes.
    >>> humanize_bytes(1)
    '1 byte'
    >>> humanize_bytes(1024)
    '1.0 kB'
    >>> humanize_bytes(1024*123)
    '123.0 kB'
    >>> humanize_bytes(1024*12342)
    '12.1 MB'
    >>> humanize_bytes(1024*12342,2)
    '12.05 MB'
    >>> humanize_bytes(1024*1234,2)
    '1.21 MB'
    >>> humanize_bytes(1024*1234*1111,2)
    '1.31 GB'
    >>> humanize_bytes(1024*1234*1111,1)
    '1.3 GB'
    """
    abbrevs = (
        (1<<50, 'PB'),
        (1<<40, 'TB'),
        (1<<30, 'GB'),
        (1<<20, 'MB'),
        (1<<10, 'kB'),
        (1, 'bytes')
    )
    if bytes == 1:
        return '1 byte'
    for factor, suffix in abbrevs:
        if bytes >= factor:
            break
    return '%.*f %s' % (precision, bytes / float(factor), suffix)

Also I would like to propose this new functions that does not need the shift and works directly with division and not need break (based on diveintopython3)
Please note that in this one the minimum results are reported in kb
for example 10 B are reported as 0.010 kb

def humanize_bytes(bytes, precision=1):
    """Return a humanized string representation of a number of bytes.
    >>> humanize_bytes(1)
    '1 byte'
    >>> humanize_bytes(1024)
    '1.0 kB'
    >>> humanize_bytes(1024*123)
    '123.0 kB'
    >>> humanize_bytes(1024*12342)
    '12.1 MB'
    >>> humanize_bytes(1024*12342,2)
    '12.05 MB'
    >>> humanize_bytes(1024*1234,2)
    '1.21 MB'
    >>> humanize_bytes(1024*1234*1111,2)
    '1.31 GB'
    >>> humanize_bytes(1024*1234*1111,1)
    '1.3 GB'
    """

    if bytes == 1:
        return '1 byte'
    if bytes < 1024:
        return '%.*f %s' % (precision, bytes, "bytes")
   
    suffixes = ['KB', 'MB', 'GB', 'TB', 'PB']
    multiple = 1024.0    #.0 force float on python 2
    for suffix in suffixes:
        bytes /= multiple
        if bytes < multiple:
            return '%.*f %s' % (precision, bytes, suffix)

[zopar]

I added a pull request for the second functions
If you like to use the first one, please let me know, I will open a pull request fro the first one.

Regards
zopar

[achillean]

Thank you for the pull request, I've merged it with the main branch and made a small tweak so it also returns a value past 1024 PB. I've also released a new version of the Shodan library on PyPi so the issue should be resolved now!