Box Static download links. #50

KuroSaru opened this Issue Oct 10, 2016 · 10 comments


None yet

6 participants


Would it be possible to provide static direct download links along with the current ones, so that it be possible to parse the csv/json on new pushes and auto grab pdfs as they are added to the list.

Taskr commented Oct 10, 2016 edited

Hi KuroSaru, here is one of my custom auto-download scripts using beautiful soup and requests to parse out links in the current JSON.

Python3+ Version (Synchronous Script & Requirements File)


Update: You can also look at my comment below dated Dec 1, 2016 for Python3.4 and 3.5 asynchronous scripts for faster downloads.

@kbandla kbandla self-assigned this Oct 10, 2016
kbandla commented Oct 10, 2016

This should not be this difficult. I will look into making this easier for everyone. @Taskr 's solution should work in the interim.

@kbandla kbandla added the enhancement label Oct 11, 2016

possible solution: -- we could just gen the share link and public static link for the json and csvs

pedrodj commented Nov 8, 2016

SyntaxError: Non-ASCII character '\xe2' in file C:\ on line
49, but no encoding declared; see for deta

Any help?

threatminer commented Nov 8, 2016 edited

Add this to the top of the script:

# -*- coding: UTF-8 -*-

The top should look like this:

# -*- coding: UTF-8 -*-
import os
import hashlib
import json
import requests
from bs4 import BeautifulSoup
pedrodj commented Nov 8, 2016

Is it possible to have that works? :)

Thank you ;)

@threatminer threatminer closed this Nov 8, 2016
@threatminer threatminer reopened this Nov 8, 2016

Here's the script with the changes I just recommended...

pedrodj commented Nov 8, 2016

Traceback (most recent call last):
File "C:\", line 6, in
import requests
ImportError: No module named requests


That's not an error with the code - you just need to install the 'requests' module.

pip install requests or easy_install requests

Taskr commented Dec 1, 2016 edited

Significantly faster auto-downloader scripts (uses asyncio) with a requirements file

Uses beautiful soup, asyncio, and aiohttp to parse out download links.

Python3.4+ Version

Python3.5+ Version

Requirements File

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment