Box Static download links. #50

Open
KuroSaru opened this Issue Oct 10, 2016 · 10 comments

Projects

None yet

6 participants

@KuroSaru

Would it be possible to provide static direct download links along with the current ones, so that it be possible to parse the csv/json on new pushes and auto grab pdfs as they are added to the list.

@Taskr
Taskr commented Oct 10, 2016 edited

Hi KuroSaru, here is one of my custom auto-download scripts using beautiful soup and requests to parse out links in the current JSON.

Python3+ Version (Synchronous Script & Requirements File)
APTnotes_sync_download.py.txt

APTnotes_sync_requirements.txt

Update: You can also look at my comment below dated Dec 1, 2016 for Python3.4 and 3.5 asynchronous scripts for faster downloads.

@kbandla kbandla self-assigned this Oct 10, 2016
@kbandla
Contributor
kbandla commented Oct 10, 2016

This should not be this difficult. I will look into making this easier for everyone. @Taskr 's solution should work in the interim.

@kbandla kbandla added the enhancement label Oct 11, 2016
@beast-fighter

possible solution: https://docs.box.com/reference#create-a-shared-link-for-a-file -- we could just gen the share link and public static link for the json and csvs

@pedrodj
pedrodj commented Nov 8, 2016

SyntaxError: Non-ASCII character '\xe2' in file C:\autodownload.py on line
49, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for deta
ils

Any help?

@threatminer
threatminer commented Nov 8, 2016 edited

Add this to the top of the script:

# -*- coding: UTF-8 -*-

The top should look like this:

#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import os
import hashlib
import json
import requests
from bs4 import BeautifulSoup
@pedrodj
pedrodj commented Nov 8, 2016

Is it possible to have autodownload.py that works? :)

Thank you ;)

@threatminer threatminer closed this Nov 8, 2016
@threatminer threatminer reopened this Nov 8, 2016
@threatminer

Here's the script with the changes I just recommended...

autodownload.py.txt

@pedrodj
pedrodj commented Nov 8, 2016

Traceback (most recent call last):
File "C:\autodownload.py", line 6, in
import requests
ImportError: No module named requests

@threatminer

That's not an error with the code - you just need to install the 'requests' module.

pip install requests or easy_install requests

@Taskr
Taskr commented Dec 1, 2016 edited

Significantly faster auto-downloader scripts (uses asyncio) with a requirements file

Uses beautiful soup, asyncio, and aiohttp to parse out download links.

Python3.4+ Version
APTnotes_async_download_python34.py.txt

Python3.5+ Version
APTnotes_async_download_python35.py.txt

Requirements File
APTnotes_async_requirements.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment