Explain Like I'm 5 - API Call, Document Export

[Rustymage]

Hi,

Sorry if this is not allowed but I'm really enjoying self-hosting Teedy. However, I'd like to have the reassurance that should I decide not to self-host I can pull all my documents out the system and onto a local hard drive.

I believe this is possible with the API, however, beyond getting an auth cookie and a total document list, I'm stuck!

How would I get the documents out? api/document/list states I have 176.

Please help, I'm willing to read and learn but API calls are new to me.

Cheers in advance!

Rusty

[jendib]

Using Teedy self-hosted is perfectly fine is you are ready to take care of the server and the maintenance yourself.

You checked the documentation here: https://demo.teedy.io/apidoc/ ?
For you case, document/list is a good start, then you can call document/ID (with the ID of the document you want) and file/list?id=ID for the list of files in this document.
After that you can download each file with file/FILEID/data (with the file ID).

[Rustymage]

I've read the documentation but got stumped at calling the API correctly, beyond getting a list of document IDs returned.

I'll try your comments. Thanks for the pointers!

I'll report back then close, hopefully I'll help someone else in future once I've worked it out.

[jendib]

If you feel it, you could even improve the documentation based on your experience :)

[vmario89]

Hi,

Sorry if this is not allowed but I'm really enjoying self-hosting Teedy. However, I'd like to have the reassurance that should I decide not to self-host I can pull all my documents out the system and onto a local hard drive.

I believe this is possible with the API, however, beyond getting an auth cookie and a total document list, I'm stuck!

How would I get the documents out? api/document/list states I have 176.

Please help, I'm willing to read and learn but API calls are new to me.

Cheers in advance!

Rusty

hi, i made a simple cURL script for this task (for backing up the backup of the backup, and to share the files to some old-fashioned poeple xD)

#!/bin/bash
BASE_URL="https://leyghis.fablabchemnitz.de:8080"
AUTH_TOKEN=$(curl -i -X POST -d username="theUser" -d password="thePassword" "$BASE_URL/api/user/login" -k|tail -n3|cut -c24-59)
TARGET_JSON_FILE="/tmp/documentlist.json"
 
#create backup directory
BACKUP_DIR="/backup/sismics-data"
mkdir -p "$BACKUP_DIR"
 
curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/document/list?limit=0" -k | jq . > "$TARGET_JSON_FILE"
 
#read the complete list of documents
jq -c '.|{documents}|.[]|.[]|{id}+{title}' "$TARGET_JSON_FILE" | while read i; do
        #parse the line and get parameters from the line
        DOC_ID=$(jq -c '.|{id}|.id' <<< $(echo $i))
        DOC_NAME=$(jq -c '.|{title}|.title' <<< $(echo $i))
        echo _debug DOC_ID=${DOC_ID:1:-1}, DOC_NAME=${DOC_NAME:1:-1}
        curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL"/api/file/zip?id="${DOC_ID:1:-1}" -k -o "$BACKUP_DIR"/"${DOC_ID:1:-1}".zip
done
 
#logout if finished
curl --silent -X POST -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/user/logout" -k
 
#remove the complete list of documents
rm "$TARGET_JSON_FILE"

tested with Teedy 1.6 on Ubuntu. You need to install "curl" and "jq" before running. Otherwise it will fail. I did not test with Teedy 1.7 or newer commits so far. Maybe sth. changed in the meantime. Last test was a half year ago.

[jendib]

It should work with newer versions also, I only added things on the API, no removal/changes.

[Rustymage]

Thanks for this!

I've tried it with limited success...

Here's the code:

#!/bin/bash
BASE_URL="https://websiteadress.co.uk"
AUTH_TOKEN=$(curl -i -X POST -d username="SuperUser" -d password="P455W0rd" "$BASE_URL/api/user/login" -k|tail -n3|cut -c24-59)
TARGET_JSON_FILE="/Users/SuperUser/backup/documentlist.json"
 
#create backup directory
BACKUP_DIR="/Users/SuperUser/backup/sismics-data"
mkdir -p "$BACKUP_DIR"
 
curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/document/list?limit=0" -k | jq . > "$TARGET_JSON_FILE"
 
#read the complete list of documents
jq -c '.|{documents}|.[]|.[]|{id}+{title}' "$TARGET_JSON_FILE" | while read i; do
        #parse the line and get parameters from the line
        DOC_ID=$(jq -c '.|{id}|.id' <<< $(echo $i))
        DOC_NAME=$(jq -c '.|{title}|.title' <<< $(echo $i))
        echo _debug DOC_ID=${DOC_ID:1:-1}, DOC_NAME=${DOC_NAME:1:-1}
        curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL"/api/file/zip?id="${DOC_ID:1:-1}" -k -o "$BACKUP_DIR"/"${DOC_ID:1:-1}".zip
done
 
#logout if finished
curl --silent -X POST -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/user/logout" -k
 
#remove the complete list of documents
rm "$TARGET_JSON_FILE"

Here's the response:

MacBook-Pro:~ SuperUser$ ./teedybackup.sh 
jq: error (at /Users/SuperUser/backup/documentlist.json:4): Cannot iterate over null (null)
{"type":"ForbiddenError","message":"You don't have access to this resource"}(base)

If I manually insert the AUTH_TOKEN I get a list of documents but then this error:

./teedybackup.sh: line 17: -1: substring expression < 0

I'm on v1.8.

Perhaps someone has a little more experience then me on this and can point me in the right direction?

Thanks in advance again!

[Rustymage]

So I've tried to find the issue and tried the code:

curl -i -X GET -H "Cookie: auth_token=1-2-3-4-5" http://IPAddress:8100/api/document/pdf/?id=DOC_ID -o /Users/SuperUser/test.pdf

All I get is a non-readable PDF downloaded...

Is this right? It doesn't appear so.

[jendib]

That's not the API call to download a PDF.
It's /api/document/<DOC_ID>/pdf?metadata=false&comments=false&fitimagetopage=true&margin=10

[Rustymage]

Thanks for the heads up. Using...

curl -i -X GET -H "Cookie: auth_token=1-2-3-4" http://IPAddress:8100/api/document/DOC_ID 1234/pdf?metadata=false&comments=false&fitimagetopage=true&margin=10 -o /Users/SuperUser/test.pdf

I get:

[1] 18617
[2] 18618
[3] 18619
-bash: -o: command not found
[2]-  Done                    comments=false
[3]+  Done                    fitimagetopage=true
(base) MacBook-Pro:~ anthonybetts$ HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Expires: 0
Content-Type: application/json
Content-Length: 61
Server: Jetty(9.4.12.v20180830)

{"type":"ValidationError","message":"margin is not a number"}

Removing the text metadata=false&comments=false&fitimagetopage=true&margin=10 or /pdf?metadata=false&comments=false&fitimagetopage=true&margin=10 just results in an unreadable file...

What am I doing wrong? Why are the PDFs downloaded unreadable?

I appreciate the help on this!

:)

[jendib]

The PDF is unreadable because it's a not a PDF, it's an error in JSON format.
The documentation says margin is a mandatory parameter. Also you need to escape the & characters for bash.

[Rustymage]

Hmmm...

HTTP/1.1 404 Not Found
Server: nginx
Date: Sun, 29 Sep 2019 18:59:38 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000;includeSubDomains; preload

So the document isn't found according to this but I know it's on the server.

I've tried the normal user account and that has no access, only the admin gets a document list.

I'm really confused about why it can't find the documents, or is it not finding the server to request from? It can list all the docs, just not download...is my install broken or the API not enabled?

[Rustymage]

So I've checked the web link given to me when I click on the export button and it works when used in a browser, however, when using curl, I get 404 Not Found.

Any ideas?

[jendib]

Maybe you're missing the auth token, it's hard to know without any specifics.

[Rustymage]

Right, so something is now working.

I can now download.

I'll place my working script here, taken from above and modified to work for me:

#!/bin/bash
BASE_URL="http://192.168.1.102:8100"
AUTH_TOKEN="direct add token here"
TARGET_JSON_FILE="/Users/backup/documentlist.json"

#create backup directory
BACKUP_DIR="/Users/backup/sismics-data"
mkdir -p "$BACKUP_DIR"

curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/document/list?limit=0" -k | jq . > "$TARGET_JSON_FILE"

#read the complete list of documents
jq -c '.|{documents}|.[]|.[]|{id}+{title}' "$TARGET_JSON_FILE" | while read i; do
        #parse the line and get parameters from the line
        DOC_ID=$(jq -c '.|{id}|.id' <<< $(echo $i))
        DOC_NAME=$(jq -c '.|{title}|.title' <<< $(echo $i))
        echo _debug DOC_ID="${DOC_ID:1:36}", DOC_NAME="${DOC_NAME:1:36}"
        curl --silent -X GET -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL"/api/file/zip?id="${DOC_ID:1:36}" -k -o "$BACKUP_DIR"/${DOC_ID:1:36}.zip
done

#logout if finished
#curl --silent -X POST -H "Cookie: auth_token=$AUTH_TOKEN" "$BASE_URL/api/user/logout" -k

#remove the complete list of documents
#rm "$TARGET_JSON_FILE"

I found {DOC_ID:1:36} needing adding as -1 wasn't liked and the AUTH_TOKEN automation wasn't liked either. I've commented out the log out so I can reuse the auth_token too.

Thanks for the help!