Skip to content
Download all content from Medium to local folder
Crystal Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
spec
src
.dockerignore
.editorconfig
.gitignore
CODE_OF_CONDUCT.md
Dockerfile
LICENSE
README.md
shard.yml

README.md

Medup

Sync all content from Medium with local folder via API

Usage

  1. Generate token on the page https://medium.com/me/settings
  2. Create environment variable MEDIUM_TOKEN=<token>
  3. Run sync command:
$ crystal run src/cli.cr -- -u miry -d posts/miry
$ crystal run src/cli.cr -- -u pftg -d posts/pftg

Docker

$ docker run -v $(pwd)/posts:/posts -it miry/medup -u miry

Development

TODO

  • Clean code
  • Standardize logging
  • Save in Markdown format with correct images
  • Extract Medium API to a separate shard
  • Create posts from local Markdown articles
  • Update a post content from local files
  • Extract posts from Publishers
  • Create Rake task management

Play ownself

Medium API

  1. Medium API documentation
  2. Verify token with sample query:
$ curl -H "Authorization: Bearer <token>" https://api.medium.com/v1/me
{"data":{"id":"number","username":"miry","name":"Michael Nikitochkin","url":"https://medium.com/@miry","imageUrl":"https://cdn-images-1.medium.com/fit/c/400/400/0*KgbjgGnH-csHuB8j."}}

Crawler

  1. Check public information not covered by API
$ curl "https://medium.com/@miry?format=json" | cut -c17- && : Remove in the front from response some strange JS code.
  1. Pagination
$ curl "https://medium.com/@miry/latest?format=json&limit=100" | cut -c17- && : Remove in the front from response some strange JS code.
  1. Post info
curl -s -H "Content-Type: application/json" https://medium.com/@miry/c35b40c499e\?format\=json\&limit\=100
  1. Stream:
$ curl -s -H "Content-Type: application/json" "https://medium.com/_/api/users/fdf238948af6/profile/stream" | cut -c17-
$ curl -s -H "Content-Type: application/json" "https://medium.com/_/api/users/fdf238948af6/profile/stream?limit=100&page=3" | cut -c17- > stream.json
$ cat stream.json| jq ".payload.references.Post[].title"
$ cat stream.json| jq ".payload.paging.next"
You can’t perform that action at this time.