Medium migration flow

[devformatters]

Since we are reshaping our feature from calling Medium API to uploading Medium source files, couple things need to discuss.

📦 Files

The packed files downloaded from Medium is quite big because it has lots of irrelevant files. Below are unpacked files:

├── blocks
│   └── blocked-users-0001.html
│  
├── bookmarks
│   └── bookmarks-0001.html
│  
├── claps
│   └── claps-0001.html
│  
├── highlights
│   └── highlights-0001.html
│  
├── interests
│   ├── publications.html
│   ├── tags.html
│   ├── topics.html
│   └── writers.html
│  
├── ips
│   └── ips-0001.html
│  
├── posts
│   ├── 2018-04-02_-----------Arendt----51bc52c880f3.html
│   ├── 2018-04-12_-----------------3a905851316e.html
│   ├── 2018-04-12_------------1253c94fa6ac.html
│   ├── 2018-05-31_Matters--------------ae9f9aa98249.html
│   ├── 2018-11-10_-Matters--------------70c1ab6d47e2.html
│   ├── 2019-03-22_Matters-------------------6dc72e6753f9.html
│   ├── 2019-03-27_---------------------c4336ab683df.html
│   ├── 2019-04-02_------------------12bdf59fe4a9.html
│   ├── 2019-04-24_----------------------2fd7c25b0934.html
│   └── draft_nn-813be4d2bd80.html
│  
├── profile
│   ├── memberships.html
│   ├── profile.html
│   └── publications.html
│  
├── pubs-following
│   └── pubs-following-0001.html
│  
├── sessions
│   └── sessions-0001.html
│  
├── topics-following
│   └── topics-following-0001.html
│  
└── users-following
    ├── users-following-0001.html
    └── users-following-0002.html

As you see, there are some user information and settings. All we need is posts folder. Comments are considered as posts from Medium view, so comments are also packed in posts folder. Do we want user to upload one package? Or just upload real posts by picking ?

🧑🏻‍🔬 Process flow

Possible process flows are here:

Based on current design, uploading packed files would be the easiest way for user but not for us. Also, some comments' contents will be listed in drafts. In opposite, uploading multi-files would be the simplest way for us and we could get the right uploaded files (real posts), but users might need to drop files couple times.

FYI, editor has an upload button on the right sidebar.

Love to hear you guys ideas 🧑🏻‍💻

[proformatters]

I think we only need to upload real posts, and no need to include comments. In addition, uploading the packed files should be more user-friendly...

[guoliu]

Great summary!

I think the core functionality as well as the PoC should be allowing user to upload multiple html files and turns them into drafts. So that it is general and robust, instead of medium.com specific and messy. If we need to simplify user interaction, we can think of a solution on top of this, preferably on the frontend.

In my opinion, picking html files instead of uploading the same pack is not that much of a burden if the user really needs this functionality, especially when html files are easily previewed everywhere. We can frame this feature as a general html upload functionality, and medium.com specific instruction can live in landing page.

If we go with this design, I think we should be able to skip intermediary storage (s3 or postgres) entirely, and just hold the html string in queue, process it, then write to draft table.

There's one small issue with design though,

editor has an upload button on the right sidebar.

if we allow multiple file upload, logically the upload entrance should be in draft page, as we discussed before. What do you think @gyuetong ?

[devformatters]

We've shipped the feature. Close it.