Skip to content

v0.0.7

Latest
Compare
Choose a tag to compare
@edsu edsu released this 01 Dec 16:31
· 17 commits to main since this release

This release of unshrtn shortens the lines of code (pun intended) quite a bit because URL resolution and metadata extraction has been pulled into to a separate Node library named metaweb. Unsrhrtn just provides the HTTP microservice and caches responses in LevelDB.

As a result unshrtn will now extract basic Facebook and Twitter metadata from pages such as images, description, keywords:

% curl http://localhost:3000/?url=https://wapo.st/2AyS2Z6

{
  "canonical": "https://www.washingtonpost.com/politics/michael-flynn-charged-with-making-false-statement-to-the-fbi/2017/12/01/e03a6c48-d6a2-11e7-9461-ba77d604373d_story.html",
  "status": 200,
  "content_type": "text/html;charset=UTF-8",
  "title": "Michael Flynn pleads guilty to lying to the FBI - The Washington Post",
  "description": "Flynn’s admission is an ominous sign for the White House, and court documents indicate he is cooperating with the special counsel.",
  "image": "https://www.washingtonpost.com/rf/image_1484w/2010-2019/WashingtonPost/2017/12/01/National-Politics/Images/2017-12-01T153351Z_1251496387_HP1EDC1178EAU_RTRMADP_3_USA-TRUMP-RUSSIA.jpg?t=20170517",
  "publisher": "Washington Post",
  "keywords": [
    "michael flynn",
    "flynn fbi",
    "flynn lying",
    "fbi",
    "sergey kislyak",
    "robert s. mueller iii",
    "united nations security council"
  ],
  "short": "https://wapo.st/2AyS2Z6",
  "long": "https://www.washingtonpost.com/politics/michael-flynn-charged-with-making-false-statement-to-the-fbi/2017/12/01/e03a6c48-d6a2-11e7-9461-ba77d604373d_story.html?utm_term=.e692489252ff"
}

Ideas for more metadata to extract are welcome here, or over in the metaweb issue tracker.