Python & command-line tool to gather text on the Web: Crawling & scraping, content extraction, metadata. TXT, Markdown, CSV & XML output.

Extract article or news by url or html, parse the title and content, output in markdown format.

Parse markdown article, download images and replace images URL's with local paths

디시인사이드 Client-Side 글 검색기 입니다.

A Google Docs HTML Cleaner: This program transforms messy HTML from Google Docs into clean code primarily using LXML with a modular mixin design pattern.

The main goal of an AI-Powered News Summarizer is to assist users in quickly understanding the main points and essential information from a large volume of news articles or textual content. By automatically summarizing news articles, it saves time and effort by providing users with a brief overview without having to read the entire article.

Arachnio client library for Python 3.10+

Crawling articles from websites

📚 Сборник полезных штук из Natural Language Processing: Определение языка текста, Разделение текста на предложения, Получение основного содержимого из html документа

NLP Web Service

(Ongoing module in development) Getting Wikipedia articles parsed content. Created for getting text corpuses data fast and easy. But can be freely used for other purpuses too

Reddit bot to preview and post hyperlinks as comments

An digital news/article scraper which extracts essential information about the content

Extract article (including images) by url and save to file.

Simple HTTP API endpoint that takes URL to any article and returns JSON object containing information about the article.

Nebula Expired Article Hunter is a marketing tool you can use to get expired content from www.archive.org A.K.A. wayback machine, you could use this kind of content to grow up your blog with evergreen information, improve your marketing campaigns without investing in writing services, or whatever you imagine is useful for.

Converts TXT files exported from Retriever Mediearkivet into JSON files with structured metadata

Ardio is a web application that converts CTV News articles into mp3 files. Currently, this only works for CTV, but in the future, I am planning on expanding it to more news soruces such as Global News or CNN.

The program can be used to scrape the content from an article from web by an input of a set of URLs in a text file or a URL. This project uses newspaper3k and python-docx libraries. The output of this program will give a neatly modified Word Document in '.docx' format with the contents of the article.

A python script to scrap articles from Prothom Alo with the Headline, Category, URL, and Summary