Skip to content
This repository has been archived by the owner on Jul 27, 2022. It is now read-only.

Commit

Permalink
[+] VERSION 1.0.0 - Copy source sebastien houzet : https://github.com…
Browse files Browse the repository at this point in the history
  • Loading branch information
sebastienhouzet committed May 13, 2019
1 parent f08a1c7 commit 29dcd81
Show file tree
Hide file tree
Showing 8 changed files with 220 additions and 0 deletions.
19 changes: 19 additions & 0 deletions CHANGELOG.md
@@ -0,0 +1,19 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [1.0.0] - 2019-05-03
### Added
- Dockerfile
- Entrypoint.sh
- LICENCE
- README.md
- docker-compose.yml
- CHANGELOG.md
- CONTRIBUTORS.md
- CONTRIBUTING.md
- twint/.gitkeep
76 changes: 76 additions & 0 deletions CONTRIBUTING.md
@@ -0,0 +1,76 @@
# Contributor Covenant Code of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.

## Our Standards

Examples of behavior that contributes to creating a positive environment
include:

* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

## Scope

This Code of Conduct applies within all project spaces, and it also applies when
an individual is representing the project or its community in public spaces.
Examples of representing a project or community include using an official
project e-mail address, posting via an official social media account, or acting
as an appointed representative at an online or offline event. Representation of
a project may be further defined and clarified by project maintainers.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

[homepage]: https://www.contributor-covenant.org

For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq
6 changes: 6 additions & 0 deletions CONTRIBUTORS.md
@@ -0,0 +1,6 @@
# CONTRIBUTORS

## TEAM
Sébastien HOUZET (yoozio.com)

## COMMUNITY
23 changes: 23 additions & 0 deletions Dockerfile
@@ -0,0 +1,23 @@
FROM ubuntu:18.04

MAINTAINER Sébastien Houzet (yoozio.com) <sebastien@yoozio.com>

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

RUN \
apt-get update && \
apt-get install -y \
git \
python3-pip

RUN \
pip3 install --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint

RUN \
apt-get clean autoclean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ENTRYPOINT ["/entrypoint.sh"]
VOLUME /twint
WORKDIR /srv/twint
56 changes: 56 additions & 0 deletions README.md
@@ -0,0 +1,56 @@
# How to use

## First use

For fisrt usage, you need to build image docker.

``` bash
docker-compose build
```

## Elasticsearch and Kibana

Start to up elaticsearch and kibana

``` bash
docker-compose up -d elasticsearch kibana
```

## Execute Twint command

``` bash
docker-compose run -v $PWD/twint:/srv/twint twint {{CMD TWINT}}
```

## Examples of command

A few simple examples to help you understand the basics:

``` bash
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username - Scrape all the Tweets from user's timeline.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username -s pineapple - Scrape all Tweets from the user's timeline containing pineapple.
docker-compose run -v $PWD/twint:/srv/twint twint twint -s pineapple - Collect every Tweet containing pineapple from everyone's Tweets.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --year 2014 - Collect Tweets that were tweeted before 2014.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --since 2015-12-20 - Collect Tweets that were tweeted since 2015-12-20.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username -o file.txt - Scrape Tweets and save to file.txt.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username -o file.csv --csv - Scrape Tweets and save as a csv file.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --email --phone - Show Tweets that might have phone numbers or email addresses.
docker-compose run -v $PWD/twint:/srv/twint twint twint -s "Donald Trump" --verified - Display Tweets by verified users that Tweeted about Donald Trump.
docker-compose run -v $PWD/twint:/srv/twint twint twint -g="48.880048,2.385939,1km" -o file.csv --csv - Scrape Tweets from a radius of 1km around a place in Paris and export them docker-compose run -v $PWD/twint:/srv/twint twint to a csv file.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username -es localhost:9200 - Output Tweets to Elasticsearch
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username -o file.json --json - Scrape Tweets and save as a json file.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --database tweets.db - Save Tweets to a SQLite database.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --followers - Scrape a Twitter user's followers.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --following - Scrape who a Twitter user follows.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --favorites - Collect all the Tweets a user has favorited.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --following --user-full - Collect full user information a person follows
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --profile-full - Use a slow, but effective method to gather Tweets from a user's profile (Gathers ~3200 Tweets, docker-compose run -v $PWD/twint:/srv/twint twint Including Retweets).
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --retweets - Use a quick method to gather the last 900 Tweets (that includes retweets) from a user's profile.
docker-compose run -v $PWD/twint:/srv/twint twint twint -u username --resume 10940389583058 - Resume a search starting from the specified Tweet ID.
```

## Datas

For datas generate by twint, you can found result on folder twint

Let's play now :)
38 changes: 38 additions & 0 deletions docker-compose.yml
@@ -0,0 +1,38 @@
version: '3'
services:
twint:
build: .
container_name: twint

elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.0.1
container_name: twint_elasticsearch
environment:
- node.name=elasticsearch
- cluster.initial_master_nodes=elasticsearch
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata01:/usr/share/elasticsearch/data
ports:
- 9200:9200

kibana:
image: docker.elastic.co/kibana/kibana:7.0.1
container_name: twint_kibana
ports:
- 5601:5601

volumes:
esdata01:
driver: local

networks:
default:
external:
name: nw_twint
2 changes: 2 additions & 0 deletions entrypoint.sh
@@ -0,0 +1,2 @@
#!/bin/bash
$@
Empty file added twint/.gitkeep
Empty file.

0 comments on commit 29dcd81

Please sign in to comment.