Skip to content

Configuration

Francesco Poldi edited this page Dec 10, 2019 · 6 revisions

Configuration Options

Variable             Type       Description
--------------------------------------------
Username             (string) - Twitter user's username
User_id              (string) - Twitter user's user_id
Search               (string) - Search terms
Geo                  (string) - Geo coordinates (lat,lon,km/mi.)
Location             (bool)   - Set to True to attempt to grab a Twitter user's location (slow).
Near                 (string) - Near a certain City (Example: london)
Lang                 (string) - Compatible language codes: https://github.com/twintproject/twint/wiki/Langauge-codes
Output               (string) - Name of the output file.
Elasticsearch        (string) - Elasticsearch instance
Year                 (string) - Filter Tweets before the specified year.
Since                (string) - Filter Tweets sent since date, works only with twint.run.Search (Example: 2017-12-27).
Until                (string) - Filter Tweets sent until date, works only with twint.run.Search (Example: 2017-12-27).
Email                (bool)   - Set to True to show Tweets that _might_ contain emails.
Phone                (bool)   - Set to True to show Tweets that _might_ contain phone numbers.
Verified             (bool)   - Set to True to only show Tweets by _verified_ users
Store_csv            (bool)   - Set to True to write as a csv file.
Store_json           (bool)   - Set to True to write as a json file.
Custom               (dict)   - Custom csv/json formatting (see below).
Show_hashtags        (bool)   - Set to True to show hashtags in the terminal output.
Limit                (int)    - Number of Tweets to pull (Increments of 20).
Count                (bool)   - Count the total number of Tweets fetched.
Stats                (bool)   - Set to True to show Tweet stats in the terminal output.
Database             (string) - Store Tweets in a sqlite3 database.
To                   (string) - Display Tweets tweeted _to_ the specified user.
All                  (string) - Display all Tweets associated with the mentioned user.
Debug                (bool)   - Store information in debug logs.
Format               (string) - Custom terminal output formatting.
Essid                (string) - Elasticsearch session ID.
User_full            (bool)   - Set to True to display full user information. By default, only usernames are shown.
Profile_full         (bool)   - Set to True to use a slow, but effective method to enumerate a user's Timeline.
Store_object         (bool)   - Store tweets/user infos/usernames in JSON objects.
Store_pandas         (bool)   - Save Tweets in a DataFrame (Pandas) file.
Pandas_type          (string) - Specify HDF5 or Pickle (HDF5 as default).
Pandas               (bool)   - Enable Pandas integration.
Index_tweets         (string) - Custom Elasticsearch Index name for Tweets (default: twinttweets).
Index_follow         (string) - Custom Elasticsearch Index name for Follows (default: twintgraph).
Index_users          (string) - Custom Elasticsearch Index name for Users (default: twintuser).
Retries_count        (int)    - Number of retries of requests (default: 10).
Resume               (string) - Resume from the latest scraped tweet ID, specify the filename that contains the ID.
Images               (bool)   - Display only Tweets with images.
Videos               (bool)   - Display only Tweets with videos.
Media                (bool)   - Display Tweets with only images or videos.
Pandas_clean         (bool)   - Automatically clean Pandas dataframe at every scrape.
Lowercase            (bool)   - Automatically convert uppercases in lowercases.
Pandas_au            (bool)   - Automatically update the Pandas dataframe at every scrape.
Proxy_host           (string) - Proxy hostname or IP.
Proxy_port           (int)    - Proxy port.
Proxy_type           (string) - Proxy type.
Tor_control_port     (int)    - Tor control port.
Tor_control_password (string) - Tor control password (not hashed).
Retweets             (bool)   - Get retweets done by the user.
Hide_output          (bool)   - Hide output.
Popular_tweets       (bool)   - Scrape popular tweets, not most recent (default: False).
Skip_certs           (bool)   - Skip certs verification for Elasticsearch, useful for SSC (default: False).
Native_retweets      (bool)   - Filter the results for retweets only (warning: a few tweets will be returned!).
Min_likes            (int)    - Filter the tweets by minimum number of likes.
Min_retweets         (int)    - Filter the tweets by minimum number of retweets.
Min_replies          (int)    - Filter the tweets by minimum number of replies.
Links                (string) - Include or exclude tweets containing one o more links. If not specified you will get both tweets that might contain links or not. (please specify `include` or `exclude`)
Source               (string) - Filter the tweets for specific source client. (example: `--source "Twitter Web Client"`)
Members_list         (string) - Filter the tweets sent by users in a given list.
Filter_retweets      (bool)   - Exclude retweets from the results.

Which are the parameters to set in which scraping function?

.Search

The accepted arguments are:

  • Username
  • User_id
  • Search
  • Geo
  • Location
  • Near
  • Lang
  • Timedelta
  • Year
  • Since
  • Until
  • Email
  • Phone
  • Verified
  • Custom
  • Show_hashtags
  • Show_cashtags
  • Limit
  • Count
  • Stats
  • To
  • All
  • Images
  • Videos
  • Media
  • Lowercase
  • Popular_tweets
  • Native_retweets
  • Min_likes
  • Min_retweets
  • Min_replies
  • Links
  • Source
  • Members_list
  • Filter_retweets
  • Resume
  • Limit

.Following and .Followers

  • Username
  • User_id
  • User_full
  • Custom (only if used User_full)
  • Resume
  • Limit

.Profile

  • Username
  • User_id
  • Retweets
  • Profile_full
  • Custom
  • Limit

.Favorites

  • Limit
  • Username
  • User_id
  • Custom

What about the other parameters?

Other parameters are used for networking options:

  • Proxy_host
  • Proxy_port
  • Proxy_type
  • Tor_control_port
  • Tor_control_password
  • Retries_count

Formatting the output:

  • Format
  • Hide_output
  • Lowercase
  • Stats

Customizing the output (CSV, JSON and plain only):

  • Custom

Specifying the output file (CSV, JSON and plain only):

  • Output

Specifying Elasticsearch attributes:

  • Elasticsearch
  • Index_tweets
  • Index_follow
  • Index_users
  • Skip_certs
  • Essid

Specifying Pandas attributes:

  • Pandas_clean
  • Pandas_au
  • Pandas_type
  • Store_pandas

Specifying SQLite database filename:

  • Database

Storing data in lists:

  • Store_object
  • Store_object_tweets_list

Debugging:

  • Debug
You can’t perform that action at this time.