Search strings with special/punctuation characters cause unexpected exceptions

[nkartashov]

While using your lib, I've run into the issue that ValueError is thrown if certain characters such as '(', ')', '[', ']', '$', '?', "'" (apostrophe) and TwitterSearch.TwitterSearchException.TwitterSearchException: Error 401: Unauthorized is produced when I use 'test=', 'test=foo' (basically anytime when I use '=' character). Code producing the aforementioned exceptions (CONSUMER_KEY, CONSUMER_SECRET, TOKEN_KEY, TOKEN_SECRET are keys specific to my application and are working):

Python 2.7.5, TwitterSearch 0.78.3

import logging
import traceback
import TwitterSearch

def download_tweets(search_string, language):
"""Returns list of tweets containing <search_string>, should be like 'en' or 'ru' """

tso = TwitterSearch.TwitterSearchOrder()
tso.addKeyword(search_string)
tso.setLanguage(language)
tso.setIncludeEntities(False)

# create a TwitterSearch object with our secret tokens
ts = TwitterSearch.TwitterSearch(
    consumer_key=CONSUMER_KEY,
    consumer_secret=CONSUMER_SECRET,
    access_token=TOKEN_KEY,
    access_token_secret=TOKEN_SECRET
)
try:
    return ts.searchTweetsIterable(tso)

except TwitterSearch.TwitterSearchException as e:
    logging.exception("%s: %s", e.code, e.message)
    logging.exception("Stack trace: %s", traceback.format_exc())
    raise e

download_tweets("test=", "en")
download_tweets("test=foo", "en")
download_tweets("test'", "en")
download_tweets("test$", "en")
download_tweets("test?", "en")
download_tweets("test(", "en")
download_tweets("test)", "en")
download_tweets("test[", "en")
download_tweets("test]", "en")

[ckoepp]

Looks like there is something wrong with the url encoding as special chars should be encoded before querying the API otherwise Twitter may respond with such errors.

I'll have a look at this issue within the next days. Thanks for reporting this issue!

As a quick and dirty solution: use url encoded strings by yourself by calling download_tweets("test%28","en") instead of download_tweets("test(","en").

[ckoepp]

I just looked at the source quickly and I forgot to apply quote_plus() at line 65 in TwitterSeachOrder.py. Shame on me :p

Will update it and write a test-case to not let foolish stuff like this happen again...