The "Panama Papers" were about 12 million leaked documents that shed light upon people around the world who were hiding money in offshore entities through the Panamanian company Mossack Fonseca. The news coverage in the immediate aftermath of the leak tended to focus on a wide range of issues from the technical aspects of a hack to the implications of various world leaders and celebrities who were exposed. In the weeks following the leak, I selected 34 articles about the issue and extracted the text content into text files (see articles below.)
Topic Modeling is an algorithm that finds “a recurring pattern of co-occurring words” in a corpus of text. This does take overall usage of a word across the entire corpus into consideration, so words used often in many documents will not appear in every topic. So, for example, "Panama" will almost certainly be used at least once in every article — the average is about 6.3 times per article. However, "Panama" will only be included in an article's keywords if it's used a significant amount of times and has close usage relationships with other words.
Today you'll skim a few articles and try your best to imitate topic modeling algorithms. Your assigned articles can be found below.
If you're interested, here is the total word usage for all 34 articles (stop words removed).
I used Topic Modeling Tool (TMT), an easy-to-use tool for using MALLET for topic modeling. TMT produces a set of CSV files and a set of HTML files with your output. Take a look at these results with 20 topics. (Remember, these aren't labeled topics, they're clusters of words that likely represent a topic.)
Are there any identifiable "topics" here? Are there any "topics" that don't seem to make sense?
If you click on one of the topics, you'll see the list of documents ordered by how closely each document corresponds with the topic. The number in parentheses is the number of times words in the topic appear in the document. Now click on one of the text files. This will show you the full text of the file, and it will also show the topics that align closely with your topic.
Take a few minutes and explore these results. Click through the network of topics and documents and see if you can find any patterns.
Once you've examined the results with 20 topics, take a look at the same articles run with 40 topics.
What differences do you see between 20 topics and 40 topics? Which set do you think are most useful to you? Why?
- https://www.wordfence.com/blog/2016/04/panama-papers-wordpress-email-connection/
- http://gizmodo.com/is-this-how-a-hacker-got-the-panama-papers-1769836788
- https://www.wordfence.com/blog/2016/04/mossack-fonseca-breach-vulnerable-slider-revolution/
- http://wptavern.com/outdated-and-vulnerable-wordpress-and-drupal-versions-may-have-contributed-to-the-panama-papers-breach
- http://www.theguardian.com/politics/2016/apr/09/david-cameron-to-launch-local-election-campaign-as-panama-papers-row-rumbles-on
- http://www.independent.co.uk/news/world/politics/the-not-completely-crazy-theory-that-russia-leaked-the-panama-papers-a6977271.html
- http://www.independent.co.uk/news/uk/politics/panama-papers-david-cameron-to-announce-tax-taskforce-to-investigate-revelations-as-he-seeks-to-a6976891.html
- http://www.bbc.com/news/world-europe-35975893
- http://time.com/4283587/these-5-facts-explain-the-massive-political-fallout-from-the-panama-papers/
- http://www.nytimes.com/2016/04/06/world/europe/panama-papers-iceland.html
- http://observer.com/2016/04/panama-papers-reveal-clintons-kremlin-connection/
- http://www.vox.com/2016/4/5/11370646/panama-papers-iceland-gunnlaugsson-resigned
- http://latino.foxnews.com/latino/news/2016/04/08/americans-lucked-out-in-panama-papers-scandal-founder-didnt-like-taking-us/
- http://www.nbcnews.com/business/business-news/why-few-americans-panama-papers-lawyer-doesn-t-want-them-n553116
- http://www.nytimes.com/2016/04/08/world/europe/vladimir-putin-panama-papers-american-plot.html
- http://www.theguardian.com/commentisfree/2016/apr/07/panama-papers-taxes-universal-basic-income-public-services
- https://www.rawstory.com/2016/04/heres-why-so-few-americans-appear-in-the-panama-papers/
- http://www.usatoday.com/story/news/2016/04/06/panama-papers-americans-with-past-financial-crimes/82704788/
- http://bigstory.ap.org/article/ef45a25e39224a7989894334aa44bfd4/why-few-americans-panama-papers-lawyer-doesnt-want-them
- http://fusion.net/story/287671/americans-panama-papers-trove/
- http://www.politico.com/agenda/story/2016/04/the-panama-papers-where-are-the-americans-000083
- http://www.nbcnews.com/storyline/panama-papers/why-are-americans-not-included-panama-papers-n551081
- http://fortune.com/2016/04/09/bad-security-panama-papers/
- http://www.forbes.com/sites/thomasbrewster/2016/04/05/panama-papers-amazon-encryption-epic-leak/#47b5058a1df5
- http://www.bbc.com/news/world-latin-america-35975503
- http://www.computerworld.com/article/3052218/security/the-massive-panama-papers-data-leak-explained.html
- http://www.computerweekly.com/news/450280758/Panama-Papers-revealed-by-graph-database-visualisation-software
- http://www.darkreading.com/vulnerabilities—threats/7-lessons-from-the-panama-papers-leak/d/d-id/1324976
- http://www.wired.co.uk/news/archive/2016-04/06/panama-papers-mossack-fonseca-website-security-problems
- http://www.wired.com/2016/04/security-week-panama-papers-law-firm-seriously-shoddy-security/
- http://www.wired.com/2016/04/reporters-pulled-off-panama-papers-biggest-leak-whistleblower-history/
- http://www.nationalreview.com/article/433696/national-security-agency-panama-papers-prove-its-worth?target=author&tid=1022644
- http://thehackernews.com/2016/04/panama-paper-corruption.html
- https://www.helpnetsecurity.com/2016/04/07/panama-papers-lax-security-practices/