The following project contains the code for two major case studies that form my senior Capstone project.
The first project aimed to understand public opinion on vernacular education in Malaysia. Ultimately, the author scraped 2580 Malay language articles and 3235 English language articles on vernacular education in Malaysia from 2017-2020 and 2015-2020 respectively. The Excel workbooks contained the raw scraped data with each sheet filtered to remove articles which did not relate to the exploration of the topic to a final 125 and 188 articles for Malay and English respectively.
The webscraper for English articles works signficantly faster than the Malay one because it uses the GoogleNews and Newspaper3k library which, unfortunately, did not work well with the Malay articles. Therefore, those articles had to be scraped using Selenium and BeautifulSoup4 which was significantly slower.
The articles were analysed using sentiment analysis, emotion analysis and topic modelling. The Malay articles all relied on Husein Zolkepli's Malaya library while the English articles used VADER, text2emotion and BERTopic respectively. The results can be viewed within the Jupyter notebooks.
For a full analysis of the results and methods please refer to the paper.
The second project is a web article providing historians' perspective on May 13, a racial riot in Malaysia after the 3rd General Elections in 1969. The web article serves as an example of what an effective public history work could be. You can view the website here.