Extended Research Project on Media Narratives of Capital City Relocation

This repository contains the code and data for the Extended Research Project (ERP) exploring the evolving media narratives surrounding the capital city relocation in Indonesia. The project employs an ensemble of three methodologies to analyse media coverage: Latent Dirichlet Allocation (LDA) for topic modelling, BERT and RoBERTa algorithms for Named Entity Recognition (NER), and sentiment analysis. These methodologies aim to track the evolution of media discourse, identify key influencers, assess public sentiment, and highlight coverage gaps and biases.

Objectives

1. Track the Media Discourse Evolution:

   • Identify prominent topics and track their change over time through temporal analysis.
   • Detect the emergence, change, or dissolution of discourse clusters.

2. Identify Key Influencers:

   • Determine the key influencers within the media discourse on the capital city relocation to IKN.
   • Identify key persons, their roles/occupations, and their affiliations (government or non-government).

3. Assess Public Sentiment:

   • Conduct sentiment analysis to determine the overall tone (positive, negative, or neutral) of the coverage.
   • Identify shifts in public opinion as reflected in the media.

4. Highlight Coverage Gaps and Biases:

   • Analyse significant coverage gaps or biases in the media portrayal of the capital relocation project.
   • Provide recommendations for more inclusive, balanced, and community-aligned future media coverage.

Data Collection

The dataset for this research was gathered through a database under the Indonesian Ministry of Finance, providing mass media and news data for Public Relations officers. The dataset was filtered using a Boolean search query focusing on Indonesia’s IKN. The dataset consists of the following columns:

• Date: Date of the news article.
• Time: Time when the system captured the news.
• URL : URL of the news article
• Title: Title of the news article.
• Summary: Summary of the news article generated by the system.

Methodologies and Code Structure

Data Exploration and Preprocessing

Code Location: erp.ipynb under "INITIAL EDA," "PREPROCESS," and "SECOND EDA"

• Filters the dataset to include only the 16 media sources specified in the main report and removes the Time column.
• Initial Exploratory Data Analysis (EDA) focuses on general distribution, missing values, and duplicates.
• A second EDA examines temporal fluctuations and the current state of the data.
• Preprocessing creates columns for non-lemmatised and lemmatised tokenised text for further analysis, generating Cleaned_Title, Cleaned_Summary, and text which is created from Title + Summary, along with text_tokenised and text_tokenised_stemmed.

Topic Modelling

Code Location: erp.ipynb under "LDA MODELLING"

• Topic modelling using Latent Dirichlet Allocation (LDA) to uncover hidden topics.
• Trains LDA models with different numbers of topics (2 to 15) to identify the best number of clusters, using the text_tokenised_stemmed column.
• Saves the optimal model, which has 9 clusters, for further analysis.
• Adds a topic column to the dataset to assign the dominant topic to each row.

Named Entity Recognition

Code Location: erp.ipynb under "NER MODELLING"

• Generates a dictionary of actor-role pairs mentioned in the news dataset.
• Applies the best-performing NER model to the dataset, focusing on the Cleaned_Summary column.
• Associates individuals (PER) with their roles (NOR) or organisations (ORG).
• Combined with manual analysis through multiple iterations to build a dictionary of Influencer and Role, which will be cross-referenced with the dataset to generate Extracted_Influencers.
• Introduces a model to determine whether an individual is affiliated with the government or a non-government entity, adding bias_score to the dataset (+1 for each government key person mentioned in Extracted_Influencers, and -1 for each non-government key person mentioned).

Sentiment Analysis

Code Location: erp.ipynb under "SA MODELLING"

• Predicts the sentiment of the news titles using a fine-tuned model trained on a manually labelled dataset.
• Due to large file size, the model.safetensor is not uploaded, but the LABELED_SA.csv used for training is available.
• Adds Tone to the dataset, which reflects the sentiment of the news article.

Discussion Sections

Topic Modelling Results

Code Location: erp.ipynb under "ANALYSIS -> OVERALL TOPICS"

• Descriptive names are assigned to LDA-identified topics for readability.
• Consistent colour coding is applied to visually differentiate topics across plots.
• Adds topic_name to the dataset by determining the name of topic through the most prominent keywords.

Named Entity Recognition Results

Code Location: erp.ipynb under "ANALYSIS -> OVERALL INFLUENCER"

• Analyses the distribution of entities mentioned per article, focusing on the number of key people identified.
• Visualises the number of influencers per article and groups articles mentioning four or more people into one bin.

Sentiment Analysis Results

Code Location: erp.ipynb under "ANALYSIS -> OVERALL TONE"

• Counts the frequency of each sentiment tone in the dataset and visualises it as a pie chart.

Overall Media Landscape

Code Location: erp.ipynb under "ANALYSIS -> MEDIA LANDSCAPE"

• Analyses the media landscape over time, examining topic movement, tone, and bias.
• Highlights key events to provide context to the observed trends.

Positive vs. Negative News Spikes

Code Location: erp.ipynb under "ANALYSIS -> TOPIC FLUCTUATION"

• Analyses spikes in news coverage across topics, focusing on positive and negative tones, generating spike_analysis.csv.
• Identifies prominent keywords, tones, and affiliations during spikes.
• Uses Jaccard similarity to compare keyword overlap in positive vs. negative spikes.
• Visualises monthly fluctuations of topics over time, highlighting significant peaks in coverage.