Tantular

A toolbox for Indonesian NLP — stateless analysis modules for Indonesian news text. Designed to be used directly by humans and AI agents — no database, no server, no configuration required.

Modules

Module	Class	What it does	Unit of analysis
text.py	TextProcessor	Clean, normalize, extract keywords, TF-IDF, dedup articles	str / List[dict]
ner.py	IndonesianNER	Named entity recognition — PERSON, ORGANIZATION, LOCATION	str (one text)
cooccurrence.py	CooccurrenceNetwork	Entity relationship network via co-occurrence	List[{text, entities}]
entity_resolver.py	EntityResolver	Fuzzy deduplication and canonical naming	List[str] (names)
sentiment.py	InsetSentiment	Lexicon-based polarity — INSET (no model download)	str
emotion.py	EmotionClassifier	Multi-label emotion scores — EmoSense-ID	str
framing.py	PhraseExtractor	Narrative framing phrases around entities	str + entity name
bow.py	BagOfWords	Unigram / n-gram counts, normalized frequencies, cross-document comparison	str / List[str]

---

Installation

From git (recommended for agents)

# Install everything
uv pip install "git+https://github.com/hariswb/tantular[all]"

# Or install only what you need
uv pip install "git+https://github.com/hariswb/tantular[ner,sentiment]"

Available extras: text, sentiment, ner, emotion, framing, all

Local development

git clone https://github.com/hariswb/tantular
cd tantular

# Install with uv (recommended)
uv pip install -e ".[all]"

# Or with plain pip
pip install -e ".[all]"

Dependencies

Always required: none — stdlib only for entity_resolver, cooccurrence, basic text

Extra	Packages
text	beautifulsoup4, langdetect
sentiment	nltk, PySastrawi
ner	transformers, torch
emotion	transformers, torch
framing	nltk

Models are loaded lazily — they are only downloaded and loaded on the first call, not at import time.

---

Usage

Text cleaning

from tantular import TextProcessor

tp = TextProcessor()

# Clean raw text
clean = tp.clean("Presiden Jokowi meresmikan <b>Jalan Tol</b> Cipali!")
# "presiden jokowi meresmikan jalan tol cipali"

# Strip HTML + normalize
text = tp.normalize("<p>Pemerintah   berhasil menurunkan  inflasi.</p>")

# Extract keywords
kw = tp.keywords(text, top_n=5)

# TF-IDF across documents
scores = tp.tfidf(["artikel satu...", "artikel dua..."])

# Clean and deduplicate a list of article dicts
articles = [{"title": "...", "content": "...", "url": "..."}]
cleaned = [tp.clean_article(a) for a in articles]
unique = tp.deduplicate(cleaned)

# Rank articles by relevance to a query
ranked = tp.rank_articles("korupsi nikel", articles, top_n=5)

---

Named Entity Recognition

from tantular import IndonesianNER

ner = IndonesianNER()   # model loads on first call

entities = ner.extract("Jokowi bertemu Prabowo di Istana Negara.")
# [Entity(text='Jokowi', entity_type='PERSON', confidence=0.99, start=0, end=6),
#  Entity(text='Prabowo', entity_type='PERSON', ...),
#  Entity(text='Istana Negara', entity_type='LOCATION', ...)]

# Summary grouped by type
ner.summary(entities)
# {'PERSON': ['Jokowi', 'Prabowo'], 'LOCATION': ['Istana Negara']}

# Batch
results = ner.extract_batch(["artikel satu...", "artikel dua..."])

---

Entity Co-occurrence Network

from tantular import CooccurrenceNetwork

net = CooccurrenceNetwork()

docs = [
    {"text": "Jokowi bertemu Prabowo di Jakarta.", "entities": ["Jokowi", "Prabowo", "Jakarta"]},
    {"text": "Prabowo hadir bersama Jokowi dalam rapat kabinet.", "entities": ["Jokowi", "Prabowo"]},
    {"text": "Jokowi resmikan proyek di Jakarta.", "entities": ["Jokowi", "Jakarta"]},
]

edges = net.compute(docs, window_type="sentence")
# [Edge(source='Jokowi', target='Prabowo', weight=3.84, count=2, evidence='...'),
#  Edge(source='Jokowi', target='Jakarta', weight=1.20, count=2, evidence='...')]

Window types: "sentence" (default), "paragraph", "sliding" (overlapping word windows). Edge weight is the Log-Likelihood (G²) score — corrects for entity frequency.

---

Entity Resolution

from tantular import EntityResolver

resolver = EntityResolver(similarity_threshold=0.85)

# Are two names the same entity?
same, score = resolver.should_merge("PT Bank Mandiri Tbk", "Bank Mandiri")
# (True, 0.91)

# Cluster a flat name list into canonical groups
groups = resolver.group(["Jokowi", "Joko Widodo", "Prabowo Subianto", "Prabowo"])
# [EntityGroup(canonical='Joko Widodo', aliases=['Jokowi'], confidence=0.87),
#  EntityGroup(canonical='Prabowo Subianto', aliases=['Prabowo'], confidence=0.91)]

# Get a replacement map for downstream processing
mapping = resolver.canonical_map(["Jokowi", "Joko Widodo", "Prabowo", "Prabowo Subianto"])
# {'Jokowi': 'Joko Widodo', 'Joko Widodo': 'Joko Widodo', ...}

---

Sentiment Analysis (INSET)

from tantular import InsetSentiment

sa = InsetSentiment()   # lexicon loaded on first call; fully offline

result = sa.analyze("Pemerintah berhasil menurunkan angka kemiskinan secara signifikan.")
# {'label': 'positive', 'polarity': 1.0, 'raw_score': 5.0}

result = sa.analyze("Korupsi merajalela dan merugikan rakyat.")
# {'label': 'negative', 'polarity': -1.0, 'raw_score': -4.0}

batch = sa.analyze_batch(["teks satu", "teks dua", "teks tiga"])

Labels: positive | negative | neutral

---

Emotion Classification (EmoSense-ID)

from tantular import EmotionClassifier

ec = EmotionClassifier()   # model loads on first call

scores = ec.classify("Kami sangat bangga dengan pencapaian luar biasa ini!")
# {'joy': 0.87, 'neutral': 0.08, 'surprise': 0.04, 'anger': 0.01, ...}

# Just the dominant emotion
label = ec.dominant("Tragedi ini membuat seluruh bangsa berduka.")
# 'sadness'

batch = ec.classify_batch(["teks satu", "teks dua"])

---

Framing Extraction

from tantular import PhraseExtractor

pe = PhraseExtractor(window_size=7)

# Single text
phrases = pe.extract(
    "Jokowi meresmikan jalan tol sebagai bagian dari program infrastruktur nasional.",
    entity="Jokowi",
)
# ['meresmikan jalan tol', 'bagian program infrastruktur nasional']

# Across multiple articles — returns ranked phrases with source counts
articles = [
    {"title": "Jokowi resmikan Tol Cipali", "content": "Presiden Joko Widodo meresmikan..."},
    {"title": "Presiden Jokowi tinjau infrastruktur", "content": "Jokowi menekankan pentingnya..."},
]
ranked = pe.extract_from_articles(articles, entity="Jokowi", min_count=1)
# [FramePhrase(phrase='resmikan tol cipali', count=2, sources=['http://...']),
#  FramePhrase(phrase='tinjau infrastruktur', count=1, sources=[...])]

---

Bag of Words

from tantular import BagOfWords

bow = BagOfWords()   # loads stopword list from package data on first call

# Single document
result = bow.analyze("Presiden menegaskan pentingnya swasembada pangan nasional.")
result.top_terms(n=10)
# [('presiden', 1), ('swasembada', 1), ('pangan', 1), ('nasional', 1)]

result.top_ngrams(n=2, top=5)   # bigrams
# [('swasembada pangan', 1), ('pangan nasional', 1)]

result.top_ngrams(n=3, top=5)   # trigrams
# [('swasembada pangan nasional', 1)]

result.vocabulary_size()  # number of unique terms

# Compare a corpus of sections
corpus = bow.compare(["teks satu...", "teks dua...", "teks tiga..."])

corpus.top_global(n=20)     # most frequent terms across all texts
corpus.distinctive(section=0, n=10)  # terms most unique to section 0 (TF-IDF)

# Heatmap data (term × section matrix) for visualisation
hm = corpus.heatmap_data(top_n=30)
# {"terms": [...], "sections": [...], "matrix": [[count, ...], ...]}

---

Full Pipeline Example

from tantular import TextProcessor, IndonesianNER, CooccurrenceNetwork, EntityResolver, InsetSentiment, PhraseExtractor

tp   = TextProcessor()
ner  = IndonesianNER()
net  = CooccurrenceNetwork()
res  = EntityResolver()
sa   = InsetSentiment()
pe   = PhraseExtractor()

articles = [
    {"title": "Jokowi resmikan tol Cipali",       "content": "Presiden Joko Widodo bersama Menteri PUPR meresmikan..."},
    {"title": "Prabowo tinjau alutsista baru",     "content": "Menteri Pertahanan Prabowo Subianto meninjau..."},
    {"title": "Jokowi dan Prabowo bahas pemilu",   "content": "Joko Widodo bertemu Prabowo untuk membahas..."},
]

# 1. Clean
for a in articles:
    a["clean"] = tp.normalize(a["title"] + ". " + a["content"])

# 2. Extract entities per article
for a in articles:
    entities = ner.extract(a["clean"])
    a["entities"] = [e.text for e in entities]

# 3. Resolve duplicates across all articles
all_names = [name for a in articles for name in a["entities"]]
mapping = res.canonical_map(all_names)
for a in articles:
    a["entities"] = list({mapping.get(e, e) for e in a["entities"]})

# 4. Co-occurrence network
docs = [{"text": a["clean"], "entities": a["entities"]} for a in articles]
edges = net.compute(docs, window_type="sentence")

# 5. Sentiment per article
for a in articles:
    a["sentiment"] = sa.analyze(a["clean"])

# 6. Framing for top entity
top_entity = edges[0].source if edges else all_names[0]
frames = pe.extract_from_articles(articles, entity=top_entity)

---

Data Files

INSET lexicon files are bundled in tantular/data/:

File	Contents
positive.tsv	3 609 positive words with weights
negative.tsv	6 609 negative words with weights
stopword-id.csv	756 Indonesian stop words

Source: Indonesia Sentiment Lexicon (INSET)

---

Models

Module	Model	Size	Auto-downloaded
ner.py	cahya/bert-base-indonesian-NER	~450 MB	Yes, on first call
emotion.py	Aardiiiiy/EmoSense-ID-Indonesian-Emotion-Classifier	~500 MB	Yes, on first call
sentiment.py	(none — lexicon only)	—	—