Topic pages

A topic page is a way of monitoring news content on a particular topic without having to express the topic as a strict boolean query. Instead, you describe the topic with a set of weighted concepts, keywords, categories, sources and locations. Each article or event then receives a score based on how many of the specified conditions it matches and how important they are, and only the results above a chosen threshold are returned - ordered by how well they match the whole topic.

Topic pages can be created and managed interactively on the Event Registry website, or programmatically using the TopicPage class described here.

Creating a topic page

from eventregistry import *
er = EventRegistry(apiKey = YOUR_API_KEY)

topic = TopicPage(er)

# describe the topic with weighted concepts and keywords.
# weights are typically in range 1 - 50; higher weight = more important condition
topic.addConcept(er.getConceptUri("renewable energy"), 50)
topic.addConcept(er.getConceptUri("solar power"), 30)
topic.addKeyword("photovoltaic", 20)
# a required condition - all results HAVE TO be annotated with this category
topic.addCategory(er.getCategoryUri("energy"), 20, required = True)
# an excluded condition - results that match it are removed
topic.addKeyword("horoscope", 0, excluded = True)

# limit the results to English and German articles from the last 3 days
topic.setLanguages(["eng", "deu"])
topic.setMaxDaysBack(3)

# the article has to reach this total weight to be included among the results
topic.setArticleThreshold(50)
topic.setEventThreshold(50)

The methods that can be used to define the topic are:

• addConcept(conceptUri, weight, label = None, conceptType = None, required = False, excluded = False)
• addKeyword(keyword, weight, required = False, excluded = False)
• addCategory(categoryUri, weight, required = False, excluded = False)
• addSource(sourceUri, weight, excluded = False)
• addSourceLocation(sourceLocationUri, weight, excluded = False)
• addSourceGroup(sourceGroupUri, weight, excluded = False)
• addLocation(locationUri, weight)

Setting required = True means that every result has to match this condition; setting excluded = True means that results matching the condition are removed. The corresponding clearConcepts(), clearKeywords(), clearCategories(), clearSources(), clearSourceLocations(), clearSourceGroups() and clearLocations() methods remove the previously added conditions.

Additional filters can be set with:

• setArticleThreshold(value) / setEventThreshold(value): the minimum total weight that an article/event has to reach to be included among the results
• setLanguages(languages): limit the results to the given language(s)
• setMaxDaysBack(maxDaysBack): the maximum allowed age of the results
• setDataTypes(dataTypes): which data types to search - news (default), pr, blog or a list of these
• setSourceRankPercentile(startPercentile, endPercentile): limit the results to sources of a certain ranking
• setSentiment(minSentiment, maxSentiment): limit the results by sentiment
• setArticleIsDuplicateFilter(value), setArticleHasDuplicateFilter(value), setArticleHasEventFilter(value): control the handling of duplicates and of articles without a known event
• restrictToSetConceptsAndKeywords(restrict), restrictToSetCategories(restrict), restrictToSetSources(restrict), restrictToSetLocations(restrict): if set to True, the results have to match at least one of the specified conditions of that type

Getting the results

Once the topic page is defined, use getArticles() and getEvents() to retrieve the matching content:

# get the first page of matching articles, sorted by how well they match the topic
res = topic.getArticles(page = 1, count = 100, sortBy = "rel")
for art in res.get("articles", {}).get("results", []):
    print(art)

# get the matching events
res = topic.getEvents(page = 1, count = 50, sortBy = "rel")

Both methods accept page, count, sortBy, sortByAsc and returnInfo parameters, analogous to the ones described on the searching for articles page.

Saving and loading topic pages

The topic page definition can be saved and restored in several ways:

# save the definition to a python dict / file
definition = topic.saveTopicPageDefinition()
topic.saveTopicPageDefinitionToFile("myTopic.json")

# restore it later
topic2 = TopicPage(er)
topic2.loadTopicPageFromDefinition(definition)
# or
topic2.loadTopicPageFromFile("myTopic.json")

If you have created a topic page in your account on the Event Registry website, you can load it by its URI:

topic = TopicPage(er)
topic.loadTopicPageFromER("your-topic-page-uri")
res = topic.getArticles(page = 1)

The list of topic pages owned by your account can be retrieved using the TopicPages class:

topicPages = TopicPages(er)
myTopicPages = topicPages.getMyTopicPages()