Skip to content
A Node.js web app that uses the Watson Discovery News service to query and view the latest trending news items.
JavaScript CSS
Branch: master
Clone or download

README.md

Build Status

Query Watson Discovery News using the Watson Discovery Service

Summary

The web is home to massive amounts of data, with more being created every day. Organizations can harness this constant stream of information to gain understanding, plan strategies, and find opportunities. Enriched news data can help your application make dynamic connections across current events faster. In this exercise, you'll start with the basics and build your own news mining web application using JavaScript, Node.js, and the Watson Discovery service. In this exercise:

  • Code is written in Node.js, with the server-side using the Express framework and the client using ReactJS.
  • You'll use the pre-built Watson Discovery News collection
  • You'll access the Watson Discovery service through the Watson Discovery API

Optionally, you can choose to:

  • Use a Slack interface to query the data
  • Push news alerts out to an RSS reader
  • Host the app on the IBM Cloud

Description

This code pattern shows you how to tap into massive data sets to mine insight. You'll build a news mining web application with the Watson Discovery service using the Watson Node.js SDK. The app demonstrates two use cases using Watson Discovery News:

  • Search: Query for the most relevant new articles about a specific topic or subject. Because the news collection is pre-enriched with natural language processing, you can query not just on keywords or categories but also on concepts, sentiment, and relations to get richer search responses.

  • Trending topics in the news: Identify popular topics over the past 24 hours. Topics can be general, or specific to an industry or category.

Flow

architecture

  1. The user interacts with the Watson Discovery News Server via the app UI.
  2. User input is processed and routed to the Watson Discovery News Server.
  3. The Watson Discovery News Server sends user requests to the Watson Discovery Service.
  4. The Watson Discovery Service queries the Watson News Collection.
  5. The Watson Discovery Service sends news articles to the RSS Reader.
  6. The Watson Discovery Service responds to Slack search requests.

Included components

  • Watson Discovery: A cognitive search and content analytics engine for applications to identify patterns, trends, and actionable insights.

Featured technologies

  • Node.js: An asynchronous event driven JavaScript runtime, designed to build scalable applications
  • React: Javascript library for building User Interfaces
  • Express: A popular and minimalistic web framework for creating API and Web server
  • RSS: RSS (Rich Site Summary) is a format for delivering regularly changing web content in our case it will be trending topics
  • Slack: Slack is a cloud-based set of team collaboration tools and services with chat bot integration
  • Botkit: Framework for creating and managing chat bots

Watch the Video

Deployment options

Click on one of the options below for instructions on deploying the app.

openshift public local

Troubleshooting

If you see the following in the logs for a Deploy to IBM Cloud using Cloud Foundry in the Stage History, click the Redeploy button. This should get past the race condition during the deployment.

Binding service discovery-news-service to app sda-watson-discovery-news-9-4 in org scott.dangelo / space dev as scott.dangelo@ibm.com...
FAILED
Could not bind to service discovery-news-service
Error: Server error, status code: 409, error code: 60016, message: An operation for service instance discovery-news-service is in progress.

Finished: FAILED

Sample output

The trending page:

demo


The query page:

demo


Deep Dive Videos

Here is a series of short videos that go into more of the details about the features of this code pattern.

  • Part 1 - Navigating the UI
  • Part 2 - Code walkthrough
  • Part 3 - Watson Discovery service APIs

Extra Credit

Now that you have built the base application, here are some additional steps you can take to add a Slack interface, and/or provide a way to send new trending news topics to your favorite RSS feed.

1. Configure Slack

To integrate a new Slack Bot into your existing Slack team, navigate to https://<my.slack.com>/apps/manage/custom-integrations, where <my.slack.com> is the Slack workspace you want to customize.

  1. From the Cutsom Integrations page, select the Bots option.

slack-integrations

2, To add a new bot, select the Add Configuration button.

slack-integrations

  1. Enter a username for the bot and click Add bot integration.

slack-bot-name

  1. Once created, save the API Token that is generated.

more-slack-config

2. Configure the application to use the Slack bot

If you runnint the app locally...

  1. Edit the .env file and enter the Slack Bot API Token saved in the previous step.
# Slack
SLACK_BOT_TOKEN=<slack_bot_token>
  1. Restart the application.

If you used the Deploy to IBM Cloud button...

If you used the Deploy to IBM Cloud option, most of the setup is automatic, but not quite all of it. We have to update a few environment variables.

  1. From the IBM Cloud dashboard find the app that was created. Click on the Runtime tab and navigate to the Environment variables tab.

  2. Add and set the Slack Bot API Token saved in the previous step.

environment-variables

  1. Save the new value and restart the application. Watch the logs for errors.

Tip: 💡 Try restaging your application if you see an error.

Search from Slack

The slack bot will respond to certain key words, below is a sample dialog. Remember to @ the bot each time, or start a private chat. Make sure to invite your bot into other channels using /invite @<my bot>.

user: @newsbot hi
newsbot: Hello.

user: @newsbot news please
newsbot: Hi there! What news are you interested in?

user: @newsbot toronto raptors
newsbot: You want me to search for news articles about `toronto raptors`?

user: @newsbot yes
newsbot: OK searching...

slack

Add a Trending Topics RSS Feed rss

On the Trending News Panel, click the RSS Feed button to launch a new tab in your browser. Cut and paste the tab URL into your favorite RSS Reader. If your RSS Feed Reader supports push notifications you can get alerted when trending topics change along with a news article for that topic.


Since RSS feed is a standard way to consume constantly changing data such as news, we can use the RSS feeds we generated to also post news articles to your organizations slack channel to track trends in your industry, or consume the feed to generate a daily digest of news and email in the morning. Other uses may include automatically posting tweets to a Twitter account on news articles on trending topics using a service called IFTTT.

Links

Learn more

  • Artificial Intelligence Code Patterns: Enjoyed this Code Pattern? Check out our other AI Code Patterns.
  • AI and Data Code Pattern Playlist: Bookmark our playlist with all of our Code Pattern videos
  • With Watson: Want to take your Watson app to the next level? Looking to utilize Watson Brand assets? Join the With Watson program to leverage exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.

License

This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.

Apache License FAQ

You can’t perform that action at this time.