Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Add files via upload
- Loading branch information
1 parent
143f166
commit 52b2f6b
Showing
64 changed files
with
19,444 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
Copyright 2017 IBM Corp. | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the | ||
Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, | ||
and to permit persons to whom the Software is furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF | ||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR | ||
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH | ||
THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,151 @@ | ||
# IBM_Watson_Discovery | ||
This is the code and challenge for "IBM Watson Discovery" By Siraj Raval on Youtube | ||
<h1 align="center" style="border-bottom: none;">🔎 Discovery Demo </h1> | ||
<h3 align="center">Use the IBM Watson Discovery service to add a cognitive search and content analytics engine to your applications to identify patterns, trends and actionable insights that drive better decision-making.</h3> | ||
<p align="center"> | ||
<a href="http://travis-ci.org/watson-developer-cloud/discovery-nodejs"> | ||
<img alt="Travis" src="https://travis-ci.org/watson-developer-cloud/discovery-nodejs.svg?branch=master"> | ||
</a> | ||
<a href="#badge"> | ||
<img alt="semantic-release" src="https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg"> | ||
</a> | ||
</p> | ||
</p> | ||
|
||
![demo](readme-images/new-demo.gif) | ||
|
||
Demo: https://discovery-news-demo.ng.bluemix.net/ | ||
|
||
## Prerequisites | ||
|
||
1. Sign up for an [IBM Cloud account](https://console.bluemix.net/registration/). | ||
1. Download the [IBM Cloud CLI](https://console.bluemix.net/docs/cli/index.html#overview). | ||
1. Create an instance of the Discovery service and get your credentials: | ||
- Go to the [Discovery](https://console.bluemix.net/catalog/services/discovery) page in the IBM Cloud Catalog. | ||
- Log in to your IBM Cloud account. | ||
- Click **Create**. | ||
- Click **Show** to view the service credentials. | ||
- Copy the `apikey` value, or copy the `username` and `password` values if your service instance doesn't provide an `apikey`. | ||
- Copy the `url` value. | ||
|
||
## Configuring the application | ||
|
||
1. In the application folder, copy the *.env.example* file and create a file called *.env* | ||
|
||
``` | ||
cp .env.example .env | ||
``` | ||
|
||
2. Open the *.env* file and add the service credentials that you obtained in the previous step. | ||
|
||
Example *.env* file that configures the `apikey` and `url` for a Discovery service instance hosted in the US East region: | ||
|
||
``` | ||
DISCOVERY_IAM_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2 | ||
DISCOVERY_URL=https://gateway-wdc.watsonplatform.net/discovery/api | ||
``` | ||
|
||
- If your service instance uses `username` and `password` credentials, add the `DISCOVERY_USERNAME` and `DISCOVERY_PASSWORD` variables to the *.env* file. | ||
|
||
Example *.env* file that configures the `username`, `password`, and `url` for a Discovery service instance hosted in the Sydney region: | ||
|
||
``` | ||
DISCOVERY_USERNAME=522be-7b41-ab44-dec3-g1eab2ha73c6 | ||
DISCOVERY_PASSWORD=A4Z5BdGENrwu8 | ||
DISCOVERY_URL=https://gateway-syd.watsonplatform.net/discovery/api | ||
``` | ||
|
||
<!-- **ADD ANY APP-SPECIFIC CONFIGURATION INSTRUCTIONS HERE** --> | ||
|
||
## Running locally | ||
|
||
1. Install the dependencies | ||
|
||
``` | ||
npm install | ||
``` | ||
|
||
1. Build the application | ||
|
||
``` | ||
npm run build | ||
``` | ||
|
||
1. Run the application | ||
|
||
``` | ||
npm start | ||
``` | ||
|
||
1. View the application in a browser at `localhost:3000` | ||
|
||
## Deploying to IBM Cloud as a Cloud Foundry Application | ||
|
||
1. Build the application | ||
|
||
``` | ||
npm run build | ||
``` | ||
|
||
1. Login to IBM Cloud with the [IBM Cloud CLI](https://console.bluemix.net/docs/cli/index.html#overview) | ||
|
||
``` | ||
ibmcloud login | ||
``` | ||
|
||
1. Target a Cloud Foundry organization and space. | ||
|
||
``` | ||
ibmcloud target --cf | ||
``` | ||
|
||
1. Edit the *manifest.yml* file. Change the **name** field to something unique. For example, `- name: my-app-name`. | ||
1. Deploy the application | ||
|
||
``` | ||
ibmcloud app push | ||
``` | ||
|
||
1. View the application online at the app URL, for example: https://my-app-name.mybluemix.net | ||
|
||
|
||
## Tests | ||
|
||
#### Unit tests | ||
Run unit tests with `npm run test-unit`, then `a` to run all tests. See the output for more info. | ||
|
||
#### Integration tests | ||
First you have to make sure your code is built: `npm run build` | ||
|
||
Then run integration tests with: `npm run test-integration-runner` | ||
|
||
## Directory structure | ||
|
||
```none | ||
. | ||
├── app.js // express routes | ||
├── config // express configuration | ||
│ ├── error-handler.js | ||
│ ├── express.js | ||
│ └── security.js | ||
├── package.json | ||
├── public // static resources | ||
├── server.js // entry point | ||
├── test // integration tests | ||
└── src // react client | ||
├── __test__ // unit tests | ||
└── index.js // app entry point | ||
``` | ||
|
||
## License | ||
|
||
This sample code is licensed under the [MIT License](https://opensource.org/licenses/MIT). | ||
|
||
## Contributing | ||
|
||
See [CONTRIBUTING](.github/CONTRIBUTING.md). | ||
|
||
## Open Source @ IBM | ||
Find more open source projects on the [IBM Github Page](http://ibm.github.io/) | ||
|
||
[getting_started]: https://www.ibm.com/watson/developercloud/doc/common/index.html | ||
[docs]: http://www.ibm.com/watson/developercloud/doc/discovery/index.html | ||
[sign_up]: https://console.ng.bluemix.net/registration/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
const queryBuilder = require('./src/query-builder'); | ||
|
||
const NEWS_ENVIRONMENT_ID = 'system'; | ||
const NEWS_COLLECTION_ID = 'news'; | ||
|
||
const DiscoveryV1 = require('watson-developer-cloud/discovery/v1'); | ||
|
||
const discovery = new DiscoveryV1({ | ||
version: '2017-08-01', | ||
url: process.env.DISCOVERY_URL || 'https://gateway.watsonplatform.net/discovery/api', | ||
}); | ||
|
||
// Bootstrap application settings | ||
const express = require('express'); | ||
const path = require('path'); | ||
|
||
const app = express(); | ||
require('./config/express')(app); | ||
|
||
function getWidgetQuery(request) { | ||
const widgetQueries = request.query.widgetQueries; | ||
|
||
if (!widgetQueries) { | ||
return null; | ||
} | ||
|
||
return widgetQueries.split(',').reduce((widgetQuery, finalWidgetQuery) => { | ||
const queryBuilderWidgetQuery = queryBuilder.widgetQueries[widgetQuery]; | ||
|
||
if (queryBuilderWidgetQuery) { | ||
const widgetAggregations = queryBuilderWidgetQuery.aggregations; | ||
|
||
if (widgetAggregations) { | ||
const currentAggregations = finalWidgetQuery.aggregations || []; | ||
delete queryBuilderWidgetQuery.aggregations; | ||
|
||
return Object.assign({}, finalWidgetQuery, queryBuilderWidgetQuery, { | ||
aggregations: currentAggregations.concat(widgetAggregations), | ||
}); | ||
} | ||
} | ||
return Object.assign({}, finalWidgetQuery, queryBuilderWidgetQuery); | ||
}, {}); | ||
} | ||
|
||
app.get('/', (req, res) => { | ||
res.sendFile(path.join(__dirname, 'build', 'index.html')); | ||
}); | ||
|
||
// setup query endpoint for news | ||
app.post('/api/query', (req, res, next) => { | ||
const queryParams = queryBuilder.build(req.body, getWidgetQuery(req)); | ||
|
||
if (queryParams.aggregations) { | ||
queryParams.aggregation = `[${queryParams.aggregations.join(',')}]`; | ||
delete queryParams.aggregations; | ||
} | ||
|
||
const params = Object.assign({}, queryParams, { | ||
environment_id: NEWS_ENVIRONMENT_ID, | ||
collection_id: NEWS_COLLECTION_ID, | ||
}); | ||
|
||
discovery.query(params, (error, response) => { | ||
if (error) { | ||
next(error); | ||
} else { | ||
res.json(response); | ||
} | ||
}); | ||
}); | ||
|
||
// error-handler settings for all other routes | ||
require('./config/error-handler')(app); | ||
|
||
module.exports = app; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
require('dotenv').config(); | ||
const app = require('./app'); | ||
const spawn = require('child_process').spawn; | ||
|
||
if (!process.env.DISCOVERY_USERNAME && !process.env.DISCOVERY_IAM_APIKEY) { | ||
// eslint-disable-next-line no-console | ||
console.log('Skipping integration tests because DISCOVERY_USERNAME and DISCOVERY_IAM_APIKEY are null'); | ||
} else { | ||
const port = 3000; | ||
const server = app.listen(port, () => { | ||
// eslint-disable-next-line no-console | ||
console.log('Server running on port: %d', port); | ||
|
||
function kill(code) { | ||
server.close(() => { | ||
// eslint-disable-next-line no-process-exit | ||
process.exit(code); | ||
}); | ||
} | ||
|
||
function runTests() { | ||
const casper = spawn('npm', ['run', 'test-integration']); | ||
casper.stdout.pipe(process.stdout); | ||
|
||
casper.on('error', (error) => { | ||
// eslint-disable-next-line no-console | ||
console.log(`ERROR: ${error}`); | ||
server.close(() => { | ||
process.exit(1); | ||
}); | ||
}); | ||
|
||
casper.on('close', kill); | ||
} | ||
|
||
runTests(); | ||
}); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
module.exports = (app) => { | ||
// catch 404 and forward to error handler | ||
app.use((req, res, next) => { | ||
const err = new Error('Not Found'); | ||
err.code = 404; | ||
err.message = 'Not Found'; | ||
next(err); | ||
}); | ||
|
||
// error handler | ||
// eslint-disable-next-line no-unused-vars | ||
app.use((err, req, res, next) => { | ||
const error = { | ||
code: err.code || 500, | ||
error: err.error || err.message, | ||
}; | ||
res.status(error.code).json(error); | ||
}); | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
// Module dependencies | ||
const express = require('express'); | ||
const bodyParser = require('body-parser'); | ||
const path = require('path'); | ||
const morgan = require('morgan'); | ||
const security = require('./security'); | ||
|
||
module.exports = function newsApp(app) { | ||
app.use((req, res, next) => { | ||
res.header('Access-Control-Allow-Origin', '*'); | ||
res.header( | ||
'Access-Control-Allow-Headers', | ||
// eslint-disable-next-line comma-dangle | ||
'Origin, X-Requested-With, Content-Type, Accept' | ||
); | ||
|
||
if (req.method === 'OPTIONS') { | ||
res.sendStatus(200); | ||
} else { | ||
next(); | ||
} | ||
}); | ||
app.enable('trust proxy'); | ||
app.use(bodyParser.json({ limit: '1mb' })); | ||
app.use(bodyParser.urlencoded({ extended: false })); | ||
|
||
// Only loaded when running in Bluemix | ||
if (process.env.VCAP_APPLICATION) { | ||
security(app); | ||
} | ||
|
||
app.use(express.static(path.join(__dirname, '..', 'build'))); | ||
app.use(morgan('dev')); | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
// security.js | ||
const secure = require('express-secure-only'); | ||
const helmet = require('helmet'); | ||
const rateLimit = require('express-rate-limit'); | ||
|
||
module.exports = function secureApp(app) { | ||
app.use(secure()); | ||
app.use(helmet()); | ||
|
||
const limiter = rateLimit({ | ||
windowMs: 30 * 1000, // seconds | ||
delayMs: 0, | ||
max: 5, | ||
message: JSON.stringify({ | ||
error: 'Too many requests, please try again in 30 seconds.', | ||
code: 429, | ||
}), | ||
}); | ||
app.use('/api/', limiter); | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
#!/bin/bash | ||
#set -x | ||
#CF_TRACE=true | ||
|
||
# Compute a unique app name using the reserved CF_APP name (configured in the | ||
# deployer or from the manifest.yml file) and the build number | ||
|
||
export TEMP_APP_NAME="discovery-news-demo-crawl-date" | ||
|
||
# write out manifest.yml | ||
echo "Writing manifest.yml..." | ||
|
||
cat > ./manifest.yml <<EOL | ||
name: ${TEMP_APP_NAME} | ||
instances: 1 | ||
memory: 512M | ||
command: npm start | ||
services: | ||
- Discovery-Demo | ||
env: | ||
DEMO_DEPLOY: 1 | ||
EOL | ||
|
||
echo "Manifest file written" | ||
|
||
cat ./manifest.yml | ||
|
||
echo "Pushing new app:$TEMP_APP_NAME" | ||
cf push $TEMP_APP_NAME | ||
export TEMP_APP_URL="https://${TEMP_APP_NAME}.${ROUTE_DOMAIN}" | ||
|
||
echo "##############################################################" | ||
echo "Deployed new app $TEMP_APP_NAME" | ||
echo "on temporary route: $TEMP_APP_URL" | ||
echo "##############################################################" | ||
|
||
# View logs | ||
#cf logs "${TEMP_APP_NAME}" --recent |
Oops, something went wrong.