Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Add files via upload
  • Loading branch information
llSourcell committed Dec 5, 2018
1 parent 143f166 commit 52b2f6b
Show file tree
Hide file tree
Showing 64 changed files with 19,444 additions and 2 deletions.
12 changes: 12 additions & 0 deletions LICENSE
@@ -0,0 +1,12 @@
Copyright 2017 IBM Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the
Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH
THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
153 changes: 151 additions & 2 deletions README.md
@@ -1,2 +1,151 @@
# IBM_Watson_Discovery
This is the code and challenge for "IBM Watson Discovery" By Siraj Raval on Youtube
<h1 align="center" style="border-bottom: none;">🔎 Discovery Demo </h1>
<h3 align="center">Use the IBM Watson Discovery service to add a cognitive search and content analytics engine to your applications to identify patterns, trends and actionable insights that drive better decision-making.</h3>
<p align="center">
<a href="http://travis-ci.org/watson-developer-cloud/discovery-nodejs">
<img alt="Travis" src="https://travis-ci.org/watson-developer-cloud/discovery-nodejs.svg?branch=master">
</a>
<a href="#badge">
<img alt="semantic-release" src="https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg">
</a>
</p>
</p>

![demo](readme-images/new-demo.gif)

Demo: https://discovery-news-demo.ng.bluemix.net/

## Prerequisites

1. Sign up for an [IBM Cloud account](https://console.bluemix.net/registration/).
1. Download the [IBM Cloud CLI](https://console.bluemix.net/docs/cli/index.html#overview).
1. Create an instance of the Discovery service and get your credentials:
- Go to the [Discovery](https://console.bluemix.net/catalog/services/discovery) page in the IBM Cloud Catalog.
- Log in to your IBM Cloud account.
- Click **Create**.
- Click **Show** to view the service credentials.
- Copy the `apikey` value, or copy the `username` and `password` values if your service instance doesn't provide an `apikey`.
- Copy the `url` value.

## Configuring the application

1. In the application folder, copy the *.env.example* file and create a file called *.env*

```
cp .env.example .env
```

2. Open the *.env* file and add the service credentials that you obtained in the previous step.

Example *.env* file that configures the `apikey` and `url` for a Discovery service instance hosted in the US East region:

```
DISCOVERY_IAM_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2
DISCOVERY_URL=https://gateway-wdc.watsonplatform.net/discovery/api
```

- If your service instance uses `username` and `password` credentials, add the `DISCOVERY_USERNAME` and `DISCOVERY_PASSWORD` variables to the *.env* file.

Example *.env* file that configures the `username`, `password`, and `url` for a Discovery service instance hosted in the Sydney region:

```
DISCOVERY_USERNAME=522be-7b41-ab44-dec3-g1eab2ha73c6
DISCOVERY_PASSWORD=A4Z5BdGENrwu8
DISCOVERY_URL=https://gateway-syd.watsonplatform.net/discovery/api
```

<!-- **ADD ANY APP-SPECIFIC CONFIGURATION INSTRUCTIONS HERE** -->

## Running locally

1. Install the dependencies

```
npm install
```

1. Build the application

```
npm run build
```

1. Run the application

```
npm start
```

1. View the application in a browser at `localhost:3000`

## Deploying to IBM Cloud as a Cloud Foundry Application

1. Build the application

```
npm run build
```

1. Login to IBM Cloud with the [IBM Cloud CLI](https://console.bluemix.net/docs/cli/index.html#overview)

```
ibmcloud login
```

1. Target a Cloud Foundry organization and space.

```
ibmcloud target --cf
```

1. Edit the *manifest.yml* file. Change the **name** field to something unique. For example, `- name: my-app-name`.
1. Deploy the application

```
ibmcloud app push
```

1. View the application online at the app URL, for example: https://my-app-name.mybluemix.net


## Tests

#### Unit tests
Run unit tests with `npm run test-unit`, then `a` to run all tests. See the output for more info.

#### Integration tests
First you have to make sure your code is built: `npm run build`

Then run integration tests with: `npm run test-integration-runner`

## Directory structure

```none
.
├── app.js // express routes
├── config // express configuration
│ ├── error-handler.js
│ ├── express.js
│ └── security.js
├── package.json
├── public // static resources
├── server.js // entry point
├── test // integration tests
└── src // react client
├── __test__ // unit tests
└── index.js // app entry point
```

## License

This sample code is licensed under the [MIT License](https://opensource.org/licenses/MIT).

## Contributing

See [CONTRIBUTING](.github/CONTRIBUTING.md).

## Open Source @ IBM
Find more open source projects on the [IBM Github Page](http://ibm.github.io/)

[getting_started]: https://www.ibm.com/watson/developercloud/doc/common/index.html
[docs]: http://www.ibm.com/watson/developercloud/doc/discovery/index.html
[sign_up]: https://console.ng.bluemix.net/registration/
76 changes: 76 additions & 0 deletions app.js
@@ -0,0 +1,76 @@
const queryBuilder = require('./src/query-builder');

const NEWS_ENVIRONMENT_ID = 'system';
const NEWS_COLLECTION_ID = 'news';

const DiscoveryV1 = require('watson-developer-cloud/discovery/v1');

const discovery = new DiscoveryV1({
version: '2017-08-01',
url: process.env.DISCOVERY_URL || 'https://gateway.watsonplatform.net/discovery/api',
});

// Bootstrap application settings
const express = require('express');
const path = require('path');

const app = express();
require('./config/express')(app);

function getWidgetQuery(request) {
const widgetQueries = request.query.widgetQueries;

if (!widgetQueries) {
return null;
}

return widgetQueries.split(',').reduce((widgetQuery, finalWidgetQuery) => {
const queryBuilderWidgetQuery = queryBuilder.widgetQueries[widgetQuery];

if (queryBuilderWidgetQuery) {
const widgetAggregations = queryBuilderWidgetQuery.aggregations;

if (widgetAggregations) {
const currentAggregations = finalWidgetQuery.aggregations || [];
delete queryBuilderWidgetQuery.aggregations;

return Object.assign({}, finalWidgetQuery, queryBuilderWidgetQuery, {
aggregations: currentAggregations.concat(widgetAggregations),
});
}
}
return Object.assign({}, finalWidgetQuery, queryBuilderWidgetQuery);
}, {});
}

app.get('/', (req, res) => {
res.sendFile(path.join(__dirname, 'build', 'index.html'));
});

// setup query endpoint for news
app.post('/api/query', (req, res, next) => {
const queryParams = queryBuilder.build(req.body, getWidgetQuery(req));

if (queryParams.aggregations) {
queryParams.aggregation = `[${queryParams.aggregations.join(',')}]`;
delete queryParams.aggregations;
}

const params = Object.assign({}, queryParams, {
environment_id: NEWS_ENVIRONMENT_ID,
collection_id: NEWS_COLLECTION_ID,
});

discovery.query(params, (error, response) => {
if (error) {
next(error);
} else {
res.json(response);
}
});
});

// error-handler settings for all other routes
require('./config/error-handler')(app);

module.exports = app;
38 changes: 38 additions & 0 deletions casper-runner.js
@@ -0,0 +1,38 @@
require('dotenv').config();
const app = require('./app');
const spawn = require('child_process').spawn;

if (!process.env.DISCOVERY_USERNAME && !process.env.DISCOVERY_IAM_APIKEY) {
// eslint-disable-next-line no-console
console.log('Skipping integration tests because DISCOVERY_USERNAME and DISCOVERY_IAM_APIKEY are null');
} else {
const port = 3000;
const server = app.listen(port, () => {
// eslint-disable-next-line no-console
console.log('Server running on port: %d', port);

function kill(code) {
server.close(() => {
// eslint-disable-next-line no-process-exit
process.exit(code);
});
}

function runTests() {
const casper = spawn('npm', ['run', 'test-integration']);
casper.stdout.pipe(process.stdout);

casper.on('error', (error) => {
// eslint-disable-next-line no-console
console.log(`ERROR: ${error}`);
server.close(() => {
process.exit(1);
});
});

casper.on('close', kill);
}

runTests();
});
}
19 changes: 19 additions & 0 deletions config/error-handler.js
@@ -0,0 +1,19 @@
module.exports = (app) => {
// catch 404 and forward to error handler
app.use((req, res, next) => {
const err = new Error('Not Found');
err.code = 404;
err.message = 'Not Found';
next(err);
});

// error handler
// eslint-disable-next-line no-unused-vars
app.use((err, req, res, next) => {
const error = {
code: err.code || 500,
error: err.error || err.message,
};
res.status(error.code).json(error);
});
};
34 changes: 34 additions & 0 deletions config/express.js
@@ -0,0 +1,34 @@
// Module dependencies
const express = require('express');
const bodyParser = require('body-parser');
const path = require('path');
const morgan = require('morgan');
const security = require('./security');

module.exports = function newsApp(app) {
app.use((req, res, next) => {
res.header('Access-Control-Allow-Origin', '*');
res.header(
'Access-Control-Allow-Headers',
// eslint-disable-next-line comma-dangle
'Origin, X-Requested-With, Content-Type, Accept'
);

if (req.method === 'OPTIONS') {
res.sendStatus(200);
} else {
next();
}
});
app.enable('trust proxy');
app.use(bodyParser.json({ limit: '1mb' }));
app.use(bodyParser.urlencoded({ extended: false }));

// Only loaded when running in Bluemix
if (process.env.VCAP_APPLICATION) {
security(app);
}

app.use(express.static(path.join(__dirname, '..', 'build')));
app.use(morgan('dev'));
};
20 changes: 20 additions & 0 deletions config/security.js
@@ -0,0 +1,20 @@
// security.js
const secure = require('express-secure-only');
const helmet = require('helmet');
const rateLimit = require('express-rate-limit');

module.exports = function secureApp(app) {
app.use(secure());
app.use(helmet());

const limiter = rateLimit({
windowMs: 30 * 1000, // seconds
delayMs: 0,
max: 5,
message: JSON.stringify({
error: 'Too many requests, please try again in 30 seconds.',
code: 429,
}),
});
app.use('/api/', limiter);
};
39 changes: 39 additions & 0 deletions deploy.sh
@@ -0,0 +1,39 @@
#!/bin/bash
#set -x
#CF_TRACE=true

# Compute a unique app name using the reserved CF_APP name (configured in the
# deployer or from the manifest.yml file) and the build number

export TEMP_APP_NAME="discovery-news-demo-crawl-date"

# write out manifest.yml
echo "Writing manifest.yml..."

cat > ./manifest.yml <<EOL
name: ${TEMP_APP_NAME}
instances: 1
memory: 512M
command: npm start
services:
- Discovery-Demo
env:
DEMO_DEPLOY: 1
EOL

echo "Manifest file written"

cat ./manifest.yml

echo "Pushing new app:$TEMP_APP_NAME"
cf push $TEMP_APP_NAME
export TEMP_APP_URL="https://${TEMP_APP_NAME}.${ROUTE_DOMAIN}"

echo "##############################################################"
echo "Deployed new app $TEMP_APP_NAME"
echo "on temporary route: $TEMP_APP_URL"
echo "##############################################################"

# View logs
#cf logs "${TEMP_APP_NAME}" --recent

0 comments on commit 52b2f6b

Please sign in to comment.