Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker-based tests run via GitHub Actions #233

Merged
merged 4 commits into from
Dec 31, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---

name: CI

on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
build:
runs-on: ubuntu-latest
name: CI

strategy:
fail-fast: false
matrix:
node-version: [15.x, 14.x, 12.x, 10.x]

steps:
- name: Checkout Repository
uses: actions/checkout@v2
with:
fetch-depth: 1

- name: Setup Node ${{ matrix.node-version }}
uses: actions/setup-node@v2
with:
always-auth: false
node-version: ${{ matrix.node-version }}

- name: Start Solr
run: npm run solr:start

- name: Run npm install
run: npm install

- name: Run Tests
run: npm run test
env:
CI: true

- name: Stop Solr
run: npm run solr:stop
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ npm test
Tests are executed against a running SOLR instance, so you might want to:
- install the schema.xml and solrconfig.xml expected by the tests. You find these in test/materials
- make sure your solr instance is running
- specifiy non-default connection params to your server in test/config.json You can inject these also on the command line through:
- specify non-default connection params to your server in test/config.json You can inject these also on the command line through:

```
mocha test/*-test.js --client.core=test-node-client --client.port=8080
Expand Down
13 changes: 13 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
version: '3'
services:
solr:
image: solr:5.5.5
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanted to get it running on the closest version to what we were running it before, and start upgrading later.

ports:
- "8983:8983"
volumes:
- data:/var/solr
command:
- solr-precreate
- testcore
volumes:
data:
130 changes: 130 additions & 0 deletions lib/client.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
const http = require('http'),
https = require('https'),
SolrError = require('./error/solr-error'),
JSONbig = require('json-bigint');

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extracted part of solr.js so that it could be reused elsewhere (in my cases, in schemaHelper)

/**
* Pick appropriate protocol based on the given `secure` flag
*
* @param {Boolean} secure -
* @return {Object} http or https module
* @api private
*/

function pickProtocol(secure){
return secure ? https : http;
}

/**
* Pick appropriate JSON serializer/deserializer library based on the given `bigint` flag
* @param {Boolean} bigint - whenever to handle big number correctly or not (the reason for not using JSONbig all the times is it has an important performance cost)
* @return {Object} JSON or JSONbig serializer/deserializer
* @api private
*/

function pickJSON(bigint){
return bigint ? JSONbig : JSON;
};

/**
* Handle HTTP JSON response from Solr
*
* @param {Function} callback(err,obj) - a function executed when the Solr server responds or an error occurs
* @param {Error} callback().err
* @param {Object} callback().obj - JSON response sent by the Solr server deserialized
*
* @api private
*/

function handleJSONResponse(request, bigint, callback){
return function onJSONResponse(response){
var text = '';
var err = null;
var data = null;

// This properly handles multi-byte characters
response.setEncoding('utf-8');

response.on('data',function(chunk){
text += chunk;
});

response.on('end',function(){
if(response.statusCode < 200 || response.statusCode > 299){
err = new SolrError(request,response,text);
if(callback) callback(err,null);
}else{
try{
data = pickJSON(bigint).parse(text);
}catch(error){
err = error;
}finally{
if(callback) callback(err,data);
}
}
});
};
};


/**
* HTTP POST request. Send update commands to the Solr server (commit, add, delete, optimize)
*
* @param {Object} params
* @param {String} params.host - IP address or host address of the Solr server
* @param {Number|String} params.port - port of the Solr server
* @param {String} params.core - name of the Solr core requested
* @param {String} params.authorization - value of the authorization header
* @param {String} params.fullPath - full path of the request
* @param {String} params.json -
* @param {Boolean} params.secure -
* @param {Boolean} params.bigint -
* @param {http.Agent} [params.agent] -
* @param {Function} callback(err,obj) - a function executed when the Solr server responds or an error occurs
* @param {Error} callback().err
* @param {Object} callback().obj - JSON response sent by the Solr server deserialized
*
* @return {http.ClientRequest}
* @api private
*/function postJSON(params,callback){
var headers = {
'content-type' : 'application/json; charset=utf-8',
'content-length': Buffer.byteLength(params.json),
'accept' : 'application/json; charset=utf-8'
};
if(params.authorization){
headers['authorization'] = params.authorization;
}
var options = {
host : params.host,
port : params.port,
method : 'POST',
headers : headers,
path : params.fullPath,
family : params.ipversion
};

if(params.agent !== undefined){
options.agent = params.agent;
}

var request = pickProtocol(params.secure).request(options);

request.on('response', handleJSONResponse(request, params.bigint, callback));

request.on('error',function onError(err){
if (callback) callback(err,null);
});

request.write(params.json);
request.end();

return request;
};

module.exports = {
handleJSONResponse,
pickJSON,
pickProtocol,
postJSON
}
127 changes: 3 additions & 124 deletions lib/solr.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,18 @@
*/

var http = require('http'),
https = require('https'),
Query = require('./query'),
Collection = require('./collection'),
querystring = require('querystring'),
format = require('./utils/format'),
SolrError = require('./error/solr-error'),
JSONStream = require('JSONStream'),
duplexer = require('duplexer'),
request = require('request'),
JSONbig = require('json-bigint'),
versionUtils = require('./utils/version'),
Promise = require('bluebird');

const { handleJSONResponse, pickJSON, pickProtocol, postJSON } = require('./client')

/**
* Expose `createClient()`.
*/
Expand Down Expand Up @@ -772,87 +771,6 @@ Client.prototype.ping = function(callback){
return this.get(this.ADMIN_PING_HANDLER, callback);
}

/**
* Pick appropriate protocol based on the given `secure` flag
*
* @param {Boolean} secure -
* @return {Object} http or https module
* @api private
*/

function pickProtocol(secure){
return secure ? https : http;
};


/**
* Pick appropriate JSON serializer/deserializer library based on the given `bigint` flag
* @param {Boolean} bigint - whenever to handle big number correctly or not (the reason for not using JSONbig all the times is it has an important performance cost)
* @return {Object} JSON or JSONbig serializer/deserializer
* @api private
*/

function pickJSON(bigint){
return bigint ? JSONbig : JSON;
};

/**
* HTTP POST request. Send update commands to the Solr server (commit, add, delete, optimize)
*
* @param {Object} params
* @param {String} params.host - IP address or host address of the Solr server
* @param {Number|String} params.port - port of the Solr server
* @param {String} params.core - name of the Solr core requested
* @param {String} params.authorization - value of the authorization header
* @param {String} params.fullPath - full path of the request
* @param {String} params.json -
* @param {Boolean} params.secure -
* @param {Boolean} params.bigint -
* @param {http.Agent} [params.agent] -
* @param {Function} callback(err,obj) - a function executed when the Solr server responds or an error occurs
* @param {Error} callback().err
* @param {Object} callback().obj - JSON response sent by the Solr server deserialized
*
* @return {http.ClientRequest}
* @api private
*/

function postJSON(params,callback){
var headers = {
'content-type' : 'application/json; charset=utf-8',
'content-length': Buffer.byteLength(params.json),
'accept' : 'application/json; charset=utf-8'
};
if(params.authorization){
headers['authorization'] = params.authorization;
}
var options = {
host : params.host,
port : params.port,
method : 'POST',
headers : headers,
path : params.fullPath,
family : params.ipversion
};

if(params.agent !== undefined){
options.agent = params.agent;
}

var request = pickProtocol(params.secure).request(options);

request.on('response', handleJSONResponse(request, params.bigint, callback));

request.on('error',function onError(err){
if (callback) callback(err,null);
});

request.write(params.json);
request.end();

return request;
};

/**
* HTTP POST request. Send update commands to the Solr server using form encoding (e.g. search)
*
Expand Down Expand Up @@ -963,44 +881,5 @@ function getJSON(params,callback){
return request;
};

/**
* Handle HTTP JSON response from Solr
*
* @param {Function} callback(err,obj) - a function executed when the Solr server responds or an error occurs
* @param {Error} callback().err
* @param {Object} callback().obj - JSON response sent by the Solr server deserialized
*
* @api private
*/

function handleJSONResponse(request, bigint, callback){
return function onJSONResponse(response){
var text = '';
var err = null;
var data = null;

// This properly handles multi-byte characters
response.setEncoding('utf-8');

response.on('data',function(chunk){
text += chunk;
});

response.on('end',function(){
if(response.statusCode < 200 || response.statusCode > 299){
err = new SolrError(request,response,text);
if(callback) callback(err,null);
}else{
try{
data = pickJSON(bigint).parse(text);
}catch(error){
err = error;
}finally{
if(callback) callback(err,data);
}
}
});
};
};

Promise.promisifyAll(Client.prototype);
Promise.promisifyAll(Client.prototype);
Loading