Skip to content

Commit

Permalink
Feature/version update pi2 (#22)
Browse files Browse the repository at this point in the history
* Feature/fix for circle (#2)

* Updated circle ci config based on https://github.com/mojaloop/central-ledger/blob/master/Onboarding.md documentation

* Updated based on NPM registry migration doc

* Updated circle yaml deployment descriptor

* Added space for consistency of formatting

* Added public access to the npm publish command

* feature/continuous integration v2 (#4)

* final draft circleci v2 config 10:27:47

* final draft circleci v2 config 11:06:58

* Feature/#95 node version upgrade (#6)

* upgraded all dependencies that could be updated. new Buffer() has been deprecated now Buffer.from will be used.
Added to .gitignore idea files and package-lock.json

* changed version

* update circle ci node version

* Feature/#62 automate the cd pipeline for central services shared (#7)

- Automated the deployment through k8s for master and develop push tags
- Automated build artifacts are published to the respective repository for master (release) and develop (snapshot) pushed tags
- Automated unit tests for PR validations with results made available
- Automated integration tests for PR validations with results made available
- Automated functional tests for PR validations with results made available
- Automated code coverage tests for PR validations with results made available to sonar-qube

* Fixed regex in CircleCI v2 config

* Feature/#148 design implement common streaming library (#12)

* added stubs for feature

* reworking the stubs

* reworking the stubs

* Updated Logging feature (#10)

* Updated Logging feature

Added capability to change the logging level
Updated the tests cases
Anded a .env file

* Updated the package.json file

removed unsused dependencies

* Feature/#148 design implement common streaming library (#11)

* initial working simple consumer

* initial working simple consumer

* Consumers working in 3 modes [flow, poll, recursive], updated logger to allow for silly and verbose levels

* updated some comments

* added new process mode for recursive sync queue processing

* adding jsdoc

* added documents, and seperated test (still to be moved into a unit test/integration framework)

* cleaned up code

* cleaned up document

* fixed jsdoc typo

* updated ints to number for jsdoc

* stream processing base for the producer

* Added commit syncs and getWatermarkOffsets

* fixed test and documentation for consumer

* updated config names and docs

* added methods for getMetaData and updated docs

* updated test client

* updated defaults to match documentation

* moving tests to the test folder

* test and commented code for producer

* Updated tape and test coverage config

* changes for miguels comments

* please tell me this is correct

* work in-progress for unit tests

* trying to get poll, recursive functions to work.

* Checking in merged code.

* Implmented all unit tests for consumer

* fixed unit tests, and standards issues

* Updated istanbul config to exclude examples from code coverage check

* fixing setup for configci

* fixing circleci

* added code header for kafkastub

* changes for producer for the send message function including basic unit tests for producer

* unused enum

* added some more tests

* adding unit tests

* added createdAt for metadata protocol

* /#148DesignImplementCommonStreamingLibraryMIG: Auto stash before merge of "feature/#148DesignImplementCommonStreamingLibraryMIG" and "origin/feature/#148DesignImplementCommonStreamingLibrary"

* added some more unit tests to get the code coverage up for the consumer

* Fixed consumer code to be above 90% code coverage

* fixed unit test coverage

* fixed unit test coverage

* fixing tests

* cleaned up consumer tests

* version bump

* proper version bump

* unit tests and coverage above threshold for producer

* Fixed consumer to handle callback function regardless if its a promise or not

* fixing circleci build

* Added test cases for Protocol

Added test cases for protocol.js file

* Implemented review comments

Added back "pp" parameter back to function and fixed unit tests

* Update index.js

* Updated protocol and re-aligned to lime. Re-aligned unit tests.

* Documented TODOs in the JSDoc comments

* cleaned up code

* algined todo comments on jsdoc

* #150 Test, document and address config for "common streaming library" (#14)

* added stubs for feature

* reworking the stubs

* reworking the stubs

* initial working simple consumer

* initial working simple consumer

* Consumers working in 3 modes [flow, poll, recursive], updated logger to allow for silly and verbose levels

* updated some comments

* added new process mode for recursive sync queue processing

* adding jsdoc

* added documents, and seperated test (still to be moved into a unit test/integration framework)

* cleaned up code

* cleaned up document

* fixed jsdoc typo

* updated ints to number for jsdoc

* stream processing base for the producer

* Added commit syncs and getWatermarkOffsets

* fixed test and documentation for consumer

* updated config names and docs

* added methods for getMetaData and updated docs

* updated test client

* updated defaults to match documentation

* Updated Logging feature (#10)

* Updated Logging feature

Added capability to change the logging level
Updated the tests cases
Anded a .env file

* Updated the package.json file

removed unsused dependencies

* moving tests to the test folder

* test and commented code for producer

* Updated tape and test coverage config

* changes for miguels comments

* please tell me this is correct

* work in-progress for unit tests

* trying to get poll, recursive functions to work.

* Checking in merged code.

* Implmented all unit tests for consumer

* fixed unit tests, and standards issues

* Updated istanbul config to exclude examples from code coverage check

* fixing setup for configci

* fixing circleci

* added code header for kafkastub

* changes for producer for the send message function including basic unit tests for producer

* unused enum

* added some more tests

* adding unit tests

* added createdAt for metadata protocol

* /#148DesignImplementCommonStreamingLibraryMIG: Auto stash before merge of "feature/#148DesignImplementCommonStreamingLibraryMIG" and "origin/feature/#148DesignImplementCommonStreamingLibrary"

* added some more unit tests to get the code coverage up for the consumer

* Fixed consumer code to be above 90% code coverage

* fixed unit test coverage

* fixed unit test coverage

* fixing tests

* cleaned up consumer tests

* version bump

* proper version bump

* unit tests and coverage above threshold for producer

* Fixed consumer to handle callback function regardless if its a promise or not

* fixing circleci build

* Added test cases for Protocol

Added test cases for protocol.js file

* Implemented review comments

Added back "pp" parameter back to function and fixed unit tests

* Update index.js

* Updated protocol and re-aligned to lime. Re-aligned unit tests.

* Documented TODOs in the JSDoc comments

* cleaned up code

* algined todo comments on jsdoc

* Updated readme.md to include documentation for Kafka

* Fixed documentation for the Producer example and md.

* Feature/#150 test document and address configfor commonstreaminglibrary (#15)

* added stubs for feature

* reworking the stubs

* reworking the stubs

* initial working simple consumer

* initial working simple consumer

* Consumers working in 3 modes [flow, poll, recursive], updated logger to allow for silly and verbose levels

* updated some comments

* added new process mode for recursive sync queue processing

* adding jsdoc

* added documents, and seperated test (still to be moved into a unit test/integration framework)

* cleaned up code

* cleaned up document

* fixed jsdoc typo

* updated ints to number for jsdoc

* stream processing base for the producer

* Added commit syncs and getWatermarkOffsets

* fixed test and documentation for consumer

* updated config names and docs

* added methods for getMetaData and updated docs

* updated test client

* updated defaults to match documentation

* Updated Logging feature (#10)

* Updated Logging feature

Added capability to change the logging level
Updated the tests cases
Anded a .env file

* Updated the package.json file

removed unsused dependencies

* moving tests to the test folder

* test and commented code for producer

* Updated tape and test coverage config

* changes for miguels comments

* please tell me this is correct

* work in-progress for unit tests

* trying to get poll, recursive functions to work.

* Checking in merged code.

* Implmented all unit tests for consumer

* fixed unit tests, and standards issues

* Updated istanbul config to exclude examples from code coverage check

* fixing setup for configci

* fixing circleci

* added code header for kafkastub

* changes for producer for the send message function including basic unit tests for producer

* unused enum

* added some more tests

* adding unit tests

* added createdAt for metadata protocol

* /#148DesignImplementCommonStreamingLibraryMIG: Auto stash before merge of "feature/#148DesignImplementCommonStreamingLibraryMIG" and "origin/feature/#148DesignImplementCommonStreamingLibrary"

* added some more unit tests to get the code coverage up for the consumer

* Fixed consumer code to be above 90% code coverage

* fixed unit test coverage

* fixed unit test coverage

* fixing tests

* cleaned up consumer tests

* version bump

* proper version bump

* unit tests and coverage above threshold for producer

* Fixed consumer to handle callback function regardless if its a promise or not

* fixing circleci build

* Added test cases for Protocol

Added test cases for protocol.js file

* Implemented review comments

Added back "pp" parameter back to function and fixed unit tests

* Update index.js

* Updated protocol and re-aligned to lime. Re-aligned unit tests.

* Documented TODOs in the JSDoc comments

* cleaned up code

* algined todo comments on jsdoc

* Updated readme.md to include documentation for Kafka

* Fixed documentation for the Producer example and md.

* bumped version up for documentation

* Added doc reference for message protocol (#16)

* added stubs for feature

* reworking the stubs

* reworking the stubs

* initial working simple consumer

* initial working simple consumer

* Consumers working in 3 modes [flow, poll, recursive], updated logger to allow for silly and verbose levels

* updated some comments

* added new process mode for recursive sync queue processing

* adding jsdoc

* added documents, and seperated test (still to be moved into a unit test/integration framework)

* cleaned up code

* cleaned up document

* fixed jsdoc typo

* updated ints to number for jsdoc

* stream processing base for the producer

* Added commit syncs and getWatermarkOffsets

* fixed test and documentation for consumer

* updated config names and docs

* added methods for getMetaData and updated docs

* updated test client

* updated defaults to match documentation

* Updated Logging feature (#10)

* Updated Logging feature

Added capability to change the logging level
Updated the tests cases
Anded a .env file

* Updated the package.json file

removed unsused dependencies

* moving tests to the test folder

* test and commented code for producer

* Updated tape and test coverage config

* changes for miguels comments

* please tell me this is correct

* work in-progress for unit tests

* trying to get poll, recursive functions to work.

* Checking in merged code.

* Implmented all unit tests for consumer

* fixed unit tests, and standards issues

* Updated istanbul config to exclude examples from code coverage check

* fixing setup for configci

* fixing circleci

* added code header for kafkastub

* changes for producer for the send message function including basic unit tests for producer

* unused enum

* added some more tests

* adding unit tests

* added createdAt for metadata protocol

* /#148DesignImplementCommonStreamingLibraryMIG: Auto stash before merge of "feature/#148DesignImplementCommonStreamingLibraryMIG" and "origin/feature/#148DesignImplementCommonStreamingLibrary"

* added some more unit tests to get the code coverage up for the consumer

* Fixed consumer code to be above 90% code coverage

* fixed unit test coverage

* fixed unit test coverage

* fixing tests

* cleaned up consumer tests

* version bump

* proper version bump

* unit tests and coverage above threshold for producer

* Fixed consumer to handle callback function regardless if its a promise or not

* fixing circleci build

* Added test cases for Protocol

Added test cases for protocol.js file

* Implemented review comments

Added back "pp" parameter back to function and fixed unit tests

* Update index.js

* Updated protocol and re-aligned to lime. Re-aligned unit tests.

* Documented TODOs in the JSDoc comments

* cleaned up code

* algined todo comments on jsdoc

* Updated readme.md to include documentation for Kafka

* Fixed documentation for the Producer example and md.

* bumped version up for documentation

* added reference to message protocol docs

* version update

* version update (#21)

* release version update
  • Loading branch information
rmothilal committed Jul 5, 2018
1 parent a52b126 commit 00f3a6e
Show file tree
Hide file tree
Showing 28 changed files with 4,909 additions and 92 deletions.
165 changes: 137 additions & 28 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,66 +1,157 @@
# CircleCI v2 Config
version: 2

defaults: &defaults
defaults_working_directory: &defaults_working_directory
working_directory: /home/circleci/project

defaults_docker_node: &defaults_docker_node
docker:
- image: circleci/node:6.10.3
- image: mhart/alpine-node:8.9.4

defaults_Dependencies: &defaults_Dependencies |
apk --no-cache add git
apk --no-cache add ca-certificates
apk --no-cache add curl
apk --no-cache add openssh-client
apk --no-cache add bash
apk add --no-cache -t build-dependencies make gcc g++ python libtool autoconf automake
npm install -g node-gyp

defaults_awsCliDependencies: &defaults_awsCliDependencies |
apk --no-cache add \
python \
py-pip \
groff \
less \
mailcap
pip install --upgrade awscli==1.14.5 s3cmd==2.0.1 python-magic
apk -v --purge del py-pip

defaults_npm_auth: &defaults_npm_auth
name: Update NPM registry auth token
command: echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc

defaults_npm_publish_version: &defaults_npm_publish
name: Update version to prerelease
command: |
source $BASH_ENV
echo "Publishing tag $CIRCLE_TAG"
npm publish --tag $CIRCLE_TAG --access public
defaults_npm_publish_release: &defaults_npm_publish_release
name: Publish NPM $RELEASE_TAG artifact
command: |
source $BASH_ENV
echo "Publishing tag $RELEASE_TAG"
npm publish --tag $RELEASE_TAG --access public
jobs:
setup:
<<: *defaults
<<: *defaults_working_directory
<<: *defaults_docker_node
steps:
- run:
name: Install general dependencies
command: *defaults_Dependencies
- checkout
- run:
name: Update NPM install
command: sudo npm install
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- node_modules

test-unit:
<<: *defaults
<<: *defaults_working_directory
<<: *defaults_docker_node
steps:
- run:
name: Install general dependencies
command: *defaults_Dependencies
- checkout
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
- run:
name: Install tape, tapes and tap-xunit
command: npm install tape tapes tap-xunit
- run:
name: Create dir for test results
command: mkdir -p ./test/results
- run:
- run:
name: Execute unit tests
command: npm -s run test:xunit > ./test/results/tape.xml
- store_artifacts:
path: ./test/results
prefix: test
- store_test_results:
path: ./test/results

test-coverage:
<<: *defaults
<<: *defaults_working_directory
<<: *defaults_docker_node
steps:
- run:
name: Install general dependencies
command: *defaults_Dependencies
- run:
name: Install AWS CLI dependencies
command: *defaults_awsCliDependencies
- checkout
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
- run:
name: Execute code coverage check
command: npm -s run test:coverage-check
- store_artifacts:
path: coverage
prefix: test
- store_test_results:
path: coverage/lcov.info
deploy:
<<: *defaults
- run:
name: Copy code coverage to SonarQube
command: |
if [ "${CIRCLE_BRANCH}" == "master" ];
then
echo "Sending lcov.info to SonarQube..."
aws s3 cp coverage/lcov.info $AWS_S3_DIR_SONARQUBE/central-services-shared/lcov.info
else
echo "Not a release (env CIRCLE_BRANCH != 'master'), skipping sending lcov.info to SonarQube."
fi
build-snapshot:
<<: *defaults_working_directory
<<: *defaults_docker_node
steps:
- run:
name: Install general dependencies
command: *defaults_Dependencies
- checkout
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: Update NPM registry auth token
command: echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc
- run:
name: Publish NPM artifact
command: npm publish --access public
name: setup environment vars for SNAPSHOT release
command: |
echo 'export RELEASE_TAG=$RELEASE_TAG_SNAPSHOT' >> $BASH_ENV
- run:
<<: *defaults_npm_auth
- run:
<<: *defaults_npm_publish_release

build:
<<: *defaults_working_directory
<<: *defaults_docker_node
steps:
- run:
name: Install general dependencies
command: *defaults_Dependencies
- checkout
- run:
name: setup environment vars for LATEST release
command: |
echo 'export RELEASE_TAG=$RELEASE_TAG_PROD' >> $BASH_ENV
- run:
<<: *defaults_npm_auth
- run:
<<: *defaults_npm_publish_release

workflows:
version: 2
Expand All @@ -69,29 +160,47 @@ workflows:
- setup:
context: org-global
filters:
branches:
ignore:
- /feature*/
- /bugfix*/
tags:
only: /.*/
branches:
ignore:
- /feature*/
- /bugfix*/
- test-unit:
context: org-global
requires:
- setup
filters:
tags:
only: /.*/
branches:
ignore:
ignore:
- /feature*/
- /bugfix*/
- /bugfix*/
- test-coverage:
context: org-global
requires:
- setup
filters:
tags:
only: /.*/
branches:
ignore:
ignore:
- /feature*/
- /bugfix*/
- deploy:
- build-snapshot:
context: org-global
requires:
- setup
- test-unit
- test-coverage
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*\-snapshot+((\.[0-9]+)?)/
branches:
ignore:
- /.*/
- build:
context: org-global
requires:
- setup
Expand All @@ -101,5 +210,5 @@ workflows:
tags:
only: /v[0-9]+(\.[0-9]+)*/
branches:
only:
- master
ignore:
- /.*/
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
LOG_LEVEL = info
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,11 @@ jspm_packages

# Optional REPL history
.node_repl_history

# --------------- #
# IntelliJ #
# --------------- #
.idea/

#package-lock.json
package-lock.json
20 changes: 12 additions & 8 deletions .istanbul.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
instrumentation:
include-all-sources: true
excludes:
[
'**/examples/**'
]
check:
global:
statements: 100
lines: 100
branches: 100
functions: 100
statements: 90
lines: 90
branches: 90
functions: 90
each:
statements: 100
lines: 100
branches: 100
functions: 100
statements: 90
lines: 90
branches: 90
functions: 90
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,8 @@ Logger.info('test message %s', 'my string');

You can also call the Logger.log method which directly calls the Winston log method and gives even more flexibility.

By default, the Logger class is setup to log to the console only, with timestamps and colorized output.
By default, the Logger class is setup to log to the console only, with timestamps and colorized output.

### Kafka

Refer to documentation in [src/kafka/readme.md](./src/kafka/readme.md)
22 changes: 14 additions & 8 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@mojaloop/central-services-shared",
"version": "0.3.0",
"version": "2.0.0-release",
"description": "Shared code for central services",
"main": "src/index.js",
"scripts": {
Expand All @@ -9,6 +9,7 @@
"test:unit": "tapes 'test/**/*.test.js'",
"test:xunit": "npm run test:unit | tap-xunit",
"test:coverage": "istanbul cover tapes -- 'test/**/*.test.js'",
"test:coveragekafka": "istanbul cover tapes -- 'test/kafka/**/*.test.js'",
"test:coverage-check": "npm run test:coverage && istanbul check-coverage"
},
"repository": {
Expand All @@ -27,16 +28,21 @@
},
"homepage": "https://github.com/mojaloop/central-services-shared#readme",
"dependencies": {
"winston": "^2.3.0"
"async": "^2.6.0",
"bluebird": "^3.5.1",
"debug": "^3.1.0",
"events": "^2.0.0",
"node-rdkafka": "^2.3.2",
"winston": "^2.4.0"
},
"devDependencies": {
"faucet": "0.0.1",
"istanbul": "^0.4.5",
"sinon": "^1.17.6",
"standard": "^8.5.0",
"tap-xunit": "^1.4.0",
"tape": "^4.6.2",
"tapes": "^4.1.0"
"istanbul": "^1.1.0-alpha.1",
"sinon": "^4.5.0",
"standard": "^10.0.3",
"tap-xunit": "2.2.0",
"tape": "^4.9.0",
"tapes": "4.1.0"
},
"publishConfig": {
"registry": "https://registry.npmjs.org/"
Expand Down
4 changes: 2 additions & 2 deletions src/encoding.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
'use strict'

exports.toBase64 = (value) => {
return new Buffer(value).toString('base64')
return Buffer.from(value).toString('base64')
}

exports.fromBase64 = (value) => {
return new Buffer(value, 'base64').toString()
return Buffer.from(value, 'base64').toString('utf8')
}
1 change: 1 addition & 0 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ exports.BaseError = require('./errors/base')
exports.ErrorCategory = require('./errors/category')
exports.NotFoundError = require('./errors/not-found')
exports.ValidationError = require('./errors/validation')
exports.Kafka = require('./kafka')
Loading

0 comments on commit 00f3a6e

Please sign in to comment.