Feature/cf api model layer#135
Conversation
|
@tara-gibbs @ongk FYI.. first cut at a layered approach |
|
From Jenkins: There was a test failure while running Jenkins tests. |
| * @returns {string} the query string | ||
| * @private | ||
| */ | ||
| function makeQueryString(options) { |
There was a problem hiding this comment.
This can probably be replaced by $httpParamSerializer later on: https://code.angularjs.org/1.4.9/docs/api/ng/service/$httpParamSerializer
There was a problem hiding this comment.
yeah I suspect that function will die anyway when we pull in the new generated stuff frankly.
There was a problem hiding this comment.
It's doing a bit more than just serializing the parameters. Also, I'm fairly certain that the order of the params actually matters to the v2 CF API (I know, right?) so the filter param has to be last.
There was a problem hiding this comment.
We shouldn't have a closure function defined in class constructor. If this is helper function, you can define it to the buttom of this file, out of the class constructor definition.
|
|
||
| var collectionName = genericCollections[j]; | ||
| function config($httpProvider) { | ||
| $httpProvider.interceptors.push(interceptor); |
There was a problem hiding this comment.
This interceptor has been added in the core part of the UI platform, adding it here will emit those events twice. We do not need this config call.
|
@wchrisjohnson @woodm1979 this is a hopefully simple example of the view -> model -> api pattern we are trying to adopt. |
|
From Jenkins: There was a test failure while running Jenkins tests. |
| registerApplicationApi.$inject = [ | ||
| '$http', | ||
| '$q', | ||
| '$cookies', |
There was a problem hiding this comment.
Do we need $cookie and $q in this API class?
|
From Jenkins: There was a test failure while running Jenkins tests. |
|
From Jenkins: There was a test failure while running Jenkins tests. |
| var that = this; | ||
| this.application = modelManager.retrieve('cloud-foundry.model.application'); | ||
| this.application.all().then(function () { | ||
| console.log(that.application.data.applications); |
There was a problem hiding this comment.
Don't need this anymore: just this.application.all()
|
Can you do me a favor and update the title of this PR so it's not WIP? Maybe prepend TEAMFOUR-137. I keep thinking you're still actively working on it so haven't reviewed much. |
|
new commit pulls out all the functions that wont be used, other than "all" and fixes lint errs |
Feature/cf api model layer
* update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin
commit 5dad190 Merge: 3b13b8e 044208f Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 15:06:44 2017 +0100 Merge remote-tracking branch 'origin/suse' into remove-old-refs commit 3b13b8e Author: Neil MacDougall <nwmac@users.noreply.github.com> Date: Thu Jun 1 14:55:30 2017 +0100 gate check commit 044208f Author: Irfan Habib <irfan.habib@suse.com> Date: Thu Jun 1 14:53:19 2017 +0100 Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin commit 1e765c2 Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 14:12:27 2017 +0100 gate check commit 5c0c89c Author: Richard Cox <richard.cox@suse.com> Date: Thu Jun 1 10:35:22 2017 +0100 Updated class name app --> console commit b6c6021 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 16:15:00 2017 +0100 e2e fix commit b2c765a Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:49:45 2017 +0100 Remove old company name commit 6f03e32 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:33:47 2017 +0100 Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation commit 6b4d47d Merge: 7f90d7e 2e01010 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:27:04 2017 +0100 Merge branch 'suse' into remove-old-refs commit 7f90d7e Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:07:38 2017 +0100 Removed references to HPE commit 97f6427 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 12:02:02 2017 +0100 Remove references to stackato commit 787eb9b Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 10:49:46 2017 +0100 Tweak readme commit 82f751d Author: Neil MacDougall <neil.macdougall@suse.com> Date: Sat May 27 18:38:20 2017 +0100 gate check commit c9cb6a2 Merge: bac7625 05cb232 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:35:27 2017 +0100 Merge branch 'suse' into remove-helion-refs commit bac7625 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:31:19 2017 +0100 Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead commit c43edb9 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 16:58:09 2017 +0100 Rename module helion.framework to app.framework
* Fixed translation typos * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
#1061) * Move brand scss into src file so that it is watched for changes. Small branding updates. * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
* Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Remove references to hcf (does not cover hcf cnsi_type) * Undo endpoints.hcf --> endpoints.cf (object comes from response from backend) * Fix failing e2e tests * CI bump * Fix failing e2e * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check * CI bump
commit 5dad190 Merge: 3b13b8e 044208f Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 15:06:44 2017 +0100 Merge remote-tracking branch 'origin/suse' into remove-old-refs commit 3b13b8e Author: Neil MacDougall <nwmac@users.noreply.github.com> Date: Thu Jun 1 14:55:30 2017 +0100 gate check commit 044208f Author: Irfan Habib <irfan.habib@suse.com> Date: Thu Jun 1 14:53:19 2017 +0100 Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin commit 1e765c2 Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 14:12:27 2017 +0100 gate check commit 5c0c89c Author: Richard Cox <richard.cox@suse.com> Date: Thu Jun 1 10:35:22 2017 +0100 Updated class name app --> console commit b6c6021 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 16:15:00 2017 +0100 e2e fix commit b2c765a Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:49:45 2017 +0100 Remove old company name commit 6f03e32 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:33:47 2017 +0100 Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation commit 6b4d47d Merge: 7f90d7e 2e01010 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:27:04 2017 +0100 Merge branch 'suse' into remove-old-refs commit 7f90d7e Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:07:38 2017 +0100 Removed references to HPE commit 97f6427 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 12:02:02 2017 +0100 Remove references to stackato commit 787eb9b Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 10:49:46 2017 +0100 Tweak readme commit 82f751d Author: Neil MacDougall <neil.macdougall@suse.com> Date: Sat May 27 18:38:20 2017 +0100 gate check commit c9cb6a2 Merge: bac7625 05cb232 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:35:27 2017 +0100 Merge branch 'suse' into remove-helion-refs commit bac7625 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:31:19 2017 +0100 Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead commit c43edb9 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 16:58:09 2017 +0100 Rename module helion.framework to app.framework
* Fixed translation typos * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
#1061) * Move brand scss into src file so that it is watched for changes. Small branding updates. * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
* Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Remove references to hcf (does not cover hcf cnsi_type) * Undo endpoints.hcf --> endpoints.cf (object comes from response from backend) * Fix failing e2e tests * CI bump * Fix failing e2e * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check * CI bump
commit 5dad190 Merge: 3b13b8e 044208f Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 15:06:44 2017 +0100 Merge remote-tracking branch 'origin/suse' into remove-old-refs commit 3b13b8e Author: Neil MacDougall <nwmac@users.noreply.github.com> Date: Thu Jun 1 14:55:30 2017 +0100 gate check commit 044208f Author: Irfan Habib <irfan.habib@suse.com> Date: Thu Jun 1 14:53:19 2017 +0100 Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin commit 1e765c2 Author: Neil MacDougall <neil.macdougall@suse.com> Date: Thu Jun 1 14:12:27 2017 +0100 gate check commit 5c0c89c Author: Richard Cox <richard.cox@suse.com> Date: Thu Jun 1 10:35:22 2017 +0100 Updated class name app --> console commit b6c6021 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 16:15:00 2017 +0100 e2e fix commit b2c765a Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:49:45 2017 +0100 Remove old company name commit 6f03e32 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 15:33:47 2017 +0100 Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation commit 6b4d47d Merge: 7f90d7e 2e01010 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:27:04 2017 +0100 Merge branch 'suse' into remove-old-refs commit 7f90d7e Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 14:07:38 2017 +0100 Removed references to HPE commit 97f6427 Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 12:02:02 2017 +0100 Remove references to stackato commit 787eb9b Author: Richard Cox <richard.cox@suse.com> Date: Tue May 30 10:49:46 2017 +0100 Tweak readme commit 82f751d Author: Neil MacDougall <neil.macdougall@suse.com> Date: Sat May 27 18:38:20 2017 +0100 gate check commit c9cb6a2 Merge: bac7625 05cb232 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:35:27 2017 +0100 Merge branch 'suse' into remove-helion-refs commit bac7625 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 17:31:19 2017 +0100 Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead commit c43edb9 Author: Richard Cox <richard.cox@suse.com> Date: Fri May 26 16:58:09 2017 +0100 Rename module helion.framework to app.framework
* Fixed translation typos * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
#1061) * Move brand scss into src file so that it is watched for changes. Small branding updates. * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check
* Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Remove references to hcf (does not cover hcf cnsi_type) * Undo endpoints.hcf --> endpoints.cf (object comes from response from backend) * Fix failing e2e tests * CI bump * Fix failing e2e * Merge Portal Proxy (#1059) * update logging to use appropriate levels. remove logging that exposed sensitive info. remove extraneous debugging logs * drop unneeded headers var * update glide deps to include logrus * update new log entries from rebase to use logger instead of log * DRY up error messages * prevent API from registering same cluster multiple times * rename and return only bool for existing CNSI endpoint check function * throw a regular Error instead of a Fatal if VCS Clients can't be parsed * List VCS clients endpoint * Using clientNotFound template in handleVCSAuth * Use logger instead of log * Refactor the use of inline markup to templates * Copt templates to container; fix file paths * fix logger call * Update the template names for clarity * Fixed logger statements * Initial commit Initial commit to get etcd stood up in a cluster * Finalize initial version of the init script & Dockerfile * Clean up and document init script * Convert LONGGGG command to multiline * Rename the Dockerfile * Fix paths * Rename folder * Rename refs to etcd folder * create PGStore mock * update tests to use mocked db returned from setupHTTPTest() * drop logging from mocks, make sure StoreSession.Options is initialized in mock * add SessionStorer interface * Expose pgsql EncryptToken function so we can use it in our tests * use insertErr instead of err * expand from 3-5 instances * Remove ENV VAR based approach * TEAMFOUR-541 Implement stolon for Postgres HA This change adds stolon related container assets to the proxy project for the sake of Postgres HA. - cut over to use stolon proxy instead of single user Postgres - DRY up params in SDL - remove ENV VARs from postflight Dockerfile - DRY up ENV VARs in the postflight script - rename the stratos user/pwd to stackato - update the README - tighten up connection string - override the default stolon startup script with mechanics necessary for HCP. - provide an initial cluster config file that directs stolon to elect a leader when multiple masters are in play. - move the etcd initial cluster def into SDL - add reconfig check - move several env vars over to SDL * TEAMFOUR-1052: Allow the API Host URL to be changed by the UI * Remove debug info * Fixed api prefix * Address feedback from Chris * TEAMFOUR-1044 - Added endpoint to validate HCF credentials * drop sensitive DB config data from logs * update vcs.go to send logs to flight recorder * HSC-1040: Move versions api so that it does not require auth * Support proxy for Noaa consumer * HSC-1093: Fix JSON parse error * add skip_ssl_validation to CNSI records * use skip_ssl_validation flag to pick which http client to use for requests * store VCS tokens in the db for reuse * Refresh token on verifySession * HSC-1097 * Set Session expiry time to 20 minutes * Correctly enable the cleanup tick in pgstore * Reordered time * s/vcs_user_id/user_guid * check skip_ssl_validation flag on HCE endpoints, remove unused param from initializeHttpClients * Extend the expires_on of the session after verifySession is called * Change distro from debian to ubuntu * HSC-1097 - expose expiry time to clients * HSC-1106: Changes to ensure no_proxy applies to the http URLs that are used * Revert etcd containers to debian jessie * Update the logger command usage Update logger command to use: - FLIGHT_RECORDER_HOST - FLIGHT_RECORDER_PORT - -u to overcome issue with the version of logger that ships with trusty src: http://serverfault.com/questions/524480/ubuntu-12-04-logger-wont-write-to-remote-syslog * Specify the full hostname * Put this config file back * Address review feedback * Don't use shadow error when there is no parent error. Added missing SkipTLSVerification * Align field [trivial] * Removed unused code * Don't log DB username password. Forward exit code but still popd * Let's return 503 error when the db is not available * HSC-1109 Add SkipSSL bool to each VCS_CLIENT entry - added an optional bool to the vcs clients dta to allow the admin to indicate whether the Console should skipp SSL to a specific VCS endpoint. - fixed a few lint errors * HSC-1134 Secure our cookies Secure out session cookies with the ‘HttpOnly’ and ‘Secure’ attributes. * Also return 503 in admin middleware * HSC-1141 Ensure DB password is set for both install & upgrade The Stackato DB password is (currently) only getting set for install, not upgrade. This pulls that statement out of the IF clause and sets it always. * Changed approach to set NO_PROXY to reduce risk * Ensure all etcd host names are added to the NO_PROXY * Tweak if statement & remove cruft * Removed unused environment variable * Do not forward the Connection header to HCF * Add HCP CA to the OS CA certificate store (#124) * Add HCP CA to the OS CA certificate store * Added new line * Added new line * Extracted len(clientData) as variable [trivial] * Use the correct global http client in all vcs requests * Oops, pass the pointer to client, not the value * Return on error or else panic! * Initial changes to embed version in exe - Remove use of CONSOLE_VERSION - Add a new var to the interface for console version - Pull version in within main.go - Dockerfile change is WRONG and prob needs changing; don’t think we have access to env vars within Dockefile * Finish up the version logic update * Remove the use of the ‘Secure’ option for now * Added a Jira to cover this * HSC-1034 - Check VCS token is valid on OAuth (#125) * Check VCS token is valid on OAuth * Moved response.status read after error check to avoid panic * Fixed HSC-1166 * HSC-1042 Log all stolon logs to FlightRecorder * Revert back to the use of ENTRYPOINT As Neil corrrectly pointed out, the behavior of CMD vs ENTRYPOINT is different. Given I’m not seeing the logs get sent to FlightRecorder, yet I can do so manually, the likely culprit is that the default stolon ENTRYPOINT is boing run and my CMD is being run afterwards. Oops. This change (hopefully) fixes that. * Update user * Fix incorrect (and missing) transport timeouts (#123) * Correctly configure transport overall timeout. Set a connection timeout. Remove unused kill channel and superfluous timer * Removed unused select blocks * Reindent portal_config after merging master * Work around certain browsers sending back an empty cookie after logout * Renamed function to removeEmptyCookie * Better fix using regex * Also moved HttpOnly option into the sessionStore defaults instead of in Save * Remove redundant call to SetCookie as this is covered by clearSession * Drop cookies from request in login * Will separate secure cookie into another PR * Secure our cookie to only be sent over HTTPS (#131) * Detect error when the DB does not have an entry for a valid session cookie id (#133) * Refactor to remove stolon, etcd and use postgres alone * Enable migration of the database from stolon HA to single instance - Update the postfllight job container to use postgres 9.4 - refactor/simplify the RUN statements to reduce layers in the db container - Enable the database to be dumped from the stolon cluster and loaded on the single instance of postgres * HSC-1250 Containers need to include Third Party license text files * Typo * Update LICENSE file location * Fix for error logic to check for db and assume session issue otherwise * Default to returning 401, only return 503 when we get dial type errors * Trim whitespaces from VCS_CLIENTS param values (#135) * 1017 refactor unit tests (#104) * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * er -> err [trivial] * make sure verifySession() always returns HTTP Errors, fix panicky tests * factor out SQL statements * return errors instead of ok * DRY up the queries, arguments, and expected rows * fix up a few more tests * update calls to initializeHTTPClients and pp.doRequest to match new function signatures * Implement Goconvey (BDD) DSL (#140) * factor out SQL statements * DRY up the queries, arguments, and expected rows * Initial effort with the goconvey DSL * fixups * clean up these tests * refactor multiple funcs into one big one * Further refactor, DRY up these tests * Edge cases, more DRYing things up (85% coverage) * Need to call the correct method - I’m an idiot! * Add goose db version tests * Refactor datastore * Refactor to dry up encrypt/decrypt - Refactor to dry up encrypt/decrypt - fix up lint errors * Fix several annoying lint errors * Further test development, slight refactoring of ReadKey metyhod name * Skip unnecessary backup/restore to speed up upgrades * HSC-1185 - detect SSL verification problems * HSC-1276 - send regular WebSocket Ping messages to avoid proxies closing the connection if the app is quiet * Send pings every 27s Protect against very aggressive proxies by sending pings more frequently * HSC-1276 - close the connection if we miss a pong * Comment: ping -> pong * No ; * Bring docker registry, image name up to date (#137) * Detect a wider range of SSL errors * HSC-1292 - Faster deployment of Portal Proxy in dev environments (#145) * HSC-1292 - Faster deployment of Portal Proxy in dev environments * Optimised Goose file * HSC-1310 - Removed Dockerfile.goose.base (#146) This has migrated to stratos-deploy * Migrate Console to use GitHub/BitBucket Personal Access Tokens * Return empty array instead of JSON null in list calls when the list is empty * Check token scopes and report to the UI * On upgrade, automatically migrate VCS from connected Code Engines * Update to go1.7. Tidied up Pre/Post flight dockerfiles. Made the world debian-jessie * Removed reference to old git oauth flow templates * Remove bad sudo * Install missing lsb-release * Pin to earlier Postgres image * Fix volume name in middleware.go * Fix volume name in migrate-vcs.go * Wait for upgrade to complete before migrating VCSes * typo * Relay underlying message from internal method * Allow deleting a VCS (#153) * Allow deleting a VCS * Fixed comment * Detect that we failed to delete a VCS and return the correct status code (#154) * When SSl errors occur, feed back detail to the caller * More concise error msg * Small update to README to cause new git tag for 4.1 * Update README to get new hash * Support for HSM endpoints (#157) * Support for HSM endpoints * Remove commented out code as per Julein's review * CF Firehose support (#160) * Firehose support WIP * Ensure each user gets the full Firehose stream even if they share credentials * Refactor common code between app stream and firehose * Reordered funcs * Further polishing * Tidy ups and tune down logging * Refactor common code between firehose and app streams * Renamed file to represent enhanced responsibilities * Fix Portal Proxy tests and convert them to GoConvey (#162) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Let's not parse strings a byte at a time * Tests for token storage/retrieval (#163) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage (#164) * Fixed version_ info and pgsql_vcs_token tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed version_ info and pgsql_vcs_token tests * Fixed passthrough tests * Fixed passthrough & auth_tests tests * Migrated OAuth requests tests to GoConvey * Update README.md * Added tests for CNSI/UAA Token storage/retrieval * Added for VCS Tokens * Fixed typos * Added additional tests to reach 80%+ coverage * Initial k8s work * Updates to logging and configuration * Fix for for more log statements * Removed unsused HCP* config properties * WIP SQLite support * Added support for SQLite database backend * Remove files not needed * Update vendor folder * Add support for deploying as a Cloud Foundry application * Remove manifest file * Addressed PR issues and fixed unit tests * Fixes * Add extra metadata to the stackatoinfo call for Cloud Foundry * Rename stackatoInfo, fix verifySession response. Remove a couple of stackato references * SQLite Fix: Ensure SQL Statements use the right param substituion char and order is always 1,2,3 etc * Params MUST be in order - can't use indices with sqlite * Add check to make sure at least 1 row gets updated * Update README.md * Omit cloud foundry metadata when not applicable * Fixed unit test * Add portal-proxy concourse Dockerfile (#173) * Remove comment in test code * Change go path default * Remove HPE terms. Rename cookie * Test commit * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Optimised Postflight Job (#174) * Optimised Postflight Job * Fix unit tests Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Fix unit tests (#175) Remove `cloud-foundry` property because this is ommitted during marshalling if it empty * Missed a couple * Tidy up glide dependency management (#177) * Cleaned up glide.yaml * Update build script to run glide * Updated glide files * Add glide to concourse image for unit tests * Updated glide.lock file * Add openssl package to builder * Downgrade sqlmock-go * Add vendor to gitignore and downgrade sqlmock-go * Componentise proxy (#179) * Initial refactor * WIP * Get websocket/firehose working * Moving stuff around to write build scripts * Moved dockerfiles to be top level * renamed components to src * plugin load mechanism * fixed imports * plugin improvements * Gulp build mechanism [WIP] * Updated Gulp build files * Fixed lint issues * Fixed further lint issues * Updated Portal Proxy builder image * Updated load plugin mechanism * Dedup vendor before building plugins/core * Changes to interfaces * Updated GeneralPlugin interface * CloudFoundry Hosting plugin * Core changes * Moved portal proxy config definition * minor fixes * Final fixes to the CF Hosting plugin * Moved code around to reflect stratos-ui structure * Update build_portal_proxy.sh * Delete `build.sh` * remove core & misc changes * build updates * Update plugin names * Update build_portal_proxy.sh * Update portal-proxy unit tests * Debugging concourse tests * minor updates * Rename `EndpointSpecification` to `EndpointPlugin` * Renamed Dockerfile.dev * renamed backend Dockerfile * Fix lint issue * Rename backend gulp tasks * Renamed Dockerfiles * updated gulpfile for funnsies * remove hosting plugin * Remove old references (#1052) * Rename module helion.framework to app.framework * Remove references to 'helion' - Three remaining instances, two in README and one in nginx/license.txt - In some places I've swapped 'helion' for 'stratos' where we might want to use 'app' instead * gate check * Tweak readme * Remove references to stackato * Removed references to HPE * Removed references to hce Includes removal of code to hide/remove the 'hce' service previously associated with a hcf applicaiton with a hce pipeline. If application pipelines are included again it'll be a fresh implementation * Remove old company name * e2e fix * Updated class name app --> console * gate check * gate check * CI bump
Merge endpoint card changes, add kubernetes logo
* Make builds work and abstract away architecture (#1) * use centos 7 as default base image * don't remove packages that aren't there * set imagePullPolicy to IfNotPresent * adjust docker org and tag * make builds work with centos 7, also revise build script to include param for architecture * adjust image list for centos 7 * Improvements in vulnerability checking (#8) * Fix install of swag Go now uses `go install` instead of `go get` to install packages to the local machine * replace docs module appropriately * Configure dependabot Introduce a basic dependabot.yaml to check for updates both in the project root for npm and in the backend directory (src/jetstream) for go * Include website and electron projects in dependabot * Fix the CVEs in the jetstream backend (#12) * Delete grufty go.sum, gut go.mod entirely and refresh dependencies * add back some dependencies needed for tests to run; update return values of methods using go-sqlmock to use pointers * Fix backend unit tests (also code quality improvements) (#15) * Fix backend unit tests - Ensure int value is formatted as such - This was causing the backend unit tests to fail, possibly by dependency change * Replace ioutil usage and decapitalize error strings * Change receiver to be a pointer in order for assignment to persist If the receiver is not pointer, the struct is copied and the assignment `r.sent = true` has no effect because r will be discarded after the function returns. --------- Co-authored-by: Richard Cox <ricox@suse.com> * Theme Builder: Cast types (#16) In this case, `prjMetadata.root` is an untyped JSON result (that is, it can be object, array, number, etc.), but FS.copySync needs to take a string; do an explicit cast here to make TypeScript happy. Signed-off-by: Mark Yen <mark.yen@suse.com> Co-authored-by: Mark Yen <mark.yen@suse.com> * Update required disk space (#17) Co-authored-by: Neil MacDougall <neil_macdougall@hotmail.com> * Migrate to github.com/pressly/goose (#18) * Migrate to goose fork * Code quality improvements from go-staticcheck * Migrate migrations to github.com/pressly/goose --------- Co-authored-by: Ivan Kapelyukhin <ikapelyukhin@suse.com> * Move repository/interfaces to its own submodule: api (#19) * Delete grufty go.sum, gut go.mod entirely and refresh dependencies * add back some dependencies needed for tests to run; update return values of methods using go-sqlmock to use pointers * Update module structure * Small update of functions/imports to get latest version * Adjust to pointers for newer version of sqlmock * Remove ioutil, add new logic to actually get expiry claim from jwt * Move api into its own module * change all dependent code besides mock implementaions * Change mock implementations manually as no gomock command is documented * Moved and regenerated mock implementations This was donw with the commands `mockgen -source=api/auth.go > api/mock/mock_auth.go as well as `mockgen -source=repository/apikeys/apikeys.go > repository/apikeys/mock/mock_apikeys.go` after installing https://github.com/golang/mock with `go install github.com/golang/mock/mockgen@v1.6.0`. This tool however is archived and should be replaced with a fork. * Build and push base images on push to develop (#11) * Add action to build base images/push to ghcr.io * Upgrade go to 1.20.7 * set all docker related env variables to anynines/ghcr.io/centos7 * Add Helm support for local repos in desktop mode (#14) * Add Helm support for local repos in desktop mode * Join errors instead of returning just the second error * Implement all methods required by interface * Properly quote struct tags * Bubble up errors, don't just drop them * no longer use deprecated ioutil package, replace with os and io * Don't capitalize error messages * Remove formatting directives from string, parameters should be appended instead * Fix the CVEs in the jetstream backend (#12) * Delete grufty go.sum, gut go.mod entirely and refresh dependencies * add back some dependencies needed for tests to run; update return values of methods using go-sqlmock to use pointers * Fix backend unit tests (also code quality improvements) (#15) * Fix backend unit tests - Ensure int value is formatted as such - This was causing the backend unit tests to fail, possibly by dependency change * Replace ioutil usage and decapitalize error strings * Change receiver to be a pointer in order for assignment to persist If the receiver is not pointer, the struct is copied and the assignment `r.sent = true` has no effect because r will be discarded after the function returns. --------- Co-authored-by: Richard Cox <ricox@suse.com> * Theme Builder: Cast types (#16) In this case, `prjMetadata.root` is an untyped JSON result (that is, it can be object, array, number, etc.), but FS.copySync needs to take a string; do an explicit cast here to make TypeScript happy. Signed-off-by: Mark Yen <mark.yen@suse.com> Co-authored-by: Mark Yen <mark.yen@suse.com> * Update required disk space (#17) Co-authored-by: Neil MacDougall <neil_macdougall@hotmail.com> * Migrate to github.com/pressly/goose (#18) * Migrate to goose fork * Code quality improvements from go-staticcheck * Migrate migrations to github.com/pressly/goose --------- Co-authored-by: Ivan Kapelyukhin <ikapelyukhin@suse.com> * Move repository/interfaces to its own submodule: api (#19) * Delete grufty go.sum, gut go.mod entirely and refresh dependencies * add back some dependencies needed for tests to run; update return values of methods using go-sqlmock to use pointers * Update module structure * Small update of functions/imports to get latest version * Adjust to pointers for newer version of sqlmock * Remove ioutil, add new logic to actually get expiry claim from jwt * Move api into its own module * change all dependent code besides mock implementaions * Change mock implementations manually as no gomock command is documented * Moved and regenerated mock implementations This was donw with the commands `mockgen -source=api/auth.go > api/mock/mock_auth.go as well as `mockgen -source=repository/apikeys/apikeys.go > repository/apikeys/mock/mock_apikeys.go` after installing https://github.com/golang/mock with `go install github.com/golang/mock/mockgen@v1.6.0`. This tool however is archived and should be replaced with a fork. * Build and push base images on push to develop (#11) * Add action to build base images/push to ghcr.io * Upgrade go to 1.20.7 * set all docker related env variables to anynines/ghcr.io/centos7 * Add Helm support for local repos in desktop mode * Add github.com/pressly/goose --------- Signed-off-by: Mark Yen <mark.yen@suse.com> Co-authored-by: Neil MacDougall <neil.macdougall@suse.com> Co-authored-by: Richard Cox <ricox@suse.com> Co-authored-by: Mark Yen <mark.yen@suse.com> Co-authored-by: Neil MacDougall <neil_macdougall@hotmail.com> Co-authored-by: Ivan Kapelyukhin <ikapelyukhin@suse.com> * Fix dependency for database migrations (#20) * Add github.com/pressly/goose * Tidy up go.mod and go.sum * Adapt #4580: Update store references in core package to refer to @stratosui/store (#26) * Adapt #4580 All references to the core package have been changed to refer to @stratosui/store. Some unnecessary newlines have been removed. The public API of @stratosui/store has been adjusted as to expose all the necessary functions. --------- Co-authored-by: Matthias Folz <mfolz@anynines.com> * Fix database migrations by casting dialect correctly (#22) * Remove unnecessary ignored return value * change cast of dialect to pointer casts, so that they actually work * add Github Actions for Backend and Frontend Tests (#29) * add Github Actions for Backend and Frontend Tests * revert recently package-lock.json * Add script to help with building as packages and fix devkit and package info * Apply changes from #4529 (#28) * Fix typo (#31) * Code quality improvements via `staticcheck` (#24) * Fix error messages They shouldn't end with punctuation or newlines and should not be capitalized * Merge variable declarations and assignments if they are on subsequent lines * General improvements according to staticcheck * Change expected error messages in tests * Tidy up go.mod/go.sum * Change error message texts in response bodies * tidy up github action workflow for backend tests to resolve wrong golang version during tests (#32) * Update to node v20 (#34) * Update to node v18 * Update to node v20 * Remove sass-lint and stratos-protractor-reporter (#37) These not very necessary dependencies caused transitive deps to minimist and underscore in versions with CVEs. * Apply changes from #4583 (#33) * Upgrade code.cloudfoundry.org/cli to v8 (#42) * fix dependency issues to compile binary for CF * update go.mod/sum for jetstream * crteate extra_plugins.go also for backend tests * update kubernetes dependencies * Upgrade code.cloudfoundry.org/cli to 8.7.3; Remove push_actor wrapper since it is redundant; fix loggregator/logcache protobuf conflicts Co-authored-by: Matthias Folz <mafolz@users.noreply.github.com> --------- Co-authored-by: Matthias Folz <mfolz@anynines.com> Co-authored-by: Matthias Folz <mafolz@users.noreply.github.com> * Cherry pick commit 69ac7dc from other branch; now works without any restrictions; original commit message: (#45) Apply patch from freudl#1 manually This results in a version that works, but only when run with GOLANG_PROTOBUF_REGISTRATION_CONFLICT=warn or compiled with -ldflags "-X google.golang.org/protobuf/reflect/protoregistry.conflictPolicy=warn" This has to be adressed with an update to the cf dependencies to at least version 9 * fix gogoproto related panic when retrieving logs upgrade github.com/cloudfoundry/noaa/consumer to 2.3.0 (#46) * Fix frontend CVEs (#40) * Update @angular/core via ng update to 11.2.14 * Upgrade @angular/cdk to 11.2.14 * Update @angular/cli to 11.2.14 * Update @angular/material to 11.2.14 * Update @ngrx/store to v11 * WIP: fix peer dependencies for angular 11 This commit does NOT compile, likely due to an error with the angular compiler and mismatched version. The next commit will upgrade all deps to angular 12 to remedy this. * Update @angular/core to v12 * Update @angular/cdk to v12 * Update @angular/material and @angular/flex-layout to v12, also @ngrx/store * Update @angular/cli to v12 * Update @angular-builders/custom-webpack to v12, replace @cfstratos/ajsf-material with original @ajsf/material * change custom SassHandler to handle new WebPack 5 Structure as well * Fix KnownKeys type issues for updated TypeScript@4.3 * Remove debug logging * Remove overrides for angular upgrade * Upgrade to angular v13 * Upgrade to angular v14 * Remove references to @cfstratos/ajsf-material * Move code coverage to different tests * WIP: Fix tests, still a lot of warnings * discontinue use of karma-coverage-instanbul-reporter * Use @cypress/request where possible * Upgrade immer * Upgrade @types/karma * Update marked; change imports accordingly * Update immer; fix typing for api-request-reducers * update @swimlane/ngx-charts; fix type issues * Replace request with maintained fork * Add npm as engine requirement to make sure overrides work * Fix up lockfile * Formatting --------- Co-authored-by: Matthias Folz <mfolz@anynines.com> Co-authored-by: Tim Meier <tmeier@anynines.com> * feat: add errorz package to combine errors as enum in there (#47) * feat: add errorz package to combine errors as enum in there * feat: missed duplicated error * fix: not worked error checks * fix: not worked error checks * rename error package * applied JRA's suggestions * applied JRA's suggestions * fix error in test suite --------- Co-authored-by: Nedim Akar <nedimakar@Nedims-MacBook-Pro.local> Co-authored-by: Nedim Akar <nedimakar@nedims-mbp.speedport.ip> Co-authored-by: Jan-Robin Aumann <jaumann@anynines.com> * allow newer pgsql versions (#51) * feat: support for PostgreSQL 15 * feat: psql 15 integration --------- Co-authored-by: Nedim Akar <nedimakar@nedims-mbp.speedport.ip> * Bump @babel/traverse from 7.23.0 to 7.23.6 (#83) Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.23.0 to 7.23.6. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.23.6/packages/babel-traverse) --- updated-dependencies: - dependency-name: "@babel/traverse" dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump @adobe/css-tools from 4.3.1 to 4.3.2 (#77) Bumps [@adobe/css-tools](https://github.com/adobe/css-tools) from 4.3.1 to 4.3.2. - [Changelog](https://github.com/adobe/css-tools/blob/main/History.md) - [Commits](https://github.com/adobe/css-tools/commits) --- updated-dependencies: - dependency-name: "@adobe/css-tools" dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Release 4.5.1 (#130) * allow newer pgsql versions (#51) * feat: support for PostgreSQL 15 * feat: psql 15 integration --------- Co-authored-by: Nedim Akar <nedimakar@nedims-mbp.speedport.ip> * Allow product name to be changed (#25) Co-authored-by: Neil MacDougall <neil.macdougall@suse.com> * Initial support for CA Certificates for endpoints (#21) * Adapt "Initial support for CA Certificates for endpoints" * Add frontend changes for ca-certs * use correct attribute * fix pgsql tests by adding empty string for ca_cert * fix tests by adding the new columns "enabled" (tokens) and "cacert" (cnsis) in various locations for the test suite * implement requested changes * fix typo * update dependencies to address cves (#88) * upgrade golang/x/net to fix CVE-2023-39325 * address https://github.com/anynines/stratos/security/dependabot/170 * Address https://github.com/anynines/stratos/security/dependabot/323 * Address https://github.com/anynines/stratos/security/dependabot/321 * run go mod tidy * run go mod tidy * Upgrade archiver for https://github.com/anynines/stratos/security/dependabot/320 * Upgrade archiver in main go.mod * Update golang/x libs everywhere, tidy up plugins/monocular module file * upgrade grpc again * Bump word-wrap from 1.2.3 to 1.2.4 in /electron (#9) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump debug in /website (#41) Bumps and [debug](https://github.com/debug-js/debug). These dependencies needed to be updated together. Updates `debug` from 4.1.1 to 4.3.4 - [Release notes](https://github.com/debug-js/debug/releases) - [Commits](debug-js/debug@4.1.1...4.3.4) Updates `debug` from 3.2.6 to 4.3.4 - [Release notes](https://github.com/debug-js/debug/releases) - [Commits](debug-js/debug@4.1.1...4.3.4) --- updated-dependencies: - dependency-name: debug dependency-type: indirect - dependency-name: debug dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Deduplicate column names in tests (#75) * add samber/lo * add helper functions to datastore to centralize column name management * add test helper funtions to generate mock rows and predefined entities * switch tests over to helper functions * cover oauth tests as much as possible * add some more missing row addons * Fix columns * Switch CNSI tests over to actual cnsi records; also add expired tokens left out during first refactor * fix column name without underscore * Remove legacy GO111MODULE flag and update swag binary (#106) * Remove legacy GO111MODULE flag and update swag binary * Add fix for type resolution * INT-3729 - Stratos - Website: Update Docusaurus to nearest stable (#111) * Working 2.0.1 🎉 * Fix broken links * fix small mistake with sed * fix sed expression * fix indent * upgrade docusaurus to 2.4.3 (#112) * Upgrade to docusaurus v3 (#119) * Add 'all versions' to drop down * Upgrade versions to docusaurus v3 * Make the theme switcher button white * Properly escape the hint on commit signing * Add workarounds for v3 upgrade * fix indent * Add context on why workaround is necessary --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Nedim Akar <148047517+nedimakar@users.noreply.github.com> Co-authored-by: Nedim Akar <nedimakar@nedims-mbp.speedport.ip> Co-authored-by: Matthias Folz <mfolz@anynines.com> Co-authored-by: Neil MacDougall <neil.macdougall@suse.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update container-push-base-images-develop.yaml (#135) Change the secrets to properly work with github package registry --------- Signed-off-by: Mark Yen <mark.yen@suse.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Richard Cox <ricox@suse.com> Co-authored-by: Mark Yen <mark.yen@suse.com> Co-authored-by: Neil MacDougall <neil_macdougall@hotmail.com> Co-authored-by: Ivan Kapelyukhin <ikapelyukhin@suse.com> Co-authored-by: Neil MacDougall <neil.macdougall@suse.com> Co-authored-by: Matthias Folz <mfolz@anynines.com> Co-authored-by: Matthias Folz <mafolz@users.noreply.github.com> Co-authored-by: Tim Meier <115975772+tmeier-a9s@users.noreply.github.com> Co-authored-by: Tim Meier <tmeier@anynines.com> Co-authored-by: Nedim Akar <148047517+nedimakar@users.noreply.github.com> Co-authored-by: Nedim Akar <nedimakar@Nedims-MacBook-Pro.local> Co-authored-by: Nedim Akar <nedimakar@nedims-mbp.speedport.ip> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
refactor cp api layer to have a view -> model -> api structure