Unmock server #318

ksaaskil · 2019-10-16T06:18:31Z

Changes

Adds unmock-server package
Contains a proxy (proxy.ts) and unmock server (server.ts)
Proxy receives requests and forwards them to the unmock server. Note that the proxy cannot look at the traffic in the case of HTTPS request, at least not without decoding with the private key. Proxy should maybe rather send back its own certificate and communicate with the target host directly.
Server receives proxied requests and looks at X-Forwarded-Host (or Host) header to determine where the request was going and serializes the request to a ISerializedRequest
ISerializedRequest is handled with unmock

Instructions

Create self-signed certificate (key.pem and cert.pem) for a given domain:

bash packages/unmock-server/scripts/prepare-cert.sh petstore.swagger.io
~~openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=*.swagger.io'~~
Create __unmock__ folder with service for petstore.swagger.io/v2
Start server: npm run compile && node packages/unmock-server/index.js. This starts the proxy at localhost:8008 and Unmock server at localhost:8000 (HTTP) and localhost:8443 (HTTPS)
Make request using the proxy, trusting the certs: http_proxy=http://localhost:8008 HTTPS_PROXY=http://localhost:8008 SSL_CERT_FILE=cert.pem curl -i http://petstore.swagger.io/v2/pet/23

Issues

Certificates in case of HTTPS

The certificate used by the Unmock server must be explicitly trusted by the client.
Alternatively, the client must accept invalid certificates (á la process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0')
How to create a certificate valid for all the domains mocked by the server? Should they be created on-fly when creating the service and bundled together? How does Hoverfly use their certificate?

Setting the proxy

Some applications and libraries respect http_proxy and https_proxy (or uppercased) environment variables, so if application is using those, requests are easy to proxy to the running Unmock proxy
Node.js does not respect the environment variable (http: support environment-defined proxy nodejs/node#8381). (But in Node.js, one can just use unmock library)
Some tools like proxychains might be useful to look at
If we build our own Docker images, they can be set to use a proxy

Managing service state

PR contains a primitive API at /api for setting state
How should this work in general?

codecov-io · 2019-10-16T06:25:11Z

Codecov Report

❗ No coverage uploaded for pull request base (dev@54c6b9f). Click here to learn what that means.
The diff coverage is 80.23%.

@@          Coverage Diff           @@
##             dev     #318   +/-   ##
======================================
  Coverage       ?   80.05%           
======================================
  Files          ?       49           
  Lines          ?     2226           
  Branches       ?      569           
======================================
  Hits           ?     1782           
  Misses         ?      444           
  Partials       ?        0

Impacted Files	Coverage Δ
packages/unmock-node/src/backend.ts	`100% <100%> (ø)`
packages/unmock-node/src/index.ts	`100% <100%> (ø)`
packages/unmock-server/src/constants.ts	`100% <100%> (ø)`
packages/unmock-server/src/server.ts	`75.8% <75.8%> (ø)`
packages/unmock-server/src/unmock.ts	`88.88% <88.88%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54c6b9f...47e0e26. Read the comment docs.

* Don't run lint and test compile. * Read key and cert from cwd. * Add script for preparing certs. * Move the script. * Fix the script. * Add API. * Add helmet. * Update script. * Re-introduce long post-install.

This reverts commit 2e051fb.

MeeshkanCain

lgtm; I had some really minor suggestions that are totally unrelated to the way things work.

Good job 👍 👍

MeeshkanCain · 2019-10-25T12:16:18Z

package.json

@@ -94,6 +95,7 @@
    "unmock-fetch": "file:packages/unmock-fetch",
    "unmock-jest": "file:packages/unmock-jest",
    "unmock-node": "file:packages/unmock-node",
+    "unmock-server": "file:packages/unmock-server",


Should this also be included in the lint-ts and lint-ts-fix scripts above?

MeeshkanCain · 2019-10-25T12:20:58Z

packages/unmock-server/src/proxy.ts

+  });
+
+  proxySocket.on("error", () => {
+    socket.write("HTTP/" + req.httpVersion + " 500 Connection error\r\n\r\n");


Could create a function to maintain the HTTP/${req.httpVersion} ${msg}\r\n\r\n pattern.

MeeshkanCain · 2019-10-25T12:21:43Z

packages/unmock-server/src/proxy.ts

+  });
+
+  socket.on("data", chunk => {
+    proxySocket.write(chunk);


Do we need any checks on the chunk being written (e.g. that it ends with \r\n\r\n)?

I'm not sure we need that, it's up to the target server to return proper data. Also not sure how that would work with encryption.

MeeshkanCain · 2019-10-25T12:22:49Z

packages/unmock-server/src/server.ts

+import { createUnmockAlgo } from "./unmock";
+
+const httpPort = 8000;
+const httpsPort = 8443;


This is linked to the port defined in proxy.ts; could benefit from a shared definition to avoid potential breakage.

Added constants.ts.

MeeshkanCain · 2019-10-25T12:26:53Z

packages/unmock-server/src/server.ts

+export const startServer = (app: express.Express) => {
+  const options = {
+    key: fs.readFileSync(`${process.cwd()}/key.pem`),
+    cert: fs.readFileSync(`${process.cwd()}/cert.pem`),


I'd say a .env file could be more useful here than cwd

Probably, will do later on if we work on this more.

Kimmo Sääskilahti added 11 commits October 14, 2019 13:33

Replace IInterceptorConstructor with IInterceptorFactory.

f65b38e

Fix typing in test.

3c21bf1

Merge branch 'dev' into replace-ctor

21d72d2

Fix TestBackend.

1fac87d

Add unmock-server package.#

48dfa73

Merge remote-tracking branch 'origin/replace-ctor' into unmock-server

41e18c2

Add ServerBackend.

faa4c14

Add proxy and server.

4589030

First iteration of unmock server.

1b99d5b

Rename stuff, add X-Forwarded-For for testing.

409e054

Fix package.

c6955cd

Kimmo Sääskilahti added 4 commits October 16, 2019 09:27

Refactor dependencies.

d215297

Refactor, add one test.

048409b

Add assert.

880ec91

Add unmock.reset.

f0138cc

ksaaskil mentioned this pull request Oct 16, 2019

Refactor: Make Backend concrete, expose algorithm. #319

Merged

Kimmo Sääskilahti added 8 commits October 16, 2019 11:32

Merge remote-tracking branch 'origin/dev' into unmock-server

ec2a13a

Delete server.ts for now.

0b87384

Tweak index.js.

b50e8c2

Add prepare-cert.sh script.

95b3ef2

Develop server. (#329)

8e856cf

* Don't run lint and test compile. * Read key and cert from cwd. * Add script for preparing certs. * Move the script. * Fix the script. * Add API. * Add helmet. * Update script. * Re-introduce long post-install.

Add a test for setting state.

4642d80

Add proxy test, does not do much.

2e051fb

Revert "Add proxy test, does not do much."

237b701

This reverts commit 2e051fb.

ksaaskil marked this pull request as ready for review October 25, 2019 10:25

Merge remote-tracking branch 'origin/dev' into unmock-server

e5b2b3e

MeeshkanCain reviewed Oct 25, 2019

View reviewed changes

Kimmo Sääskilahti added 2 commits October 28, 2019 08:50

Add unmock-server to lint scripts.

b7f5507

Add function for formatting HTTP response.

9dd1e4e

Move constants to a file.

47e0e26

ksaaskil merged commit 4113a9f into dev Oct 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unmock server #318

Unmock server #318

ksaaskil commented Oct 16, 2019 •

edited

Loading

codecov-io commented Oct 16, 2019 •

edited

Loading

MeeshkanCain left a comment

MeeshkanCain Oct 25, 2019

ksaaskil Oct 28, 2019

MeeshkanCain Oct 25, 2019

ksaaskil Oct 28, 2019

MeeshkanCain Oct 25, 2019

ksaaskil Oct 28, 2019

MeeshkanCain Oct 25, 2019

ksaaskil Oct 28, 2019

MeeshkanCain Oct 25, 2019

ksaaskil Oct 28, 2019

Unmock server #318

Unmock server #318

Conversation

ksaaskil commented Oct 16, 2019 • edited Loading

Changes

Instructions

Issues

Certificates in case of HTTPS

Setting the proxy

Managing service state

codecov-io commented Oct 16, 2019 • edited Loading

Codecov Report

MeeshkanCain left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ksaaskil commented Oct 16, 2019 •

edited

Loading

codecov-io commented Oct 16, 2019 •

edited

Loading