[discuss] improvements to mocks in the next version

[ptrthomas]

also paging @michaelpro1 as this is related to #1159

lots to unpack here, so will start with bullet points - feel free to ask questions and I can elaborate

1. while switching to graal-js, currently doing a overhaul of core GraalJS and runner and reporter improvements - performance, streams, config #1281
2. have come up with a new way to create mocks which will be pure JS, here is an example: payments.js
3. but we will still retain the old way of mocks as gherkin feature files
4. one big difference in the new approach is that each JS file handles a single route (resource) instead of having multiple routes (Scenarios) in one monolithic feature file
5. I'd like to keep the "feature mocks" simple, and here is where I've started to re-write it for graal-js: MockHandler.java
6. I'm proposing that we keep the feature mocks "one at a time" like before, without calculating scores etc. right now the code is a little too complicated for my liking, also I think the users who like the feature mocks may find the rules too complex, I also want to keep it lightweight from a code complexity and runtime performance standpoint
7. but actually, if you look at the above file - it should be easy to support multiple features, because we can load multiple scenarios into an in-memory collection. even if we do this, I'd like to go with a simple strategy of "first feature" and "first scenario" wins
8. my proposal is that power-users who want advanced mocking can use the JS approach, which BTW is a general-purpose server-implementation that can even render HTML / AJAX - you can find out more in the code if you are interested. so this way you can get the outcome of being able to "aggregate multiple mocks into one" because that is how it is by default

let me know your thoughts :) and @michaelpro1 feel free to weigh in with design suggestions based on your experience with the old mocks

[michaelpro1]

Pure javascript looks good too, I'm thinking to improve readability you'll need to add few more helper functions. e.g. an example

===main.js===
if (pathStartsWith('/cats/{apiVersion}')) //This stores {apiVersion} in pathParams and creates subPath var
  require('cats.js')()

else if (pathStartsWith('/dogs/{apiVersion}'))
  require('dogs.js')()

else if (pathStartsWith('/pets/{apiVersion}')) 
  require('pets.js')()

else
  response = {
    status: 404
    body: 'not found'
  }

===cats.js===
session.cats = [:]

if (subPathMatches('/food') && request.method in ['GET', 'HEAD']) //This matches /cats/v1/food
  response.body = 'kitekat'
else if (subPathMatches('') && request.method == 'GET')
  response.body = session.cats

This is just an example, but allowing some kind of router-subrouter will help split the more advanced mocks. This is where well thought out helper methods would help avoid boiler-plate and the mocks maintainable.
Reverting the .feature style mocks into first-match then allows for some migration path forward.

[ptrthomas]

@michaelpro1 great, thanks for taking a look. I also would lean towards being more "opinionated" for e.g.

• no need for a "main" entry point router. btw the ServerConfig is a place where you can plug-in fancy central behavior if really needed: https://github.com/intuit/karate/blob/rewrite/karate-core2/src/main/java/com/intuit/karate/server/ServerConfig.java#L48
• else you just add JS files for more end-points. note that the directories matter, e,g, /api/cats/kittens.js
• and the "global" session is how state can be shared if needed - you can use a DB if needed, it is all interfaces / pluggable
• yes some helpers for subPathMatches() may evolve, but for now I think may be not

[michaelpro1]

• else you just add JS files for more end-points. note that the directories matter, e,g, /api/cats/kittens.js

This seems OK for simple mocks (without wildcard/regex params), but for advanced usage maybe ServerContext can also lookout for featureServer.js or something like that?

• and the "global" session is how state can be shared if needed - you can use a DB if needed, it is all interfaces / pluggable

Are you planning on using the non-locking request processing as I see with ThreadLocals?
I ran into a lot of trouble in Gatling/JMeter tests because either underlying objects weren't Concurrent/Synchronized or there was no longer locking of them during updates or deletes.

• yes some helpers for subPathMatches() may evolve, but for now I think may be not

Cool let me know if you want me to add them once you are done

[ptrthomas]

but for advanced usage

here's where my PoV is that this is a "mock" ;) that said, I'll wait for the advanced use cases to become obvious. i the meantime feel free to provide examples of these advanced use cases. my hypothesis is that the way it is right now is sufficient to build a complete back-end (including HTML templating BTW)

and yes, I'm going with ThreadLocal all the way

[michaelpro1]

I do find that mocks start off small but tend to grow larger as systems start interacting with more and more things.

One example would be BenTen's Karate mocks, and another is more generic Test Harness that could encompass OAuth, Microservices and even some external APIs which may or may not have wildcard path parameters.

Deferring all logic to the underlying JS allows your codebase to be minimal, as routing, code re-use etc. is handled by underlying JS/GraalJS Engine while still allowing for opinionated mock organization when main file is omitted.

Templating and matching are definately the more powerful facets of Karate, though its unusual for REST API to render HTML out. They definitely come in handy with JSON request/responses though.

[ptrthomas]

@michaelpro1 noted. I think there is a need for router.js, not ruling it out - but I would like to see a real-life need / example first. I 100% see a need to do some fancy mapping e.g. /foo/v1/* --> /api/dogs.js

regarding html - you can try run ServerRunner - navigate to localhost:8080/index and take a look

[michaelpro1]

Ahh you're using Thymeleaf for that, do you think it would make more sense to apply the HTML logic if either

• Content-Type: */*html
• Template name ends with .htm
• Content includes <html>, but this is bound to run into trouble

Otherwise using Thymeleaf is excellent 👍 at this kind of thing, just have to make sure that its cache is cleared on file refresh if it's using any.

[ptrthomas]

@michaelpro1 I think you are still thinking REST, this is way more ambitious - intended to be "the web framework I always wanted". for e.g. this is not just Thymeleaf - but combined with GraalJS - more on this later but for now see what happens at runtime to <div th:insert="person" ka:target="this"></div>

[hujunhaorobert]

Hi @ptrthomas and @michaelpro1,

I had been using Karate Mock 0.9.5 for a while. And noticed there were big changes on scenario-matching algorithm from 0.9.5 to 0.9.6. In 0.9.6 calculating scores makes the mock scenario-matching rule more complex. And did not see any document to explain the scenario matching rule before diving into the change commits #1159.

I like all of the mock improvement ideas and especially the bullet points 1, 5, 6, 7 listed at the beginning. And expect the mock improvement release coming soon.

[ptrthomas]

@hujunhaorobert thanks for the feedback, this helps. keep watching the twitter / linkedin feeds for updates, we'll announce when RC releases are ready

[ptrthomas]

@hujunhaorobert I've just brought the rewrite branch up to par for mocks. there are breaking changes to the Java API but not the JS API. I curated this commit to serve as a reference and one of the hard things ahead is to write an upgrade / migration guide: a83f76e

would be good if you can follow the development guide on the wiki and try things locally

[hujunhaorobert]

@ptrthomas Just noticed your message. Thanks for your effort on the mock server improvement. Yes, I will try that locally firstly and then give feedback to you.

[ptrthomas]

closing this thread, use #1373 for any more discussion