Home

Requirements

The requirement is to design and develop services that work in synergy using gRPC to stream a comma-separated sequence of prime numbers up to a given number (and including if a prime number itself) over RESTful endpoints.

Design Proposal

We can write two services namely prime-generator and prime-proxy.

• prime-generator will be responsible for efficiently generating a sequence of prime numbers up until a given number and stream it to prime-proxy over gRPC when requested.

• prime-proxy can then stream these prime numbers as a comma-separated sequence of numbers over RESTful endpoints. These endpoints could also be made available over CORS to avoid cross origin issues.

The RESTful endpoints could be made available for consumption behind a reverse-proxy server as shown in the diagram above. Since both the prime-generator and prime-proxy services are stateless we could scale these services hassle-free by spinning-up multiple container instances and have them load balanced behind a reverse proxy.

Project Layout

The complete project is laid out under following directories:

• prime-generator

  This directory hosts all the resources, source and test code pertaining to prime-generator service.

• prime-proxy

  This directory hosts all the resources, source and test code pertaining to prime-proxy service. It also contains the end-to-end integration tests.

• prime-common

  This directory hosts all the common code.

• prime-protobuf

  gRPC proto files are maintained in a separate SBT project as suggest by akka-grpc docs. This is done to avoid maintaining proto files in multiple locations which could soon lead to maintenance nightmare with the rise of number of proto files.

  The project can be cross-compiled to multiple Scala versions. This is helpful if gRPC client and server are on different Scala versions.

  Once jars are published, the following library dependency could be added to both client and server build.sbt:

  libraryDependencies += "com.iamsmkr" %% "prime-protobuf" % ""0.1"" % "protobuf-src"

• deploy

  This directory hosts all kubernetes deployment manifest files under deploy/k8s and a convenience script to install minikube and kubectl under deploy/script.

Testing Services

See README on how to run tests.

Unit Testing

• Prime Generator

  Unit test cases are implemented for prime generator utility as property-based tests. The utility itself generates a continuous stream of prime numbers, the test suite however tests for validity of a the stream up until a given number only. For any number less than 2, an empty stream is tested to be expected.

• gRPC Service

  gRPC service implementation is property-based unit tested. For any number less than 2, GrpcServiceException is tested to be expected.

• RESTful Routes

  prime-proxy routes are unit tested using Route Testkit to return valid responses. For any invalid endpoints reached, the services is tested to expect proper error messages to be presented to the user.

Integration Testing

• End-to-end

  Simplest of all test suites, end-to-end integration tests attempt to make REST API calls to actual proxy-server instance and validates responses.

  Note: End-to-end integration testing requires both prime-proxy and prime-generator services to be up and running.

What are other REST APIs for?

Every REST endpoint implementation was a step towards fulfilling the requirement of implementing an REST endpoint that streams prime numbers as a comma-separated sequence (I decided to keep all the endpoints for my future references). Although we use Akka HTTP day-in day-out at our work but I wasn't aware of Akka HTTP support for Source Streaming.

The endpoints were implemented in the following order:

• /prime/23/seq

  I tried implementing in-memory comma-separated sequence of prime numbers knowing that it would result into OOM issue when present with a very large number. So I decided to put an upper circuit of 10000 prime numbers to be returned by the API.

• /prime/23/sse

  Then I thought it should be enough to implement the required API as ServerSentEvents.

• /prime/23/csv-stream

  Stumbling upon Akka HTTP support for CSV Source Streaming, I tried the example explained in the docs but realised that it won't be possible to implement a comma-separated sequence of prime numbers with the default CsvEntityStreamingSupport class.

• /prime/23

  With the help of awesome Stackoverflow community, I was able to implement a custom CsvEntityStreamingSupport instance that fulfilled the requirement of streaming comma-separated sequence of prime numbers.