Repository navigation
Case Study: Generating Unit Tests to get to 95% coverage in under 30 mins
Yureshwar Ravuri edited this page Jul 1, 2025
·
7 revisions
Generating unit tests with genAI is incredible fast and quite thorough. We can get to ~90% coverage with minimal effort. Here is an interesting comparison of LLMs performance The generation of the unit tests itself takes under 5 mins. We then need to read, validate, run and extend these unit tests. As with all genAI output manual verification and validation is critical.
-
Here is the original file for NanoHealth method for generating OTP
- As with all startups under pressure to deliver features, Nanohealth too prioritised velocity of features and overlooked code coverage.
- As number of customers grew, there was a demand for more changes in this functionality.
- The team was starting to break more things with every new feature and this was causing delays and stress. This was also slowing down the pace of release of new features and hence affecting the velocity of revenue.
- Below are the unit tests generated by genAI.
- These tests were generated in multiple iterations using Cody and Claude Sonnet 3.5. All iterations cumulatively took less than 20 mins.
- The developer spent about 30 mins to review, run and finalise the unit-tests.
- The genAI generated tests were manually reviewed for relevance, accuracy, necessity (remove some unnecessary unit-tests), coverage and edge cases.
- Some unit tests were added manually.
- Some generated tests were edited by hand.
Assume the role of a senior software architect specializing in Node.js and JavaScript testing. Analyze the provided code, which has been refined for error handling, input validation, sanitization, type strictness, security enhancements, and documentation. Your task is to design a robust suite of unit tests that thoroughly validate its functionality.
Leverage the existing documentation and inline comments to ensure that all logic branches, conditions, and edge cases are covered. Write unit tests that verify expected behaviors, handle erroneous inputs, and validate security-related checks. Ensure that all errors in the unit tests themselves are handled properly to avoid misleading results.
Additionally, document the unit tests thoroughly. Provide clear function descriptions, expected inputs/outputs, and inline comments explaining the purpose of each test case. Maintain consistency in formatting and structure, ensuring the tests are easy to understand and maintain.
Do not suggest any setup for the testing framework; assume the testing environment is already configured. Strictly focus on writing well-documented and structured unit tests with proper error handling.
Provide a detailed report outlining the test coverage, identified gaps (if any), and scenarios included. At the end of the analysis, present the complete set of unit tests with inline comments and documentation in a structured final output, ensuring clarity and ease of implementation.
The time for manual review will vary with code complexity. Despite this added time for manual review, genAI will nonetheless reduce time/effort for unit-tests from 75%(complex functionality) to 95%(simple functionality).
Note 1: This time estimate assumes that the unit-test libraries and frameworks are installed and working. Installing these is a one-time effort for the project.
Note 2: Below tests were generated before we added input sanitisation. There were more unit tests generated after we added the input sanitisation but for illustrative purposes, the below file is adequate.
# @title ABHA Web Services Controller Test Suite
# @description Comprehensive test suite for AbhaWebServicesController that handles ABHA
# (Ayushman Bharat Health Account) web service integration testing.
#
# Key Features Tested:
# - Aadhaar OTP generation
# - Input validation
# - Error handling
# - Authentication flows
# - Response formatting
# - Rate limiting
# - Service maintenance
#
# Test Coverage Areas:
# 1. Valid parameter handling
# 2. Invalid parameter validation
# 3. Authentication scenarios
# 4. Service response handling
# 5. Error conditions
# 6. Network timeouts
# 7. Security aspects
# 8. Performance monitoring
#
# @author [Your Team Name]
# @version 1.0
# @since [Date]
RSpec.describe AbhaWebServicesController, type: :controller do
describe '#generate_aadhar_otp' do
# Test data setup
let(:valid_aadhaar) { "123412341234" }
let(:invalid_aadhaar) { "123456" }
let(:auth_token) { "valid_token_123" }
# @section Valid Parameters Testing
# Tests successful OTP generation with valid inputs
context 'with valid parameters' do
before do
allow(VerhoeffValidator).to receive(:validate_aadhaar)
.with(mock_aadhaar_numbers[:valid])
.and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha)
.and_return(mock_auth_tokens[:valid])
allow(AbhaWebService).to receive(:generate_aadhar_otp)
.and_return(mock_responses[:success])
end
it 'generates OTP successfully' do
post :generate_aadhar_otp, params: { aadhar: mock_aadhaar_numbers[:valid] }
expect(response).to have_http_status(:success)
expect(JSON.parse(response.body)["status"]).to eq("success")
end
end
context 'with invalid parameters' do
it 'returns error for missing Aadhaar' do
post :generate_aadhar_otp, params: {}
expect(JSON.parse(response.body)["status"]).to eq("error")
expect(JSON.parse(response.body)["message"]).to eq("Aadhaar number is required")
end
it 'returns error for invalid Aadhaar format' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(invalid_aadhaar).and_return(false)
post :generate_aadhar_otp, params: { aadhar: invalid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
expect(JSON.parse(response.body)["message"]).to include("Invalid Aadhaar number")
end
end
context 'with authentication failures' do
before do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(nil)
end
it 'returns error for authentication failure' do
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
expect(JSON.parse(response.body)["message"]).to include("Authentication failed")
end
end
context 'with ABHA service responses' do
before do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token)
end
it 'handles 400 response' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "400", resp_body: "Bad Request" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["code"]).to eq(400)
end
it 'handles 401 response' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "401", resp_body: "Unauthorized" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["code"]).to eq(401)
end
it 'handles 429 response' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "429", resp_body: "Too Many Requests" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["code"]).to eq(429)
end
it 'handles 500 response' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "500", resp_body: "Server Error" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["code"]).to eq(500)
end
end
context 'with malformed JSON responses' do
before do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token)
end
it 'handles invalid JSON in response body' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "200", resp_body: "invalid json" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
expect(JSON.parse(response.body)["message"]).to include("Invalid response format")
end
it 'handles missing transaction ID in success response' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "200", resp_body: "{}" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
end
end
context 'with network timeouts' do
before do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token)
end
it 'handles connection timeout' do
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_raise(Timeout::Error)
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
expect(JSON.parse(response.body)["code"]).to eq(500)
end
end
context 'with encryption scenarios' do
it 'handles encryption failures' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token)
allow(RsaEncryption).to receive(:oaep_mgf1_padding).and_raise(OpenSSL::PKey::RSAError)
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["status"]).to eq("error")
end
end
context 'with rate limiting' do
it 'handles concurrent requests' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token)
3.times do
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
end
expect(response).to have_http_status(:success)
end
end
context 'with special characters' do
it 'sanitizes input parameters' do
post :generate_aadhar_otp, params: { aadhar: "123412341234<script>" }
expect(JSON.parse(response.body)["status"]).to eq("error")
end
end
context 'with different Aadhaar formats' do
it 'handles Aadhaar with spaces' do
post :generate_aadhar_otp, params: { aadhar: "1234 1234 1234" }
expect(JSON.parse(response.body)["status"]).to eq("error")
end
it 'handles Aadhaar with dashes' do
post :generate_aadhar_otp, params: { aadhar: "1234-1234-1234" }
expect(JSON.parse(response.body)["status"]).to eq("error")
end
end
context 'with token expiration scenarios' do
it 'handles expired authentication token' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return('expired_token')
allow(AbhaWebService).to receive(:generate_aadhar_otp).and_return({ resp_status: "401", resp_body: "Token expired" })
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["code"]).to eq(401)
end
it 'attempts token refresh on expiry' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
allow(AbhaWebService).to receive(:authenticate_abha).and_return(auth_token, 'new_token')
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(AbhaWebService).to have_received(:authenticate_abha).twice
end
end
context 'with response caching' do
it 'caches successful responses' do
allow(Rails.cache).to receive(:write)
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(Rails.cache).to have_received(:write)
end
end
context 'with performance monitoring' do
it 'logs response time' do
allow(Rails.logger).to receive(:info)
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(Rails.logger).to have_received(:info).with(/Response time:/)
end
end
context 'with multiple OTP requests' do
it 'handles sequential OTP requests' do
allow(VerhoeffValidator).to receive(:validate_aadhaar).with(valid_aadhaar).and_return(true)
2.times do
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(response).to have_http_status(:success)
end
end
end
context 'with service maintenance mode' do
it 'handles service maintenance windows' do
allow(AbhaWebService).to receive(:maintenance_mode?).and_return(true)
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(JSON.parse(response.body)["message"]).to include("maintenance")
end
end
context 'with request validation' do
it 'validates request headers' do
request.headers['X-API-Version'] = 'invalid'
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(response).to have_http_status(:bad_request)
end
it 'validates content type' do
request.headers['Content-Type'] = 'text/plain'
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(response).to have_http_status(:unsupported_media_type)
end
end
context 'with response format validation' do
it 'validates response structure' do
post :generate_aadhar_otp, params: { aadhar: valid_aadhaar }
expect(response.content_type).to eq('application/json')
expect(JSON.parse(response.body)).to include('status')
end
end
end
end