GROOVY-11879: A very simple DSL over the JDK's HTTP client

[paulk-asert]

groovy-http-builder (incubating)

A small DSL over JDK java.net.http.HttpClient making it more pleasant to use while staying lightweight.

Goals

• Keep implementation small and easy to maintain.
• Use only JDK HTTP client primitives (we optionally allow Jsoup for HTML parsing as a minor breakage of this rule).
• Make common request setup declarative with Groovy closures.
• Handle only the simple cases that often pop up in scripting and not the full use cases that Apache Geb covers.
• Include JSON/XML/HTML response parsing hooks while intentionally keeping request hooks minimal.

Example

import groovy.http.HttpBuilder

def http = HttpBuilder.http {
    baseUri 'https://example.com/'
    header 'User-Agent', 'my-app/1.0'
}

def res = http.get('/api/items') {
    query page: 1, size: 10
}

assert res.status == 200
println res.body

query(...) encodes keys/values as URI query components (RFC 3986 style), e.g. spaces become %20.

Non-DSL Equivalent (JDK HttpClient)

import java.net.URI
import java.net.URLEncoder
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse
import java.nio.charset.StandardCharsets

def baseUri = 'https://example.com/'
def query = [page: 1, size: 10]
        .collect { k, v ->
            "${URLEncoder.encode(k.toString(), StandardCharsets.UTF_8)}=" +
            URLEncoder.encode(v.toString(), StandardCharsets.UTF_8)
        }
        .join('&')

def target = URI.create(baseUri).resolve("/api/items?${query}")

def client = HttpClient.newHttpClient()
def request = HttpRequest.newBuilder(target)
        .header('User-Agent', 'my-app/1.0')
        .GET()
        .build()

def response = client.send(request, HttpResponse.BodyHandlers.ofString())

assert response.statusCode() == 200
println response.body()

JSON get Example

import static groovy.http.HttpBuilder.http

def github = http 'https://api.github.com'
def res = github.get('/repos/apache/groovy')

assert res.status == 200
assert res.json.license.name == 'Apache License 2.0'
assert res.parsed.license.name == 'Apache License 2.0' // auto-parsed from Content-Type

Non-DSL Equivalent (JDK HttpClient)

import groovy.json.JsonSlurper

import java.net.URI
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse

def client = HttpClient.newHttpClient()
def request = HttpRequest.newBuilder(URI.create('https://api.github.com/repos/apache/groovy'))
        .GET()
        .build()

def response = client.send(request, HttpResponse.BodyHandlers.ofString())
def payload = new JsonSlurper().parseText(response.body())

assert response.statusCode() == 200
assert payload.license.name == 'Apache License 2.0'

JSON post Example

def result = http.post('/api/items') {
    json([name: 'book', qty: 2])
}

assert result.status == 200
assert result.json.ok

XML get Example

def result = http.get('/api/repo.xml')

assert result.status == 200
assert result.xml.license.text() == 'Apache License 2.0'
assert result.parsed.license.text() == 'Apache License 2.0' // auto-parsed from Content-Type

HTML get Example (jsoup)

@Grab('org.jsoup:jsoup:1.22.1')
import static groovy.http.HttpBuilder.http

def client = http('https://mvnrepository.com')
def res = client.get('/artifact/org.codehaus.groovy/groovy-all') {
    header 'User-Agent', 'Mozilla/5.0 (Macintosh)'
}

assert res.status == 200

def license = res.parsed.select('div.metadata-row span.badge.badge-license')*.text().join(', ')
assert license == 'Apache 2.0'

Non-DSL Equivalent (JDK HttpClient + jsoup)

@Grab('org.jsoup:jsoup:1.22.1')
import org.jsoup.Jsoup

import java.net.URI
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse

def client = HttpClient.newHttpClient()
def request = HttpRequest.newBuilder(URI.create('https://mvnrepository.com/artifact/org.codehaus.groovy/groovy-all'))
        .header('User-Agent', 'Mozilla/5.0 (Macintosh)')
        .GET()
        .build()

def response = client.send(request, HttpResponse.BodyHandlers.ofString())
def document = Jsoup.parse(response.body())

assert response.statusCode() == 200
def license = document.select('div.metadata-row span.badge.badge-license')*.text().join(', ')
assert license == 'Apache 2.0'

HTML login Example

@Grab('org.jsoup:jsoup:1.22.1')
import static groovy.http.HttpBuilder.http

def app = http {
    baseUri 'http://myapp.com'
    followRedirects true
    header 'User-Agent', 'Mozilla/5.0 (Macintosh)'
}

def loginPage = app.get('/login')
assert loginPage.status == 200
assert loginPage.html.select('h1').text() == 'Please Login'

def afterLogin = app.post('/login') {
    form(username: 'admin', password: 'p@ssw0rd')
}

assert afterLogin.status == 200
assert afterLogin.html.select('h1').text() == 'Admin Section'

Form URL-Encoding Helper

This example shows the form helper.

def result = http.post('/login') {
    form(username: 'admin', password: 'p@ssw0rd')
}

assert result.status == 200

form(...) encodes values as application/x-www-form-urlencoded and sets
Content-Type automatically (unless you override it with header).
result.parsed dispatches by response Content-Type:

• application/json and application/*+json -> JSON object
• application/xml, text/xml, and application/*+xml -> XML object
• text/html -> jsoup Document if jsoup is found on the classpath, otherwise raw string body
• anything else -> raw string body

Try It

From the repository root:

./gradlew :groovy-http-builder:test

[codecov-commenter]

Codecov Report

❌ Patch coverage is 71.71053% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.6870%. Comparing base (b37141f) to head (46b3c44).

Files with missing lines	Patch %	Lines
...der/src/main/groovy/groovy/http/HttpBuilder.groovy	73.6000%	21 Missing and 12 partials ⚠️
...lder/src/main/groovy/groovy/http/HttpResult.groovy	62.9630%	5 Missing and 5 partials ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##               master      #2401        +/-   ##
==================================================
+ Coverage     66.6787%   66.6870%   +0.0083%     
- Complexity      29858      29895        +37     
==================================================
  Files            1382       1384         +2     
  Lines          116145     116297       +152     
  Branches        20481      20505        +24     
==================================================
+ Hits            77444      77555       +111     
- Misses          32358      32383        +25     
- Partials         6343       6359        +16     

Files with missing lines	Coverage Δ
...lder/src/main/groovy/groovy/http/HttpResult.groovy	62.9630% <62.9630%> (ø)
...der/src/main/groovy/groovy/http/HttpBuilder.groovy	73.6000% <73.6000%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sbglasius]

Great idea!

[daniellansun]

+1

[daniellansun]

Merged. Thanks.