Skip to content

perf: use textproto.TrimString for HTTP header parsing to improve performance #4257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

1911860538
Copy link
Contributor

@1911860538 1911860538 commented Jun 3, 2025

Replaced strings.TrimSpace with textproto.TrimString in HTTP header parsing functions to improve performance and better align with HTTP spec.
textproto.TrimString, which is widely used in net/http, trims only ASCII whitespace—making it both sufficient and more efficient for HTTP headers like X-Forwarded-For and Accept.

Copy link

codecov bot commented Jun 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.92%. Comparing base (3dc1cd6) to head (84814fa).
Report is 129 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4257      +/-   ##
==========================================
- Coverage   99.21%   98.92%   -0.30%     
==========================================
  Files          42       44       +2     
  Lines        3182     3434     +252     
==========================================
+ Hits         3157     3397     +240     
- Misses         17       26       +9     
- Partials        8       11       +3     
Flag Coverage Δ
?
--ldflags="-checklinkname=0" -tags sonic 98.85% <100.00%> (?)
-tags go_json 98.85% <100.00%> (?)
-tags nomsgpack 98.90% <100.00%> (?)
go-1.18 ?
go-1.19 ?
go-1.20 ?
go-1.21 ?
go-1.23 98.92% <100.00%> (?)
go-1.24 98.92% <100.00%> (?)
macos-latest 98.92% <100.00%> (-0.30%) ⬇️
ubuntu-latest 98.92% <100.00%> (-0.30%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@1911860538
Copy link
Contributor Author

test code

package main

import (
	"net"
	"net/textproto"
	"strings"
	"testing"
)

func validateHeaderOriginal(header string) (clientIP string, valid bool) {
	if header == "" {
		return "", false
	}
	items := strings.Split(header, ",")
	for i := len(items) - 1; i >= 0; i-- {
		ipStr := strings.TrimSpace(items[i])
		ip := net.ParseIP(ipStr)
		if ip == nil {
			break
		}
		if (i == 0) || (!isTrustedProxy(ip)) {
			return ipStr, true
		}
	}
	return "", false
}

func validateHeaderOptimized(header string) (clientIP string, valid bool) {
	if header == "" {
		return "", false
	}
	items := strings.Split(header, ",")
	for i := len(items) - 1; i >= 0; i-- {
		ipStr := textproto.TrimString(items[i])
		ip := net.ParseIP(ipStr)
		if ip == nil {
			break
		}
		if (i == 0) || (!isTrustedProxy(ip)) {
			return ipStr, true
		}
	}
	return "", false
}

func parseAcceptOriginal(acceptHeader string) []string {
	parts := strings.Split(acceptHeader, ",")
	out := make([]string, 0, len(parts))
	for _, part := range parts {
		if i := strings.IndexByte(part, ';'); i > 0 {
			part = part[:i]
		}
		if part = strings.TrimSpace(part); part != "" {
			out = append(out, part)
		}
	}
	return out
}

func parseAcceptOptimized(acceptHeader string) []string {
	parts := strings.Split(acceptHeader, ",")
	out := make([]string, 0, len(parts))
	for _, part := range parts {
		if i := strings.IndexByte(part, ';'); i > 0 {
			part = part[:i]
		}
		if part = textproto.TrimString(part); part != "" {
			out = append(out, part)
		}
	}
	return out
}

func isTrustedProxy(_ net.IP) bool {
	// For benchmarking, assume all proxies are untrusted
	return false
}

func BenchmarkValidateHeader(b *testing.B) {
	header := " 192.168.1.1 ,  10.0.0.1 , 172.16.0.1 "

	b.Run("original", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			validateHeaderOriginal(header)
		}
	})

	b.Run("optimized", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			validateHeaderOptimized(header)
		}
	})
}

func BenchmarkParseAccept(b *testing.B) {
	accept := "text/html ; q=0.9 , application/json , image/png ; q=0.8"

	b.Run("original", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			parseAcceptOriginal(accept)
		}
	})

	b.Run("optimized", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			parseAcceptOptimized(accept)
		}
	})
}

validateHeader benchmark result

goos: darwin
goarch: amd64
pkg: github.com/gin-gonic/gin/test/trimespace
cpu: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
BenchmarkValidateHeader
BenchmarkValidateHeader/original
BenchmarkValidateHeader/original-8         	10694073	       112.3 ns/op
BenchmarkValidateHeader/optimized
BenchmarkValidateHeader/optimized-8        	 9857858	       106.4 ns/op
PASS

Process finished with the exit code 0

parseAccept benchmark result

goos: darwin
goarch: amd64
pkg: github.com/gin-gonic/gin/test/trimespace
cpu: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
BenchmarkParseAccept
BenchmarkParseAccept/original
BenchmarkParseAccept/original-8         	 8826144	       137.4 ns/op
BenchmarkParseAccept/optimized
BenchmarkParseAccept/optimized-8        	 8665436	       128.1 ns/op
PASS

Process finished with the exit code 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant