Possible to use inside an http.Handler? #4
Comments
|
Great to hear that! I believe it shouldn't be a problem running QuickJS inside Nonetheless, |
|
Thanks so much for your response. I wrote a quick little program to benchmark it and performance seems quite great. package main
import (
"fmt"
"net/http"
"strconv"
"time"
"github.com/lithdew/quickjs"
)
func main() {
http.ListenAndServe(":3000", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
one := time.Now()
js := quickjs.NewRuntime()
defer js.Free()
fmt.Println(time.Since(one))
two := time.Now()
context := js.NewContext()
defer context.Free()
fmt.Println(time.Since(two))
three := time.Now()
result, err := context.Eval(`1 + 2 * 100 * Math.random()`)
if err != nil {
fmt.Println(err)
return
}
defer result.Free()
fmt.Println(time.Since(three))
w.Write([]byte(strconv.Itoa(int(result.Int64()))))
}))
}Happily churns through 500/req/sec 😬 I was a bit surprised that I needed to stick Also since I'm not too familiar with |
|
The problem is that QuickJS allocates memory in the thread it's initialized within. As goroutines get scheduled and moved around to different threads, this means that a goroutine holding an instance to a Hence why The issue is demonstrated in this test that was made here which then led to Guideline 5 being established: https://github.com/lithdew/quickjs/blob/master/quickjs_test.go#L130 Removing Thinking about it, by the way, typically it is advised against creating a single runtime per HTTP request handled (doing so would cause possibly hundreds of runtimes being created, which are expensive to initialize). What could be done instead is creating a worker pool of |
|
Ah that makes more sense. I'll give that test a whirl! I'll also bench the worker pattern and post my results here. From what I saw it was actually creating the context that took the most time: Thanks for taking the time to share your knowledge! |
|
Hey there! I wanted to check-in after I did some additional research. I was able to get pooling working with the following code: quickjs.gopackage quickjs
import (
"context"
"fmt"
"runtime"
"github.com/lithdew/quickjs"
)
// NewWorker creates a new worker
// Worker is not goroutine safe
func NewWorker() *Worker {
runtime := quickjs.NewRuntime()
return &Worker{runtime}
}
// Worker struct
type Worker struct {
runtime quickjs.Runtime
}
// Eval function
func (w *Worker) Eval(ctx context.Context, js string, v interface{}) (err error) {
context := w.runtime.NewContext()
defer context.Free()
result, err := context.Eval(js)
if err != nil {
return err
}
switch t := v.(type) {
case *string: // we expect a string
if !result.IsString() {
return fmt.Errorf("Eval expected a string, but received %q", result)
}
*t = result.String()
result.Free()
return nil
case *int64:
if !result.IsNumber() {
return fmt.Errorf("Eval expected a number, but received %q", result)
}
*t = result.Int64()
result.Free()
return nil
case *bool:
if !result.IsBool() {
return fmt.Errorf("Eval expected a boolean, but received %q", result)
}
*t = result.Bool()
result.Free()
return nil
case nil: // ignore the output
result.Free()
return nil
default:
result.Free()
return fmt.Errorf("quickjs.Eval: unable to coerce into expected value %t", t)
}
}
// Close the worker
func (w *Worker) Close() {
w.runtime.Free()
}
// NewPool creates a pool of workers
func NewPool(size int) *Pool {
requestCh := make(chan request, size)
pool := &Pool{requestCh}
// start all available workers
for i := 0; i < size; i++ {
go pool.worker(i)
}
return pool
}
// Pool of workers
type Pool struct {
requestCh chan request
}
// TODO: sometimes goroutines get scheduled on the same thread.
// then they're locked to that thread. You can see this with
// -trace. Is there anyway to force the goroutines to get
// scheduled on different threads?
//
// Even better, is it possible to remove this requirement
// entirely? By cleaning up state after each run or something?
func (p *Pool) worker(id int) {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
w := NewWorker()
defer w.Close()
for req := range p.requestCh {
err := w.Eval(req.ctx, req.js, req.result)
req.response <- response{err}
}
}
type request struct {
ctx context.Context
js string
result interface{}
response chan response
}
type response struct {
err error
}
// Eval the worker
func (p *Pool) Eval(ctx context.Context, js string, result interface{}) (err error) {
response := make(chan response)
p.requestCh <- request{ctx, js, result, response}
res := <-response
return res.err
}
// Close the pool
func (p *Pool) Close() {
close(p.requestCh)
}quickjs_test.gopackage quickjs_test
import (
"context"
"fmt"
"runtime"
"testing"
"time"
"github.com/matthewmueller/duo/internal/quickjs"
"github.com/tj/assert"
"golang.org/x/sync/errgroup"
)
const slow = `
function slow(baseNumber){
let result = 0;
for (var i = Math.pow(10, 7); i >= 0; i--) {
result += Math.atan(i) * Math.tan(i);
};
return Math.floor(result)
}
slow(10)
`
func TestWorker(t *testing.T) {
ctx := context.Background()
var eg errgroup.Group
eg.Go(func() error {
worker := quickjs.NewWorker()
defer worker.Close()
var result int64
now := time.Now()
err := worker.Eval(ctx, slow, &result)
assert.NoError(t, err)
fmt.Println(result)
fmt.Println(time.Since(now))
var result2 int64
now2 := time.Now()
err2 := worker.Eval(ctx, slow, &result2)
assert.NoError(t, err2)
fmt.Println(time.Since(now2))
return nil
})
eg.Wait()
// assert.Equal(t, "11", v)
}
const slow = `
function slow(baseNumber){
let result = 0;
for (var i = Math.pow(10, 7); i >= 0; i--) {
result += Math.atan(i) * Math.tan(i);
};
return Math.floor(result)
}
slow(10)
`
func TestPool(t *testing.T) {
ctx := context.Background()
pool := quickjs.NewPool(runtime.NumCPU())
defer pool.Close()
var eg errgroup.Group
for i := 0; i < 204; i++ {
eg.Go(func() error {
var result int64
err := pool.Eval(ctx, slow, &result)
assert.NoError(t, err)
assert.Equal(t, int64(-2898551), result)
return nil
})
}
assert.NoError(t, eg.Wait())
}Overall, it's working quite fast, but I tested it with some CPU intensive tasks and I noticed that by using I was wondering:
Thanks! |
|
Hm, the only way you'd be able to circumvent goroutines being stuck on the same thread is by manually setting the goroutine's thread affinity using cgo. http://pythonwise.blogspot.com/2019/03/cpu-affinity-in-go.html The alternative is to also manually manage a thread pool over cgo. |
Thanks for putting this package together! It works great out of the box without any additional installation scripts.
I was wondering about Guideline 5.
Does this make QuickJS unsuitable to run inside
http.Handlers where you can have many goroutines servicing requests at the same time? The use-case in mind is server-side rendering.The text was updated successfully, but these errors were encountered: