New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can we call a controller method as scheduled(e.g. using crontab)? #1165

Open
zheeeng opened this Issue Jan 4, 2019 · 13 comments

Comments

Projects
None yet
2 participants
@zheeeng
Copy link

zheeeng commented Jan 4, 2019

I want to call an API as scheduled. Is there any solution to exec some API once timeout?

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 5, 2019

Or is there any way I can build API trigger callback to feed to a crontab instance?

@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 5, 2019

Hello, are you talking about SSE? If not, could you, please, give me an API code snippet example that you can imagine of this usage so I can implement it?

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 5, 2019

Hello, are you talking about SSE? If not, could you, please, give me an API code snippet example that you can imagine of this usage so I can implement it?

I'm coming from NodeJS community and I used to use built-in scheduling solution of Eggjs and Thinkjs. They have different implementation. I can't find a built-in solution in Iris.

I made this feature by myself through this solution:

// cron.go
package cron

import (
	"fmt"

	"github.com/kataras/iris"
	"github.com/robfig/cron"
)

var cronInstace *cron.Cron

var taskFunc = make(map[string]func())

func GetCrontabInstance() *cron.Cron {
	if cronInstace != nil {
		return cronInstace
	}
	cronInstace = cron.New()
	cronInstace.Start()

	iris.RegisterOnInterrupt(func() {
		cronInstace.Stop()
	})
	return cronInstace
}

func AddTaskFuc(name string, schedule string, f func()) {
	if _, ok := taskFunc[name]; !ok {
		fmt.Println("Add a new task:", name)

		cInstance := GetCrontabInstance()
		cInstance.AddFunc(schedule, f)

		taskFunc[name] = f
	} else {
		fmt.Println("Don't add same task `" + name + "` repeatedly!")
	}
}
// task.go
package cron

import (
	"bytes"
	"net/http"
	"os"
)

func init() {
	AddTaskFuc(
		"pushWechatTemplatedMessage",
		"15 * * * * *",
		func() {
			port := os.Getenv("PORT")
			if port == "" {
				port = "8080"
			}

			client := &http.Client{}
			req, _ := http.NewRequest(
				"POST",
				"http://localhost:"+port+"/api/v1/somepath",
				bytes.NewReader([]byte(`requestBody`)),
			)

			client.Do(req)
		},
	)
}
// main.go
package main

import (
	_ "pathToProject/cron"
}

// blabla

There is an annoying problem that I can't touch the context and call API directly. Thinkjs and Eggjs provided us with this feature.

Moreover, to handle the situation that app may be deployed in many machines, I have to use some distributed solutions to prevent calling API multiple times at the scheduled time.

@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 6, 2019

There is an annoying problem that I can't touch the context and call API directly

You can touch the Context, you can implement and change its methods as well, see the _examples/routing/custom-context.

You can also call another route action within a handler, see the context#Exec function, example can be found at: _examples/routing/route-state/main.go#L36.

However, I don't think that these will help you with the issue you are describing. I saw the ThinkJS and Eggjs but they are not doing anything special that is http-relative, you have the github.com/robfig/cron package and you can use it to call a handler, a handler is just a function, or you can send an http request as you do on your code snippet. In other hand, if you have some clients waiting for updates based on cron you can use SSE, this is why it exists and it's 100% http-relative, an example of SSE and brokers can be found at: _examples/http_responsewriter/sse/main.go.

If I don't understand correctly, continue your code snippet with the iris-side, so far I didn't see how you imagine Iris to help you with this (remember: handlers are functions, you can call them to perform a re-cache of the database for example). Thank you a lot. Waiting for more information, I want to help you, I also saw that you posted somewhere else for that but don't expect an answer from them, no need for this, I am here for you.

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 6, 2019

Thanks a lot for your reply.

I'm looking for a built-in scheduling mechanism. I knew the usage of context#Exec and learned a lot from custom context and SSE's implementation.
I have implemented a controller handler that will call a third party API. Now I want it to be called periodically

At present, I made my feature by using cron as the snippet above does. The sad side is that I feel it is non-relative about the framework. I have to import a net package to call some API from localhost plus its API route.

I can call this.ctx.service.db.cleandb() in EggJS, and declare a task througth add config:

module.exports = [{
  cron: '0 */1 * * *',
  handle: 'crontab/test',
  type: 'all'
}]

They both prevent me from calling API by send a http request. A more elegent solution is like what spring cron does:

@Scheduled(fixedRate=5000)
public void doSomething() {
    // something that should execute periodically
}

I wonder have a way to declare the handler's scheduing. Or, there have a method on app instance that could call a hanlder by passing a constructed a mock request to trigger.

import (
    "net/http"
    "iris"
)

func main () {
    app := iris.New()
    cron.AddTaskFunc('1 * * * * *', func () {
        // notice here, the url is a route path
        mockRequest := http.NewRequest("GET", "/api/v1/somepath", nil)
        app.Exec(mockRequest)
    })
}
@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 6, 2019

I see @zheeeng, you can already do that by using the app.ServeHTTP, here is a fully and working example:

package main

import (
	"fmt"
	"net/http"

	"github.com/kataras/iris"
	"github.com/robfig/cron"
)

var cronInstace *cron.Cron

var taskFunc = make(map[string]func())

func GetCrontabInstance() *cron.Cron {
	if cronInstace != nil {
		return cronInstace
	}
	cronInstace = cron.New()
	cronInstace.Start()

	iris.RegisterOnInterrupt(func() {
		cronInstace.Stop()
	})
	return cronInstace
}

func AddTaskFunc(name string, schedule string, f func()) {
	if _, ok := taskFunc[name]; !ok {
		fmt.Println("Add a new task:", name)

		cInstance := GetCrontabInstance()
		cInstance.AddFunc(schedule, f)

		taskFunc[name] = f
	} else {
		fmt.Println("Don't add same task `" + name + "` repeatedly!")
	}
}

type cronResponseWriter struct {
	headers http.Header
	body    []byte
	code    int
}

func newCronResponseWriter() *cronResponseWriter {
	return &cronResponseWriter{
		headers: make(http.Header),
		code:    iris.StatusOK,
	}
}

func (w *cronResponseWriter) Header() http.Header { return w.headers }

func (w *cronResponseWriter) Write(body []byte) (int, error) {
	w.body = body
	return len(body), nil
}

func (w *cronResponseWriter) WriteHeader(code int) {
	w.code = code
}

func main() {
	app := iris.New()
	AddTaskFunc("example scheduled task", "@every 5s", func() {
		mockRequest, _ := http.NewRequest("GET", "/api/v1/somepath?reftype=cron", nil)
		app.ServeHTTP(newCronResponseWriter(), mockRequest)
	})

	app.Get("/api/v1/somepath", func(ctx iris.Context) {
		// if _, ok := ctx.ResponseWriter().Naive().(*cronResponseWriter); ok {
		//  // do cron-relative things...
		//  app.Logger().Infof("cron: path executed: \"%s\"", ctx.Path())
		//  return
		// }
		//
		// You can also make use of a url parameter to see
		// who is the client which calling this route (this is totally optionally ofc),
		// see below:
		if reftype := ctx.URLParam("reftype"); reftype == "cron" {
			// do cron-relative things...
			app.Logger().Infof("cron: path executed: \"%s\"", ctx.Path())
			return
		}

		// ...
		ctx.Writef("from web")
	})

	app.Run(iris.Addr(":8080"))
}

Tell me if that works for you!

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 7, 2019

Thx for your help, it works as expected!

@kataras kataras added the question label Jan 7, 2019

@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 7, 2019

You are very welcomed! Do you want from Iris to give a type of response writer like this and provide some basic tooling over cron vs web serve-time?

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 8, 2019

I personally expect to have a convenient response-writer-maker. Or a wrapped func for the Applicatoin::ServeHTTP, we can use it like app.Execute(method, header, requestBody).

@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 8, 2019

I personally expect to have a convenient response-writer-maker

About the response writer, it's also easy, we have response writer maker thing you mention, we have a response writer and response recorder makers inside the iris/context package. They can be used as well. But you need a way to tell if that comes from the cron task without any request body changes or headers or url, so a new type is the best next thing. We could add a "cron response writer", but it's just 10 lines of code and user has the ability to do more things if she/he implement by her/his own, it's better to keep accepting a general http.ResponseWriter interface and let the user decide what response writer want to use and what need to check inside the handler to tell if that comes from a cron task or the protocol.

Or a wrapped func for the Applicatoin::ServeHTTP

You can still make wrapped func for the Application#ServeHTTP, see examples in the iris/core/router package, we have plenty of them.

we can use it like app.Execute(method, header, requestBody).

No, this will limit the usecases, it's better to accept a *http.Request so you can handle different use cases, it's not hard to create a new request with a body, just http.NewRequest.

Will think further of it tomorrow and explore what it's easier but also keeps all "advanced/low-level" features for experienced users.

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 17, 2019

@kataras Sorry for bothering you again, I need some help on a problem likes using app. ServeHTTP

I'm made a proxy service and pass down the ctx.ResponseWriter() and ctx.Request() to proxy. And I want to modify the proxied response:

// A controller handle
func RewriteResponse(resp *http.Response) (err error) {
        // rewrite response
        // ...

        return nil
}

func (c *controller) PostUpload() {
	if u, err := url.Parse("http://api.somewhereforuploading.com/"); err != nil {
            log.Fatal(err)
	} else {
		proxy := httputil.NewSingleHostReverseProxy(u)
		proxy.ModifyResponse = RewriteResponse
		proxy.ServeHTTP(ctx.ResponseWriter(), ctx.Request())
        }
}

The only place for manipulating response is the proxy.ModifyResponse callback and have no way to use context. Does it is possible that handle the context in another HTTPServer?

@zheeeng

This comment has been minimized.

Copy link

zheeeng commented Jan 18, 2019

Can we wait for the app.ServeHTTP are executed and manipulate the response just like using http.Get ?

@kataras

This comment has been minimized.

Copy link
Owner

kataras commented Jan 19, 2019

If you mean if you can get the *http.Response, I assume that you can do it by using a custom response recorder (see context/response_recorder.go that iris uses on ctx.Record() to record and modify a previous handler's response inside a next middleware) or you can use the std httptest.NewRecorder if you want so.

import "net/http/httptest"
// [...]
rec := httptest.NewRecorder()
app.ServeHTTP(rec, http.NewRequest(...))

However, I see here that you use the current request's request and response values, you can't just use the ctx.Recorder().GetBody/SetBody/... because the handler will reset it after the request-response cycle is done. Lucky for you, I made the response recorder available even outside of handlers, you can get a fresh response recorder by using the rec := context.AcquireResponseRecorder() and init it by rec.BeginRecord(ctx.ResponseWriter()), example (not-tested but it should work):

import "github.com/kataras/iris/context"
// [...]

rec := context.AcquireResponseRecorder()
rec.BeginRecord(ctx.ResponseWriter())
proxy.ServeHTTP(rec, ctx.Request())

// modify the response or whatever HERE
// [...]
//
// and when you done always run EndResponse
// to release this "object", we have a sync.Pool for these writers to reduce allocations:
rec.EndResponse() 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment