feat: render with multiple files and debug mode

Signed-off-by: shawn <xw2002121@gmail.com> feat: add subcommand clean Signed-off-by: shawn <xw2002121@gmail.com> refactor: rm apache thrift in internal/mocks (cloudwego#1474) also: fix(server): invoker return err if no apache codec fix(server): listening on loopback addr chore: remove github.com/stretchr/testify direct dependency (cloudwego#1475) chore: upgrade gopkg to v0.1.0 (cloudwego#1477) feat: add kitex_render_meta.yaml for render Signed-off-by: shawn <xw2002121@gmail.com> feat: add remote repo for render and middlewares Signed-off-by: shawn <xw2002121@gmail.com> chore(generic): add generic base using gopkg base (cloudwego#1482) fix(gonet): adjust gonet server read timeout to avoid read error (cloudwego#1481) fix: remove redundant line Signed-off-by: shawn <xw2002121@gmail.com> fix: fix single command Signed-off-by: shawn <xw2002121@gmail.com> refactor: use github.com/cloudwego/gopkg/protocol/thrift/apache (cloudwego#1483) * added README.md for pkg/protocol/bthrift * pkg/protocol/bthrift: not import kitex other pkgs * pkg/protocol/bthrift/apache: clear up -> thrift.go * pkg/generic: no apache codec * pkg/remote/codec/thrift: deprecate BinaryProtocol * pkg/utils: deprecate apache codec * tool: add `thrift_import_path` by default feat: add template render subcommand using .tpl Signed-off-by: shawn <xw2002121@gmail.com> fix: fix some issue Signed-off-by: shawn <xw2002121@gmail.com> perf: custom allocator for fast codec ReadString/ReadBinary (cloudwego#1427) chore: remove useless reflection api (cloudwego#1433) optimize(lb): rebalance when instance weights updated (cloudwego#1397) fix: support setting PurePayload Transport Protocol (cloudwego#1436) bug fixes: fix render Signed-off-by: shawn <xw2002121@gmail.com> bug fixes: fix bug in generator_test Signed-off-by: shawn <xw2002121@gmail.com> bug fixes: add gen-path Signed-off-by: shawn <xw2002121@gmail.com> bug fixes: treat option as unknown command Signed-off-by: shawn <xw2002121@gmail.com> bug fix: fix unused code in command and flag Signed-off-by: shawn <xw2002121@gmail.com> bug fixes: fix lint Signed-off-by: shawn <xw2002121@gmail.com> fix: use gofumpt -extra to fix golanci lint Signed-off-by: shawn <xw2002121@gmail.com> fix: use gofumpt to avoid golangci lint Signed-off-by: shawn <xw2002121@gmail.com> add InitType perf: custom allocator for fast codec ReadString/ReadBinary (cloudwego#1427) chore: remove useless reflection api (cloudwego#1433) optimize(lb): rebalance when instance weights updated (cloudwego#1397) fix: support setting PurePayload Transport Protocol (cloudwego#1436) fix: fix a bug "unknown service xxx" when using CombineService client by not writing IDLServiceName and searching serviceInfo by method name (cloudwego#1402) fix: support setting PurePayload with new style (cloudwego#1438) refactor(multi-services): refactoring service and method routing for multi-services (cloudwego#1439) refactor: move apache code to separated pkg (cloudwego#1381) Co-authored-by: QihengZhou <qiheng.zhou@bytedance.com> Co-authored-by: Yi Duan <duanyi.aster@bytedance.com> refactor(generic): refactor existing generic to have new ServiceInfo which has the generic's reader and writer info directly (cloudwego#1408) chore: upgrade go directive version to 1.17 of go.mod (cloudwego#1415) refactor: get rid of apache TApplicationException (cloudwego#1389) feat(generic): support grpc json generic for client (cloudwego#1411) feat: add PrependError for thriftgo (cloudwego#1420) feat(thrift): generic fastcodec (cloudwego#1424) feat(tool): no apache for fastcodec (cloudwego#1426) * use bthrift.PrependError * updated kitex tool MinDepVersion to 0.11.0 * fixed `undefined: bthrift.KitexUnusedProtection` chore: fixed undefined KitexUnusedProtection (cloudwego#1428) test: works without apache code (cloudwego#1429) chore: update CI version and readme community (cloudwego#1431) refactor: new generic interface without thrift apache (cloudwego#1434) fix(generic): fix payload length check for http generic (cloudwego#1442) chore: pick and fix conflict commits from develop branch (cloudwego#1457) Co-authored-by: Jayant <xiezhengyao@bytedance.com> chore: update dependency (cloudwego#1447) chore(ci): disable cache for lint and staticchecks (cloudwego#1451) refactor(test): perf optimize and log loc correct (cloudwego#1455) chore(ci): speed up multiple ci processes 8min -> 1min (cloudwego#1454) * rm unused codeconv, it didn't work as expected due to quota and user experience * don't use cache for self-hosted runners * cache for github hosted runners hotfix: multi service registry issue chore(ci): optimized bench tests. it takes <1m now (cloudwego#1461) chore(test): fix xorshift64 in consist_test.go (cloudwego#1462) chore: fix grpc keepalive test by start server responsiblly (cloudwego#1463) refactor: deprecate bthrift, use cloudwego/gopkg (cloudwego#1441) * tool: generates code for using cloudwego/gopkg * internal/mocks: updated thrift using the latest tool * pkg/utils/fastthrift: moved to cloudwego/gopkg * pkg/remote/codec/thrift: uses skipdecoder of cloudwego/gopkg * pkg/remote/codec/thrift: add fastcodec as a fallback, and always use fastcodec for Ex * pkg/generic: uses cloudwego/gopkg for pkg/protocol/bthrift: * type BinaryWriter = gopkgthrift.NocopyWriter * Removed ThriftFastCodec, moved to cloudwego/gopkg before releasing * Removed bthrift/exception.go, moved to cloudwego/gopkg before releasing * Removed bthrift/test, only for unknownfields testing * bthrift/unknown: moved to cloudwego/gopkg refactor(generic): remove apache thrift.TProtocol from generic (cloudwego#1450) add test for command and flag add test for command and flag resolve conflicts refactor: rm apache thrift in pkg/generic & netpollmux (cloudwego#1470) fix: allow HEADERS frame with empty header block fragment (cloudwego#1466) refactor: rm apache thrift in internal/mocks (cloudwego#1474) also: fix(server): invoker return err if no apache codec fix(server): listening on loopback addr chore: remove github.com/stretchr/testify direct dependency (cloudwego#1475) chore: upgrade gopkg to v0.1.0 (cloudwego#1477) perf: custom allocator for fast codec ReadString/ReadBinary (cloudwego#1427) resolve conflicts feat: add init Signed-off-by: shawn <xw2002121@gmail.com> feat: render with multiple files and debug mode Signed-off-by: shawn <xw2002121@gmail.com> feat: add subcommand clean Signed-off-by: shawn <xw2002121@gmail.com> feat: add kitex_render_meta.yaml for render Signed-off-by: shawn <xw2002121@gmail.com> feat: add remote repo for render and middlewares Signed-off-by: shawn <xw2002121@gmail.com> chore(generic): add generic base using gopkg base (cloudwego#1482) fix(gonet): adjust gonet server read timeout to avoid read error (cloudwego#1481) fix: remove redundant line Signed-off-by: shawn <xw2002121@gmail.com> fix: fix single command Signed-off-by: shawn <xw2002121@gmail.com> feat: add userDefinedMiddleware Signed-off-by: shawn <xw2002121@gmail.com> feat: add middleware Signed-off-by: shawn <xw2002121@gmail.com>
ShawnJeffersonWang · Aug 7, 2024 · 7e5b6c7 · 7e5b6c7
1 parent af1949e
commit 7e5b6c7
Show file tree

Hide file tree

Showing 9 changed files with 530 additions and 164 deletions.
diff --git a/tool/cmd/kitex/args/tpl_args.go b/tool/cmd/kitex/args/tpl_args.go
@@ -16,12 +16,15 @@ package args
 
 import (
 	"fmt"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"strings"
+
 	"github.com/cloudwego/kitex/tool/internal_pkg/generator"
 	"github.com/cloudwego/kitex/tool/internal_pkg/log"
 	"github.com/cloudwego/kitex/tool/internal_pkg/tpl"
 	"github.com/cloudwego/kitex/tool/internal_pkg/util"
-	"os"
-	"path/filepath"
 )
 
 // Constants .
@@ -40,7 +43,7 @@ const (
 	ServiceFileName     = "*service.go"
 	ExtensionFilename   = "extensions.yaml"
 
-	ClientMockFilename = "client_mock.go"
+	MultipleServicesFileName = "multiple_services.go"
 )
 
 var defaultTemplates = map[string]string{
@@ -55,35 +58,30 @@ var defaultTemplates = map[string]string{
 	ServiceFileName:     tpl.ServiceTpl,
 }
 
-var mockTemplates = map[string]string{
-	ClientMockFilename: tpl.ClientMockTpl,
+var multipleServicesTpl = map[string]string{
+	MultipleServicesFileName: tpl.MultipleServicesTpl,
 }
 
 const (
-	DefaultType = "default"
-	MockType    = "mock"
+	DefaultType          = "default"
+	MultipleServicesType = "multiple_services"
 )
 
 type TemplateGenerator func(string) error
 
 var genTplMap = map[string]TemplateGenerator{
-	DefaultType: GenTemplates,
-	MockType:    GenMockTemplates,
+	DefaultType:          GenTemplates,
+	MultipleServicesType: GenMultipleServicesTemplates,
 }
 
 // GenTemplates is the entry for command kitex template,
 // it will create the specified path
 func GenTemplates(path string) error {
-	for key := range defaultTemplates {
-		if key == BootstrapFileName {
-			defaultTemplates[key] = util.JoinPath(path, "script", BootstrapFileName)
-		}
-	}
 	return InitTemplates(path, defaultTemplates)
 }
 
-func GenMockTemplates(path string) error {
-	return InitTemplates(path, mockTemplates)
+func GenMultipleServicesTemplates(path string) error {
+	return InitTemplates(path, multipleServicesTpl)
 }
 
 // InitTemplates creates template files.
@@ -92,8 +90,18 @@ func InitTemplates(path string, templates map[string]string) error {
 		return err
 	}
 
-	for k, v := range templates {
-		if err := createTemplate(filepath.Join(path, k+".tpl"), v); err != nil {
+	for name, content := range templates {
+		var filePath string
+		if name == BootstrapFileName {
+			bootstrapDir := filepath.Join(path, "script")
+			if err := MkdirIfNotExist(bootstrapDir); err != nil {
+				return err
+			}
+			filePath = filepath.Join(bootstrapDir, name+".tpl")
+		} else {
+			filePath = filepath.Join(path, name+".tpl")
+		}
+		if err := createTemplate(filePath, content); err != nil {
 			return err
 		}
 	}
@@ -159,15 +167,54 @@ func (a *Arguments) Init(cmd *util.Command, args []string) error {
 	return nil
 }
 
-func (a *Arguments) Render(cmd *util.Command, args []string) error {
+func (a *Arguments) checkTplArgs() error {
+	if a.TemplateDir != "" && a.RenderTplDir != "" {
+		return fmt.Errorf("template render --dir and -template-dir cannot be used at the same time")
+	}
+	if a.RenderTplDir != "" && len(a.TemplateFiles) > 0 {
+		return fmt.Errorf("template render --dir and --file option cannot be specified at the same time")
+	}
+	return nil
+}
+
+func (a *Arguments) Root(cmd *util.Command, args []string) error {
 	curpath, err := filepath.Abs(".")
 	if err != nil {
 		return fmt.Errorf("get current path failed: %s", err.Error())
 	}
-	if len(args) < 2 {
-		return fmt.Errorf("both template directory and idl is required")
+	log.Verbose = a.Verbose
+
+	for _, e := range a.extends {
+		err := e.Check(a)
+		if err != nil {
+			return err
+		}
+	}
+
+	err = a.checkIDL(args)
+	if err != nil {
+		return err
+	}
+	err = a.checkServiceName()
+	if err != nil {
+		return err
+	}
+	err = a.checkTplArgs()
+	if err != nil {
+		return err
+	}
+	// todo finish protobuf
+	if a.IDLType != "thrift" {
+		a.GenPath = generator.KitexGenPath
+	}
+	return a.checkPath(curpath)
+}
+
+func (a *Arguments) Template(cmd *util.Command, args []string) error {
+	curpath, err := filepath.Abs(".")
+	if err != nil {
+		return fmt.Errorf("get current path failed: %s", err.Error())
 	}
-	a.RenderTplDir = args[0]
 	log.Verbose = a.Verbose
 
 	for _, e := range a.extends {
@@ -177,22 +224,26 @@ func (a *Arguments) Render(cmd *util.Command, args []string) error {
 		}
 	}
 
-	err = a.checkIDL(cmd.Flags().Args()[1:])
+	err = a.checkIDL(args)
 	if err != nil {
 		return err
 	}
 	err = a.checkServiceName()
 	if err != nil {
 		return err
 	}
+	err = a.checkTplArgs()
+	if err != nil {
+		return err
+	}
 	// todo finish protobuf
 	if a.IDLType != "thrift" {
 		a.GenPath = generator.KitexGenPath
 	}
 	return a.checkPath(curpath)
 }
 
-func (a *Arguments) Clean(cmd *util.Command, args []string) error {
+func (a *Arguments) Render(cmd *util.Command, args []string) error {
 	curpath, err := filepath.Abs(".")
 	if err != nil {
 		return fmt.Errorf("get current path failed: %s", err.Error())
@@ -206,29 +257,68 @@ func (a *Arguments) Clean(cmd *util.Command, args []string) error {
 		}
 	}
 
-	err = a.checkIDL(cmd.Flags().Args())
+	err = a.checkIDL(args)
 	if err != nil {
 		return err
 	}
 	err = a.checkServiceName()
 	if err != nil {
 		return err
 	}
+	err = a.checkTplArgs()
+	if err != nil {
+		return err
+	}
 	// todo finish protobuf
 	if a.IDLType != "thrift" {
 		a.GenPath = generator.KitexGenPath
 	}
 	return a.checkPath(curpath)
 }
 
-func (a *Arguments) TemplateArgs(version, curpath string) error {
+func (a *Arguments) Clean(cmd *util.Command, args []string) error {
+	curpath, err := filepath.Abs(".")
+	if err != nil {
+		return fmt.Errorf("get current path failed: %s", err.Error())
+	}
+
+	magicString := "// Kitex template debug file. use template clean to delete it."
+	err = filepath.WalkDir(curpath, func(path string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		content, err := os.ReadFile(path)
+		if err != nil {
+			return fmt.Errorf("read file %s faild: %v", path, err)
+		}
+		if strings.Contains(string(content), magicString) {
+			if err := os.Remove(path); err != nil {
+				return fmt.Errorf("delete file %s failed: %v", path, err)
+			}
+		}
+		return nil
+	})
+	if err != nil {
+		return fmt.Errorf("error cleaning debug template files: %v", err)
+	}
+	fmt.Println("clean debug template files successfully...")
+	os.Exit(0)
+	return nil
+}
+
+func (a *Arguments) TemplateArgs(version string) error {
 	kitexCmd := &util.Command{
 		Use:   "kitex",
 		Short: "Kitex command",
+		RunE:  a.Root,
 	}
 	templateCmd := &util.Command{
 		Use:   "template",
 		Short: "Template command",
+		RunE:  a.Template,
 	}
 	initCmd := &util.Command{
 		Use:   "init",
@@ -245,42 +335,72 @@ func (a *Arguments) TemplateArgs(version, curpath string) error {
 		Short: "Clean command",
 		RunE:  a.Clean,
 	}
+	kitexCmd.Flags().StringVar(&a.GenPath, "gen-path", generator.KitexGenPath,
+		"Specify a code gen path.")
+	templateCmd.Flags().StringVar(&a.GenPath, "gen-path", generator.KitexGenPath,
+		"Specify a code gen path.")
 	initCmd.Flags().StringVarP(&a.InitOutputDir, "output", "o", ".", "Specify template init path (default current directory)")
 	initCmd.Flags().StringVarP(&a.InitType, "type", "t", "", "Specify template init type")
-	renderCmd.Flags().StringVarP(&a.ModuleName, "module", "m", "",
+	renderCmd.Flags().StringVar(&a.RenderTplDir, "dir", "", "Use custom template to generate codes.")
+	renderCmd.Flags().StringVar(&a.ModuleName, "module", "",
 		"Specify the Go module name to generate go.mod.")
 	renderCmd.Flags().StringVar(&a.IDLType, "type", "unknown", "Specify the type of IDL: 'thrift' or 'protobuf'.")
 	renderCmd.Flags().StringVar(&a.GenPath, "gen-path", generator.KitexGenPath,
 		"Specify a code gen path.")
-	renderCmd.Flags().StringVarP(&a.TemplateFile, "file", "f", "", "Specify single template path")
-	renderCmd.Flags().VarP(&a.Includes, "Includes", "I", "Add IDL search path and template search path for includes.")
-	initCmd.SetUsageFunc(func() {
-		fmt.Fprintf(os.Stderr, `Version %s
-Usage: kitex template init [flags]
+	renderCmd.Flags().StringArrayVar(&a.TemplateFiles, "file", []string{}, "Specify single template path")
+	renderCmd.Flags().BoolVar(&a.DebugTpl, "debug", false, "turn on debug for template")
+	renderCmd.Flags().StringVarP(&a.IncludesTpl, "Includes", "I", "", "Add IDL search path and template search path for includes.")
+	renderCmd.Flags().StringVar(&a.MetaFlags, "meta", "", "Meta data in key=value format, keys separated by ';' values separated by ',' ")
+	templateCmd.SetHelpFunc(func(*util.Command, []string) {
+		fmt.Fprintln(os.Stderr, `
+Template operation
+
+Usage:
+  kitex template [command]
+
+Available Commands:
+  init        Initialize the templates according to the type
+  render      Render the template files 
+  clean       Clean the debug templates
+	`)
+	})
+	initCmd.SetHelpFunc(func(*util.Command, []string) {
+		fmt.Fprintln(os.Stderr, `
+Initialize the templates according to the type
 
-Examples:
-  kitex template init -o /path/to/output
-  kitex template init
+Usage:
+  kitex template init [flags]
 
 Flags:
-`, version)
+  -o, --output string        Output directory
+  -t, --type   string        The init type of the template
+	`)
 	})
-	renderCmd.SetUsageFunc(func() {
-		fmt.Fprintf(os.Stderr, `Version %s
-Usage: template render [template dir_path] [flags] IDL
-`, version)
-	})
-	cleanCmd.SetUsageFunc(func() {
-		fmt.Fprintf(os.Stderr, `Version %s
-Usage: kitex template clean
+	renderCmd.SetHelpFunc(func(*util.Command, []string) {
+		fmt.Fprintln(os.Stderr, `
+Render the template files 
 
-Examples:
-  kitex template clean
+Usage:
+  kitex template render [flags]
 
 Flags:
-`, version)
+  --dir          string         Output directory
+  --debug        bool           Turn on the debug mode
+  --file         stringArray    Specify multiple files for render
+  -I, --Includes string         Add an template git search path for includes.
+  --meta         string         Specify meta data for render
+  --module       string         Specify the Go module name to generate go.mod.
+  -t, --type     string         The init type of the template
+	`)
+	})
+	cleanCmd.SetHelpFunc(func(*util.Command, []string) {
+		fmt.Fprintln(os.Stderr, `
+Clean the debug templates
+
+Usage:
+  kitex template clean
+	`)
 	})
-	// renderCmd.PrintUsage()
 	templateCmd.AddCommand(initCmd, renderCmd, cleanCmd)
 	kitexCmd.AddCommand(templateCmd)
 	if _, err := kitexCmd.ExecuteC(); err != nil {

diff --git a/tool/cmd/kitex/main.go b/tool/cmd/kitex/main.go
@@ -77,9 +77,9 @@ func main() {
 		log.Warn("Get current path failed:", err.Error())
 		os.Exit(1)
 	}
-	if os.Args[1] == "template" {
-		err = args.TemplateArgs(kitex.Version, curpath)
-	} else if !strings.HasPrefix(os.Args[1], "-") {
+	if len(os.Args) > 1 && os.Args[1] == "template" {
+		err = args.TemplateArgs(kitex.Version)
+	} else if len(os.Args) > 1 && !strings.HasPrefix(os.Args[1], "-") {
 		err = fmt.Errorf("unknown command %q", os.Args[1])
 	} else {
 		// run as kitex

diff --git a/tool/internal_pkg/generator/custom_template.go b/tool/internal_pkg/generator/custom_template.go
@@ -254,7 +254,7 @@ func parseMeta(metaFlags string) (map[string][]string, error) {
 			values := strings.Split(kv[1], ",")
 			meta[key] = values
 		} else {
-			return nil, fmt.Errorf("Invalid meta format: %s\n", pair)
+			return nil, fmt.Errorf("invalid meta format: %s", pair)
 		}
 	}
 	return meta, nil
@@ -354,9 +354,8 @@ const kitexRenderMetaFile = "kitex_render_meta.yaml"
 
 // Meta represents the structure of the kitex_render_meta.yaml file.
 type Meta struct {
-	Templates  []Template             `yaml:"templates"`
-	MWs        []MiddlewareForResolve `yaml:"middlewares"`
-	ExtendMeta []ExtendMeta           `yaml:"extend_meta"`
+	Templates []Template             `yaml:"templates"`
+	MWs       []MiddlewareForResolve `yaml:"middlewares"`
 }
 
 type MiddlewareForResolve struct {
@@ -366,10 +365,6 @@ type MiddlewareForResolve struct {
 	Path string `yaml:"path"`
 }
 
-type ExtendMeta struct {
-	key string
-}
-
 func readMetaFile(metaPath string) (*Meta, error) {
 	metaData, err := os.ReadFile(metaPath)
 	if err != nil {