Add the bindata source as a target to the Makefile

[zeripath]

make backend fails if parallelism is turned on with make -jx.

This happens because when the executable starts to be built the sources
for the bindata files are not available.

If however, we add a target that makes the sources for the bindata -
i.e. the generate target - then parallelism will work again.

References: #11756

Signed-off-by: Andrew Thornton art27@cantab.net

[silverwind]

How about doing it the other way around and making a proper dependency chain? I think it'd be more in line with our other targets like frontend where that target is a simple phony target that has the actual files as dependencies.

.PHONY: generate
generate: $(BINDATA_DEST)

$(BINDATA_DEST): $(TAGS_PREREQ)

[zeripath]

Because generate does not and should not only produce BINDATA

[silverwind]

It should not matter if we specify all generated files or not. For example WEBPACK_DEST also only specifies two files while actually generating a lot more. Make will work fine if there's at least one correct target file.

[zeripath]

What we do not want is a situation whereby generate will only run if you are using bindata.

[silverwind]

generate is phony so will always run. You can even put it into $(EXECUTABLE), it would have the same effect as it does now.

[zeripath]

Damn we have to use .NOTPARALLEL

[silverwind]

Can we not just put the generate command into $(EXECUTABLE)? That will ensure it'll always run correctly, even when parallel I think.

[silverwind]

Ah wait, no that can't work because of the release tasks which expect generate to have ran.

[silverwind]

From what I gather .NOTPARALLEL does not take any arguments, just plain .NOTPARALLEL: anywhere in the makefile should disable parallelism for the whole makefile.

[silverwind]

Actually, .NOTPARALLEL is a no-go imho. It would prevent useful stuff like

make -j2 watch-frontend watch-backend

[silverwind]

So tar preserves 1-second precision timestamp on extraction, which is good enough for make to not trigger a frontend rebuild. It's also seen by running this from a src tarball extraction:

$ make webpack
make: Nothing to be done for 'webpack'.
$ make fomantic
make: Nothing to be done for 'fomantic'.

So I think the best solution is adding frontend to generate dependencies and move node-check to targets $(FOMANTIC_DEST) $(WEBPACK_DEST). It should only actually run the node build if web_src files are modified.

[silverwind]

So I got some success with below changes but there's no node-check in the chain anymore (it seems to trigger builds when its there). Note I reduced fomantic/webpack tracking files to 1 so make -j2 fomantic does not that build run twice in parallel.

diff --git a/Makefile b/Makefile
index 890332fde..baa54ec1e 100644
--- a/Makefile
+++ b/Makefile
@@ -89,14 +89,14 @@ LDFLAGS := $(LDFLAGS) -X "main.MakeVersion=$(MAKE_VERSION)" -X "main.Version=$(G

 GO_PACKAGES ?= $(filter-out code.gitea.io/gitea/integrations/migration-test,$(filter-out code.gitea.io/gitea/integrations,$(shell $(GO) list -mod=vendor ./... | grep -v /vendor/)))

 FOMANTIC_CONFIGS := semantic.json web_src/fomantic/theme.config.less web_src/fomantic/_site/globals/site.variables
-FOMANTIC_DEST := web_src/fomantic/build/semantic.js web_src/fomantic/build/semantic.css
+FOMANTIC_DEST := web_src/fomantic/build/semantic.js
 FOMANTIC_DEST_DIR := web_src/fomantic/build

 WEBPACK_SOURCES := $(shell find web_src/js web_src/less -type f) $(FOMANTIC_DEST)
 WEBPACK_CONFIGS := webpack.config.js
-WEBPACK_DEST := public/js/index.js public/css/index.css
+WEBPACK_DEST := public/js/index.js
 WEBPACK_DEST_ENTRIES := public/js public/css public/fonts public/img/webpack public/serviceworker.js

 BINDATA_DEST := modules/public/bindata.go modules/options/bindata.go modules/templates/bindata.go
 BINDATA_HASH := $(addsuffix .hash,$(BINDATA_DEST))
@@ -519,18 +519,18 @@ install: $(wildcard *.go)
 .PHONY: build
 build: frontend backend

 .PHONY: frontend
-frontend: node-check $(FOMANTIC_DEST) $(WEBPACK_DEST)
+frontend: $(FOMANTIC_DEST) $(WEBPACK_DEST)

 .PHONY: backend
-backend: go-check generate $(EXECUTABLE)
+backend: go-check $(EXECUTABLE)

 .PHONY: generate
-generate: $(TAGS_PREREQ)
+generate: frontend $(TAGS_PREREQ)
 	CC= GOOS= GOARCH= $(GO) generate -mod=vendor -tags '$(TAGS)' $(GO_PACKAGES)

-$(EXECUTABLE): $(GO_SOURCES) $(TAGS_PREREQ)
+$(EXECUTABLE): generate $(GO_SOURCES) $(TAGS_PREREQ)
 	CGO_CFLAGS="$(CGO_CFLAGS)" $(GO) build -mod=vendor $(GOFLAGS) $(EXTRA_GOFLAGS) -tags '$(TAGS)' -ldflags '-s -w $(LDFLAGS)' -o $@

 .PHONY: release
 release: frontend generate release-windows release-linux release-darwin release-copy release-compress release-sources release-docs release-check

[silverwind]

Regarding node-check and go-check, maybe they can be made a simple if condition that checks supplied make arguments so it won't mess up dependency tracking:

NODE_GOALS: frontend webpack fomantic

ifeq ($(filter $(NODE_GOALS),$(MAKECMDGOALS)),)
  # do node check
endif

[silverwind]

I'm questioning if it's really worth it to introduce the frontend dependency in the backend chain. For example the dev setup recommends frontend/backend ran in two tabs. Now if we have the dependency frontend will build in the backend tab if its unclean and then again in the frontend tab.

I think we're better off with a recommendation to not enable make parallism in README.md for the regular builds. There's nothing gained from enabling it anyways because all our steps depend on each other.