Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add garble support #1793

Merged
merged 23 commits into from
Sep 13, 2022
Merged

Add garble support #1793

merged 23 commits into from
Sep 13, 2022

Conversation

leaanthony
Copy link
Member

@leaanthony leaanthony commented Aug 26, 2022

This PR adds support for the garble obfuscation tool. It generates different bindings in the wailsjs directory to support this.

Fixes #1634

CC @Osbornnnnn @aivaki

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Aug 26, 2022

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 654ca76
Status: ✅  Deploy successful!
Preview URL: https://0ec9bb72.wails.pages.dev
Branch Preview URL: https://feature-garble-support.wails.pages.dev

View logs

v2/internal/app/app_dev.go Outdated Show resolved Hide resolved
Copy link
Contributor

@Snider Snider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lookin goooood

Copy link
Contributor

@Snider Snider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :)

@leaanthony
Copy link
Member Author

CC @Osbornnnnn @aivaki
Looking for your feedback 👍

@aiwaki
Copy link

aiwaki commented Sep 3, 2022

CC @Osbornnnnn @aivaki Looking for your feedback 👍

Sorry I didn't answer for a long time. It's all cool, but I have no idea how to test it right now (how do I get a CLI?)
Or what is required of us at all? I really don't understand this until the end

@leaanthony
Copy link
Member Author

Please try the above PR to see if this fixes your problem. If you are unsure how to do this, please follow this guide. Let us know how you get on 👍

@aiwaki
Copy link

aiwaki commented Sep 4, 2022

Please try the above PR to see if this fixes your problem. If you are unsure how to do this, please follow this guide. Let us know how you get on 👍

I have installed the CLI, but...

Edited:
This is due to the -obfuscate flag

$ wails build -race -trimpath -obfuscate -m -tags="desktop,production" -compiler="garble" -garbleargs="-tiny -literals -seed=random" -ldflags="-w -s -H windowsgui"

Wails CLI v2.0.0-beta.44.2

App Type:               desktop
Platforms:              windows/amd64
Compiler:               C:\Users\makse\go\bin\garble.exe
Build Mode:             Production
Obfuscate:              true
Skip Frontend:          false
Compress:               false
Package:                true
Clean Build Dir:        false
LDFlags:                "-w -s -H windowsgui"
Tags:                   [desktop,production]
Race Detector:          true

Building target: windows/amd64
------------------------------
  - Generating obfuscated bindings:

ERROR:
# github.com/wailsapp/wails/v2/internal/frontend/runtime
vendor\github.com\wailsapp\wails\v2\internal\frontend\runtime\runtime_prod_desktop.go:9:5: RuntimeDesktopJS redeclared in this block
        vendor\github.com\wailsapp\wails\v2\internal\frontend\runtime\runtime_dev_desktop.go:9:5: other declaration of RuntimeDesktopJS

exit status 2

If Wails is useful to you or your company, please consider sponsoring the project:
https://github.com/sponsors/leaanthony


exit status 1

@aiwaki
Copy link

aiwaki commented Sep 4, 2022

The application interface breaks anyway, the controls don't work (when you click on the button, it doesn't do anything)
(i suppose it's because it compiled without -obfuscate)

garble flags don't work as they should, strings aren't hidden(

@leaanthony
Copy link
Member Author

@aiwaki
Copy link

aiwaki commented Sep 5, 2022

Did you follow the instructions? https://feature-garble-support.wails.pages.dev/docs/next/guides/obfuscated

For some reason, it just doesn't compile together with -obfuscate
I'll try to fix it today

@Osbornnnnn
Copy link

Osbornnnnn commented Sep 5, 2022

I tested it too. But have the same error.
image

@leaanthony
Copy link
Member Author

leaanthony commented Sep 5, 2022

That error suggests you are using the 44.3 release library in your projects. Update go.mod to use your local checkout. The instructions are here but not explicit in the sub-sections. I'll update the docs 👍https://wails.io/docs/next/guides/bleeding-edge

@aiwaki
Copy link

aiwaki commented Sep 6, 2022

That error suggests you are using the 44.3 release library in your projects. Update go.mod to use your local checkout. The instructions are here but not explicit in the sub-sections. I'll update the docs 👍https://wails.io/docs/next/guides/bleeding-edge

I have updated go.mod and other, but the error has not disappeared. I'll try again

@leaanthony
Copy link
Member Author

@Snider - did you have any issues with this?

@Snider
Copy link
Contributor

Snider commented Sep 6, 2022

@Snider - did you have any issues with this?

I trust your judgment; I have not checked out the PR and done a build test; would you like me to do this?

I'm happy with Wails' securing the user's application code; it is a good move for the users of Wails.io; excited to see the many ways we can put it to use.

I'll wait for confirmation, but I'm happy if the team is happy. +1

@leaanthony
Copy link
Member Author

Pushed fixes as suggested. Works locally. Thanks for taking the time to review this @stffabi 🙏

@Snider
Copy link
Contributor

Snider commented Sep 10, 2022

Pushed fixes as suggested. Works locally. Thanks for taking the time to review this @stffabi 🙏

Tested, and I get a working build.

Copy link
Collaborator

@stffabi stffabi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just added another comment which came to my mind.

v2/cmd/wails/internal/commands/dev/dev.go Outdated Show resolved Hide resolved
@stffabi
Copy link
Collaborator

stffabi commented Sep 10, 2022

Pushed fixes as suggested. Works locally. Thanks for taking the time to review this @stffabi 🙏

You're welcome.

@stffabi
Copy link
Collaborator

stffabi commented Sep 10, 2022

I think we are good to go. 👍

@leaanthony
Copy link
Member Author

I was doing some testing and unless my local install is borked, it looks like the generation of the bindings isn't working properly (sometimes leaves the unobfuscated versions and doesn't generate the new ones). I'm not going to have time to look at it for a short while so if anyone would like to test/debug, that would be awesome. Thank you 🙏

@AlbinoDrought
Copy link
Contributor

AlbinoDrought commented Sep 10, 2022

I can't reproduce regular bindings not being replaced with obfuscated bindings, but I can reproduce the opposite:

  • wails build -obfuscate: frontend/wailsjs/go/main/App.js replaced with version using ObfuscatedCall
  • wails build: frontend/wailsjs/go/main/App.js not changed, still using ObfuscatedCall
Reproducible ObfuscatedCall bindings remaining
mkdir wails-obfs-stay
cd wails-obfs-stay
# install this PR
git clone https://github.com/wailsapp/wails.git
pushd wails/v2/cmd/wails
git checkout feature/garble-support
go install
popd
# create fresh project using this PR
wails init -n obfs-remain-test
cd obfs-remain-test
echo "replace github.com/wailsapp/wails/v2 => ../wails/v2" >> go.mod
# build obfuscated, works
wails build -obfuscate
cat frontend/wailsjs/go/main/App.js | grep ObfuscatedCall || (echo "Not obfuscated" && exit 1)
# build regular, it's still obfuscated
wails build
cat frontend/wailsjs/go/main/App.js | (! grep ObfuscatedCall) || (echo "Still obfuscated" && exit 1)

I was able to work around this using a patch to also regenerate bindings if we're building the frontend:

From deeb5e0ecd8c536f0328b0f942bcb7d078ca9623 Mon Sep 17 00:00:00 2001
From: AlbinoDrought <snip>
Date: Sat, 10 Sep 2022 12:34:05 -0700
Subject: [PATCH] fix: also perform early generation of plain bindings if we'll
 be building the frontend

---
 v2/pkg/commands/build/build.go | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/v2/pkg/commands/build/build.go b/v2/pkg/commands/build/build.go
index 2c8a44fa..a74b0cf3 100644
--- a/v2/pkg/commands/build/build.go
+++ b/v2/pkg/commands/build/build.go
@@ -130,11 +130,14 @@ func Build(options *Options) (string, error) {
 		}
 	}
 
-	if options.Obfuscate {
+	if options.Obfuscate || !options.IgnoreFrontend {
 		err := generateBindings(options)
 		if err != nil {
 			return "", err
 		}
+	}
+
+	if options.Obfuscate {
 		if !lo.Contains(options.UserTags, "obfuscated") {
 			options.UserTags = append(options.UserTags, "obfuscated")
 		}
@@ -166,7 +169,11 @@ func Build(options *Options) (string, error) {
 }
 
 func generateBindings(options *Options) error {
-	options.Logger.Print("  - Generating obfuscated bindings: ")
+	if options.Obfuscate {
+		options.Logger.Print("  - Generating obfuscated bindings: ")
+	} else {
+		options.Logger.Print("  - Generating bindings: ")
+	}
 	defer func() {
 		options.Logger.Println("Done.")
 	}()
-- 
2.37.1

but I don't fully understand the binding generation process (when are normal bindings usually generated?)

As a bonus, my Typescript project doesn't build on latest if I rm frontend/wailsjs -rf because of the missing definition files. It succeeds with the above patch.

Reproducible failing TS build on latest
go install github.com/wailsapp/wails/v2/cmd/wails@latest
wails init -n ts-rm-test -t vue-ts
cd ts-rm-test
wails build # success
rm frontend/wailsjs -rf
wails build # failure

src/components/HelloWorld.vue(3,21): error TS2307: Cannot find module '../../wailsjs/go/main/App' or its corresponding type declarations.
src/components/HelloWorld.vue(11,25): error TS7006: Parameter 'result' implicitly has an 'any' type.

EDIT: seems like regular bindings are only generated during wails dev or an explicit wails generate module, see #1683 (comment)

@leaanthony
Copy link
Member Author

Wow, thanks so much @AlbinoDrought for taking the time to test and write up. What you're seeing is similar to what I noticed and it looks like some funkiness with build tags. What you've noted will be a huge help. I hope to get it fixed soon.

@Snider
Copy link
Contributor

Snider commented Sep 11, 2022

Wow, thanks so much @AlbinoDrought for taking the time to test and write up. What you're seeing is similar to what I noticed and it looks like some funkiness with build tags. What you've noted will be a huge help. I hope to get it fixed soon.

Have you looked into testing frameworks for Go, this is an ideal issue to create a test case; if we want Wails to get used more; probably a good idea.

@leaanthony
Copy link
Member Author

It's a fair point. The library part of Wails is really hard to test as there's not much in the way of desktop UI testing outside of JVMs. The CLI part is, however, a fine candidate for it. There's already a number of tests in areas that get quite hairy and I think this has been promoted to "quite hairy".

@Snider
Copy link
Contributor

Snider commented Sep 11, 2022

It's a fair point. The library part of Wails is really hard to test as there's not much in the way of desktop UI testing outside of JVMs. The CLI part is, however, a fine candidate for it. There's already a number of tests in areas that get quite hairy and I think this has been promoted to "quite hairy".

Yes, this is a feature that can not fail, ever, for any reason; any release that has a break with any security feature would be a massive reputation hit for wails.io; I'm sure it won't take too much time to confirm the base functionality either.

If you build out a demo app for the UI, I'm sure it is possible to get https://www.selenium.dev via the JS driver to handle GUI testing in GitHub Actions.

I'm busy for a while, but one day, this is a type of thing I could help with, personal finances allowing.

@leaanthony
Copy link
Member Author

As far as I'm aware, selenium and the other are built for browser automation and not webviews. Would be interesting to understand how similar projects tackle it.

@Snider
Copy link
Contributor

Snider commented Sep 11, 2022

As far as I'm aware, selenium and the other are built for browser automation and not webviews. Would be interesting to understand how similar projects tackle it.

It might need some double-sided sticky tape and blue tac, but I don't see why it wouldn't work; since browsers, these days are just webview wrappers... some original definition hacking would be required probably, but should work fine afaik.

https://www.npmjs.com/package/selenium-webdriver

@AlbinoDrought
Copy link
Contributor

AlbinoDrought commented Sep 11, 2022

Selenium would at least work great out of the box for the server started by wails dev. Cypress is a similar project, but has a much better developer experience in my opinion. The downside is that Cypress has some proprietary elements (the libraries you use are MIT licensed, but some features are locked behind their optional SaaS dashboard without workarounds)

Here's a sample test with Cypress: https://github.com/AlbinoDrought/sample-wails-robotgo-test/blob/master/cypress/cypress/e2e/greeting.cy.js

describe('Greetings', () => {
  it('should say hello after entering our name', () => {
    cy.visit('http://localhost:34115')
    
    cy.get('#input #name').type('AlbinoDrought')
    cy.get('#input button').click()

    cy.get('#result').should('contain', 'Hello AlbinoDrought, It\'s show time!')
  })
})

Here's what it looks like while running in dev mode:

image

I'm not sure of any great solutions for testing the compiled versions. Depending on how much interactivity is needed, a robotgo + OpenCV wrapper like this could be used: https://github.com/AlbinoDrought/sample-wails-robotgo-test/blob/master/main_test.go

func TestGreeting(t *testing.T) {
	root := "test/TestGreeting"

	// Click the input
	input := mustFindInDesktop(root, "001_input.png")
	clickGCV(input)

	// Type our name
	robotgo.TypeStr("AlbinoDrought")

	// Click the greeting button
	button := mustFindInDesktop(root, "002_button.png")
	clickGCV(button)

	// Must see our greetings text
	time.Sleep(1 * time.Second)
	mustFindInDesktop(root, "003_greetings.png")
}
RunningTest.c.webm

But running compiled-UI tests like this in headless CI is likely another can of worms.

@Snider
Copy link
Contributor

Snider commented Sep 11, 2022

You can run chromium headless for CI testing; that's what I used to do in 2009~ to make PDFs server-side with HTML_Unit; it won't be a standard procedure so it will need dev time probably.

But I don't see why it couldn't work, and selenium doesn't hide the essential features behind a paywall service, if not selenium, then any system that is 100% open source should be OK.

Selenium IDE might not look nice, but it works and is free... someone could donate some time to their project and make it more excellent...

https://www.selenium.dev/selenium-ide/
https://github.com/SeleniumHQ/selenium-ide/releases

@leaanthony
Copy link
Member Author

Good discussion. I feel there's a distinction to be made here about what gets tested. For me, there's 3 things:

  • The Wails CLI
  • The Wails library
  • The final application

The first 2 should be covered by Go's standard tooling plus maybe a testing library. The last one could be covered by Selenium if your concern is what the application does and there might be a use case for using it to check the default templates work correctly, but it's potentially limited benefit as it only checks that it's working in dev mode. I looked fairly heavily into desktop GUI automation and like I said previously, there's a lot around the JVM but not much else. My conclusion was pretty much what @AlbinoDrought came to: robotgo.

@leaanthony
Copy link
Member Author

leaanthony commented Sep 12, 2022

Ok, just pushed a few changes:

  • -obfuscate flag is now -obfuscated (It was getting confusing!)
  • wails build now generates bindings by default unless -skipbindings is given. This was the reason it wasn't generating non-obfuscated bindings previously.
  • wails dev now does not have the option for -obfuscated. I don't believe it's necessary.
  • wails dev flag -noGen has been renamed to -skipbindings
  • Refactored bindings generation (I really want to rename the command wails generate bindings!)

So this seems to work pretty well for me. Let me know how you get on 👍

@AlbinoDrought
Copy link
Contributor

(Switched to 822baa5 on my end, running great 👍 )

Copy link
Collaborator

@stffabi stffabi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@leaanthony leaanthony merged commit 052b922 into master Sep 13, 2022
@leaanthony leaanthony deleted the feature/garble-support branch September 13, 2022 00:05
@leaanthony
Copy link
Member Author

Thanks everyone for all the effort. We got there! 💥 🚀 💥

@Snider
Copy link
Contributor

Snider commented Sep 13, 2022

Well done, you lot, some slick improvements and teamwork :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support integers for method call IDs
6 participants