Generate apps with SecTester #743

jwoertink · 2022-03-19T17:43:27Z

Fixes #740

Right now this is in WIP to get things rolling, but a few questions still need answering...

Major parts left to handle:

- How do we test this? We can test that the code gets generated, but we now say you need a NEXPLOIT_TOKEN to exist, and it doesn't. Also, even if we did, this would end up connecting to Bright on each PR, and nightly run.
- Figure out how the API only specs will run. If you're not doing a browser app, you should still have the option to test that your API endpoints are good
- Need to update the generated github CI so it includes the -Dwith_sec_tests, but only if you've added that in.
- Figure out where to point users for API documentation. e.g. "Ok, I've added this in, my built-in tests run... now what? How do I add new ones?"

/cc. @bararchy

src/app_with_sec_tester/spec/flows/security_spec.cr.ecr

jwoertink · 2022-03-19T17:47:58Z

src/app_with_sec_tester/spec/setup/sec_tester.cr

+require "lucky_sec_tester"
+
+# Signup for a `NEXTPLOT_TOKEN` at
+# [NeuraLegion](https://app.neuralegion.com/signup)


The company name is now called Bright, but as of this comment, the Github org and this URL are still pointed to NeuraLegion... I don't want to confuse people, so this will need to change once the company's name is fully moved over.

src/generators/web.cr

bararchy · 2022-03-20T18:59:43Z

@jwoertink if we need to configure the API key for Lucky let me know and I can make sure the Lucky Org in Bright app has multiple engines to run all needed parallel scans :)

grepsedawk · 2022-03-20T19:00:50Z

Is SecTester free for private source?

bararchy · 2022-03-20T19:28:55Z

@grepsedawk yeha :) you can signup for an API key for free https://app.neuralegion.com/signup

… pass again

src/web_app_skeleton/.github/workflows/ci.yml.ecr

jwoertink · 2022-04-01T22:50:05Z

spec/integration/init_web_spec.cr

@@ -17,7 +17,7 @@ describe "Initializing a new web project" do
      File.delete("test-project/.env")
      compile_and_run_specs_on_test_project
      File.read("test-project/Procfile").should contain "test_project"
-      File.read(".github/workflows/ci.yml").should contain "postgres"
+      File.read("test-project/.github/workflows/ci.yml").should contain "postgres"


turns out this has been wrong for a while 😅

…y single time

…them so they only run for the CI

bararchy · 2022-04-03T10:19:26Z

@jwoertink I saw you're using node12 which should be OK, I verified in the docs that it's supported: https://docs.brightsec.com/docs/installation-options#npm

So anything 10-14 should be good, I also looked for the touch thingy but I can't find it. One reason for process to not be up is something in the ENV? I'll try and add more logs. can you check locally with the API key and with LOG_LEVEL=DEUBG? see that at least locally we get it working

jwoertink · 2022-04-03T18:55:08Z

That touch happens on the LuckySecTester shard itself Since that shard doesn't actually need to run any nexploit checks.

https://github.com/luckyframework/lucky_sec_tester/blob/afc4d99ee1b8fbd371a2b9b0a5f0ac1bee185859/.github/workflows/ci.yml#L49-L50

I can use a newer version of node, but it just seems like the scanner.client isn't being called or something. I'll try locally and see what I can find out.

bararchy · 2022-04-03T20:14:04Z

@jwoertink can we try and re-run using latest luckyframework/lucky_sec_tester@346ceb3 ?

… we turn down to medium threshold. Also pussing a full URL to the Target for the javascript checks

bararchy · 2022-04-03T21:31:17Z

src/app_with_sec_tester/spec/flows/security_spec.cr.ecr

+    scan_with_cleanup do |scanner|
+      api_headers = HTTP::Headers{"Content-Type" => "application/json", "Accept" => "application/json"}
+      target = scanner.build_target(Api::SignIns::Create, headers: api_headers) do |t|
+        t.body = {"user" => {"email" => "aa%40aa.com", "password" => "123456789"}}.to_json


@jwoertink as this is a JSON API call, do we actually need to encode the @ here?

probably not....

jwoertink · 2022-04-04T18:21:22Z

it's green!!!

jwoertink · 2022-04-04T18:21:54Z

.github/workflows/ci.yml

-      - uses: actions/setup-node@v3.1.0
-        with:
-          node-version: "12.x"


Just use the node that comes installed on ubuntu-latest

jwoertink · 2022-04-04T18:22:32Z

.github/workflows/weekly.yml

-  push:
-    branches: [main]
-  pull_request:


This was causing this to run on every single commit. It should be a separate PR, but I needed it to quiet down with all the testing.

jwoertink · 2022-04-04T18:22:56Z

.github/workflows/weekly.yml

      - uses: actions/setup-node@v3.1.0
        with:
          node-version: "12.x"


this actually needs to go away

jwoertink · 2022-04-04T18:24:12Z

spec/integration/sec_tester_spec.cr

@@ -0,0 +1,23 @@
+require "../spec_helper"
+
+{% if env("RUN_SEC_TESTER_SPECS") == "1" %}


Anyone that wants to run this spec locally would need a Bright API key... That's not really going to happen, so we can treat this like the Heroku specs. As long as this spec passes in CI, and the rest pass locally, we should be fine

bararchy · 2022-04-04T18:29:13Z

Amazing work @jwoertink! 🟢 light!

…ng a few suggestions

robacarp · 2022-04-05T22:32:13Z

src/web_app_skeleton/script/setup.ecr

@@ -22,7 +22,7 @@ print_done
 <%- end -%>

 notice "Installing shards"
-shards install | indent
+shards install --ignore-crystal-version | indent


Is this a "hey i need this to work for now" contribution which should be removed? Or should this be kept in?

This was a tough one... Without it, all tests again < 1.2.2 fail because of https://github.com/luckyframework/lucky_sec_tester/blob/346ceb3567110e433863dd1997afcec6243db3e5/shard.yml#L7 That's an easy one to drop down, but then we can only drop down to 1.1.1 https://github.com/NeuraLegion/sec_tester/blob/9db160150b4cc15ebb6d6722a25ee55a5c5af3ce/shard.yml#L7 and CLI tests against 1.0.0

lucky_cli/.github/workflows/ci.yml

Line 35 in 89be926

- 1.0.0

Do I just raise the min test to 1.1.1 for CLI since 1.4 comes out tomorrow? Then I could update the other shard, and remove this

Alright, I updated the LuckySecTester to be >=1.1.1. It would still fail due to checking 1.0.0. So I still have to either use the flag, or drop 1.0.0 support 🤔 I'm sort of ok with going with the latter. I feel like once 1.4.0 drops, few people will be wanting to upgrade Lucky and continue using 1.0.0....

I don't have any reason not to have this flag in there, I was just curious. It's complicated, and I get that.

robacarp

This seems fine. The way this pattern scales makes me nervous, but I get that we need to have a few dice in the bucket before we can have enough to roll yahtzee.

inital setup to adding in the sec tester. Fixes #740

5fa84d9

jwoertink commented Mar 19, 2022

View reviewed changes

src/app_with_sec_tester/spec/flows/security_spec.cr.ecr Outdated Show resolved Hide resolved

jwoertink commented Mar 19, 2022

View reviewed changes

src/app_with_sec_tester/spec/flows/security_spec.cr.ecr Outdated Show resolved Hide resolved

jwoertink commented Mar 19, 2022

View reviewed changes

src/generators/web.cr Outdated Show resolved Hide resolved

jwoertink added 4 commits March 27, 2022 14:42

updating the flow security_spec with some clarity for a few tests

7da39fa

Updating the generated CI config to include running the sec tests.

41e65ee

typo and some docs

ac63606

flip flop the setup and ignore the crystal version so hopefully specs…

e3f2f75

… pass again

jwoertink commented Mar 29, 2022

View reviewed changes

src/web_app_skeleton/.github/workflows/ci.yml.ecr Show resolved Hide resolved

jwoertink added 4 commits March 29, 2022 18:50

Adding in the integration spec to test the sec_tests.

f5f6f78

actually this should run the sec tests

258c373

Oops... point to the correct CI file

20ab2a9

ensure we're always in the project directory

e347017

jwoertink commented Apr 1, 2022

View reviewed changes

jwoertink added 8 commits April 1, 2022 16:04

merged main. Fixed conflict in spec

0fc43d2

run all the setup and stuff in the right place with the right files

c11c9e4

the global CI needs nexplot so it just exists everywhere

451a5e6

make sure the weekly runs the nexploit repeater

563467e

These workflows are expensive enough. We don't need them running ever…

b9f79de

…y single time

make sure only one instance of the scanner is used per spec

92222d4

Not everyone will be able to run the nexploit checks locally. Moving …

ed8b509

…them so they only run for the CI

no clue how I managed that....

8808a3d

SSL is off during specs which will cause the cookie check to fail, so…

f28f949

… we turn down to medium threshold. Also pussing a full URL to the Target for the javascript checks

bararchy reviewed Apr 3, 2022

View reviewed changes

jwoertink added 4 commits April 3, 2022 14:53

make this an enum

1784c7d

debugging to hopefully see what's up

31e058f

See if we can just use the node version already installed

c4f33a1

Merge branch 'main' into issues/740

06e5ad4

jwoertink marked this pull request as ready for review April 4, 2022 18:20

jwoertink commented Apr 4, 2022

View reviewed changes

bararchy approved these changes Apr 4, 2022

View reviewed changes

jwoertink added 3 commits April 4, 2022 18:12

make sure that the weekly CI is setup correctly for SecTester. Updati…

0f123dd

…ng a few suggestions

merge conflict

5d39f40

make sure to catch the formatter so generated CIs don't fail

7d306c5

robacarp reviewed Apr 5, 2022

View reviewed changes

robacarp approved these changes Apr 5, 2022

View reviewed changes

matthewmcgarvey approved these changes Apr 6, 2022

View reviewed changes

jwoertink merged commit 88c10fc into main Apr 6, 2022

jwoertink deleted the issues/740 branch April 6, 2022 14:34

jwoertink mentioned this pull request Apr 8, 2022

Document LuckySecTester luckyframework/website#971

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate apps with SecTester #743

Generate apps with SecTester #743

jwoertink commented Mar 19, 2022 •

edited

jwoertink Mar 19, 2022

bararchy commented Mar 20, 2022

grepsedawk commented Mar 20, 2022

bararchy commented Mar 20, 2022

jwoertink Apr 1, 2022

bararchy commented Apr 3, 2022

jwoertink commented Apr 3, 2022

bararchy commented Apr 3, 2022

bararchy Apr 3, 2022

jwoertink Apr 3, 2022

jwoertink commented Apr 4, 2022

jwoertink Apr 4, 2022

jwoertink Apr 4, 2022

jwoertink Apr 4, 2022

jwoertink Apr 4, 2022

bararchy commented Apr 4, 2022

robacarp Apr 5, 2022

jwoertink Apr 5, 2022

jwoertink Apr 5, 2022

robacarp Apr 5, 2022

robacarp left a comment

		@@ -0,0 +1,23 @@
		require "../spec_helper"

		{% if env("RUN_SEC_TESTER_SPECS") == "1" %}

Generate apps with SecTester #743

Generate apps with SecTester #743

Conversation

jwoertink commented Mar 19, 2022 • edited

Choose a reason for hiding this comment

bararchy commented Mar 20, 2022

grepsedawk commented Mar 20, 2022

bararchy commented Mar 20, 2022

Choose a reason for hiding this comment

bararchy commented Apr 3, 2022

jwoertink commented Apr 3, 2022

bararchy commented Apr 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jwoertink commented Apr 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bararchy commented Apr 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robacarp left a comment

Choose a reason for hiding this comment

jwoertink commented Mar 19, 2022 •

edited