Add validation for http header #296

kmashint · 2017-10-08T02:25:29Z

Thanks for goss! This adds a validation for for http header "name: value" pairs as below.

http:
  https://www.google.com:
    allow-insecure: false
    no-follow-redirects: false
    timeout: 3000
    status: 200
    header: ["Content-Type: text/html"]
    body: ["google"]

kmashint · 2017-10-08T02:54:02Z

The Travis build failed due to the extra http header attribute, but maybe that's expected when adding a new test attribute.

aelsabbahy

Seems like the tests need to be updated to support the new header feature.

aelsabbahy · 2017-10-17T19:49:39Z

glide.yaml

 - package: github.com/fatih/color
- package: github.com/aelsabbahy/go-ps
-  version: unix_race_condition


Moving off of this branch will break things. This was added due to the race condition explained here: #267

kmashint · 2017-10-17T21:05:09Z

Thanks, will review and update.

kmashint · 2017-10-18T14:42:43Z

@aelsabbahy I found the missing test expectations and adjusted.

aelsabbahy

Thanks, I think this PR is pretty close now.

A few notes:

Glide.lock is out of sync
Can you update the http test in goss-shared.yaml to test that this works

Also, see inline comments.

Thanks for taking the time to add this :)

glide.lock

resource/http.go

kmashint · 2017-10-21T00:21:01Z

I've adjusted with some more updates. I'm making some newbie mistakes with the go project. It all makes sense when I relate concepts to Java, Python, JS but in this new ecosystem I don't know what I don't know.

aelsabbahy · 2017-11-07T16:50:12Z

This is almost ready for merging, just need a test in goss-shared.yaml showing that this works.

kmashint · 2017-12-04T00:54:00Z

I'm working through some problems trying to enable the test for the http header validation. On a Raspberry Pi / debian linux it's working fine but I can't correlate the error from travis-ci build to what's really causing it. I'll try to reproduce the error locally.

berney · 2018-04-09T18:07:46Z

This would be a very handy feature. Has there been any further progress on getting the test case finished?

kmashint · 2018-04-10T00:24:01Z

I have a test case that works in actual use on Raspberry Pi / debian linux but there's something broken when it runs in the travis integration. I'll try again in the next day or two.

kmashint · 2018-04-10T02:10:53Z

I tested building on Raspberry Pi and it works for its executable. I tested building on Ubuntu on Windows and that works for the amd64 executable there. But something still fails on the CentOS7 test on the Travis integration server. I tried building on a local VirtualBox with CentsOS7 and that worked as well.

kmashint · 2018-04-14T18:05:55Z

I've tested locally successfully on CentOS7, Ubuntu, R-Pi and rebased on the latest master but I still get a build error from Travis CI, maybe when it's comparing [[ centos7 == \a\r\c\h ]].
@aelsabbahy , @berney , have you seen this before? Is this something I can fix or is it elsewhere?
It would be good to finish this off.

Using default tag: latest
latest: Pulling from aelsabbahy/goss_centos7
++docker_exec /goss/centos7/goss-linux-amd64 --vars /goss/vars.yaml -g /goss/centos7/goss.yaml validate
++docker exec goss_int_test_centos7_amd64 /goss/centos7/goss-linux-amd64 --vars /goss/vars.yaml -g /goss/centos7/goss.yaml validate
...
Count: 84, Failed: 0, Skipped: 0
+[[ centos7 == \a\r\c\h ]]
+egrep -q 'Count: 83, Failed: 0'
+rv=1
+docker rm -vf 7fcd348d3633254b8ce7ff49886c44aa7c08234ab5ddc66367340dff2fa82c76
7fcd348d3633254b8ce7ff49886c44aa7c08234ab5ddc66367340dff2fa82c76
+exit 1
make: *** [centos7] Error 1
The command "make" exited with 2.
Done. Your build exited with 1.

berney · 2018-04-20T02:37:44Z

I'm travelling at the moment, I will try mid next week if the issue is still outstanding.

timbirk · 2018-05-27T10:51:02Z

@berney I could pick this up if you want?? I feel like I discussed HTTP header validation a long time ago but can't remember.

I think it would be a useful addition. I'd especially like it to be able to validate that a header doesn't exist.

For example, I might want to validate that the no-cache header doesn't exist on my high traffic heavily cached website.

berney · 2019-01-08T07:18:29Z

I have been using a private build with this fix and it is working well. This PR should be able to be merged, the correct number of passed tests should be in the egrep line for the testing.

Conflicts: glide.lock glide.yaml integration-tests/goss/goss-shared.yaml integration-tests/test.sh

kmashint · 2019-02-28T01:40:52Z

I've merged from master and update the expected test counts in test.sh but it's now giving an error with archlinux:
manifest for base/archlinux:latest not found

berney · 2019-02-28T13:06:04Z

base/archlinux is deprecated use archlinux/base. See https://hub.docker.com/r/base/archlinux/.

$ docker search archlinux
NAME                                  DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED
base/archlinux                        Deprecated repository, use archlinux/base in���   331                                     [OK]

I don't know why but they seemed to have used base as the owner/project name rather than archlinux. I'm not even sure base is the archlinux project (e.g. from the maintainers or not).

This reverts commit 6ec8613.

kmashint · 2019-04-07T00:13:44Z

@aelsabbahy @berney I've been trying to debug more strange cloud build errors, this time with DNS tests. They run locally but for some reason fail in the Travis cloud build. Any ideas? I'd like to finally finish this!

Why does this fail?  Tried timeouts but that didn't help.  It works on CentOS 7.4.1708 in VirtualBox.
Using dnslookup and set type=txt with txt._test.dnstest.io does give "Hello DNS" but not as an address per se.
DNS: TXT:txt._test.dnstest.io: resolvable:
Expected
    <bool>: false
to equal
    <bool>: true
DNS: TXT:txt._test.dnstest.io: addrs: skipped
Total Duration: 1.000s
Count: 90, Failed: 1, Skipped: 1'

kmashint · 2019-04-08T01:52:09Z

I did a complete rebuild of my CentOS7 VM and now I'm getting similar errors, but they are even from master, not just my branch, so it's a side effect of underlying platform updates.

Conflicts: integration-tests/test.sh

kmashint · 2019-05-03T02:01:16Z

@berney Phew! Finally all checks passed!

kmashint · 2019-05-03T20:37:25Z

@aelsabbahy This is up to date again and passing tests after merging away the problematic DNS tests.

kmashint · 2019-05-23T01:52:45Z

@aelsabbahy @berney This merged the latest changes again.

aelsabbahy · 2019-05-23T01:58:33Z

Awesome, I'll take a look at it this week and merge it.

Many thanks for this, great enhancement!

kmashint · 2019-07-28T13:01:16Z

@aelsabbahy @berney Any luck with merging this? It would be good to start using it for some of my work projects where I'm currently falling back to bash+curl to validate http headers.

kmashint · 2019-09-22T12:16:17Z

@aelsabbahy @berney Just pinging again for merge. It would be good to start using it for some of my work projects where I'm currently falling back to bash+curl to validate http headers.

aelsabbahy · 2019-11-23T21:10:09Z

Github rebasing of PRs is a bit confusing.. Not sure the proper way to do this.. But I ended up just doing a rebase locally and pushing it up as a different PR.

It looks like all the author information is preserved, I want to make sure you're credited for the changes.

Thanks a bunch for this!

* Add validation for http header * Add http headers: [] to goss-expected tests. * Add http header test in goss-shared.yaml. * Update expected test counts.

aelsabbahy requested changes Oct 17, 2017

View reviewed changes

stell-aelsabbahy approved these changes Oct 18, 2017

View reviewed changes

aelsabbahy approved these changes Oct 18, 2017

View reviewed changes

aelsabbahy requested changes Oct 18, 2017

View reviewed changes

glide.lock Outdated Show resolved Hide resolved

resource/http.go Show resolved Hide resolved

aelsabbahy mentioned this pull request Nov 7, 2017

Add verification for redirect #299

Closed

kmashint force-pushed the validate-http-header branch 3 times, most recently from a020e82 to a6023ff Compare December 3, 2017 19:55

kmashint added 6 commits April 14, 2018 13:55

Add validation for http header

2de1aec

Fix indenting to tabs.

bd5f555

Add lost fix for unix_race_condition in go-ps

958a82f

Add http headers: [] to goss-expected tests.

fc47d8b

Fix typo in expected tests.

c345dd9

Add http header test in goss-shared.yaml.

9319140

kmashint force-pushed the validate-http-header branch from b77c51d to 9319140 Compare April 14, 2018 17:56

Try http test github.com/ with header and body.

f54b86f

kmashint force-pushed the validate-http-header branch from f3eca91 to f54b86f Compare April 14, 2018 19:09

Keith Mashinter and others added 2 commits February 27, 2019 09:58

Update expected test counts.

690e965

Merge remote-tracking branch 'upstream/master' into validate-http-header

430b05a

Conflicts: glide.lock glide.yaml integration-tests/goss/goss-shared.yaml integration-tests/test.sh

kmashint added 8 commits March 23, 2019 07:40

Update base/archlinux to archlinux/base

5948420

Empty commit to rebuild

7bc0d41

Empty commit to rebuild

7428120

Ensure minor_ver is treated as base-10.

fdafa36

Ensure minor_ver is treated as base-10.

6ec8613

Allow retry -r 1s in test.sh

7c71c8b

Allow longer retry -r 10s in test.sh

b3ee23a

Revert "Ensure minor_ver is treated as base-10."

6110ab7

This reverts commit 6ec8613.

Merge branch 'master' into validate-http-header

4491af8

Conflicts: integration-tests/test.sh

kmashint added 2 commits May 13, 2019 21:39

Merge branch 'master' into validate-http-header

abb8373

Merge branch 'master' into validate-http-header

855e61f

aelsabbahy self-requested a review May 23, 2019 23:52

Merge branch 'master' into validate-http-header

0bc2bbc

aelsabbahy mentioned this pull request Nov 23, 2019

Pr 296 #498

Merged

aelsabbahy closed this in #498 Nov 24, 2019

kmashint referenced this pull request Nov 25, 2019

Pr 296 (#498)

ee6ae7f

* Add validation for http header * Add http headers: [] to goss-expected tests. * Add http header test in goss-shared.yaml. * Update expected test counts.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add validation for http header #296

Add validation for http header #296

kmashint commented Oct 8, 2017

kmashint commented Oct 8, 2017

aelsabbahy left a comment

aelsabbahy Oct 17, 2017

kmashint commented Oct 17, 2017

kmashint commented Oct 18, 2017

aelsabbahy left a comment

kmashint commented Oct 21, 2017

aelsabbahy commented Nov 7, 2017

kmashint commented Dec 4, 2017

berney commented Apr 9, 2018

kmashint commented Apr 10, 2018 •

edited

Loading

kmashint commented Apr 10, 2018 •

edited

Loading

kmashint commented Apr 14, 2018

berney commented Apr 20, 2018

timbirk commented May 27, 2018

berney commented Jan 8, 2019

kmashint commented Feb 28, 2019

berney commented Feb 28, 2019

kmashint commented Apr 7, 2019 •

edited

Loading

kmashint commented Apr 8, 2019

kmashint commented May 3, 2019

kmashint commented May 3, 2019

kmashint commented May 23, 2019

aelsabbahy commented May 23, 2019

kmashint commented Jul 28, 2019

kmashint commented Sep 22, 2019

aelsabbahy commented Nov 23, 2019

Add validation for http header #296

Add validation for http header #296

Conversation

kmashint commented Oct 8, 2017

kmashint commented Oct 8, 2017

aelsabbahy left a comment

Choose a reason for hiding this comment

aelsabbahy Oct 17, 2017

Choose a reason for hiding this comment

kmashint commented Oct 17, 2017

kmashint commented Oct 18, 2017

aelsabbahy left a comment

Choose a reason for hiding this comment

kmashint commented Oct 21, 2017

aelsabbahy commented Nov 7, 2017

kmashint commented Dec 4, 2017

berney commented Apr 9, 2018

kmashint commented Apr 10, 2018 • edited Loading

kmashint commented Apr 10, 2018 • edited Loading

kmashint commented Apr 14, 2018

berney commented Apr 20, 2018

timbirk commented May 27, 2018

berney commented Jan 8, 2019

kmashint commented Feb 28, 2019

berney commented Feb 28, 2019

kmashint commented Apr 7, 2019 • edited Loading

kmashint commented Apr 8, 2019

kmashint commented May 3, 2019

kmashint commented May 3, 2019

kmashint commented May 23, 2019

aelsabbahy commented May 23, 2019

kmashint commented Jul 28, 2019

kmashint commented Sep 22, 2019

aelsabbahy commented Nov 23, 2019

kmashint commented Apr 10, 2018 •

edited

Loading

kmashint commented Apr 10, 2018 •

edited

Loading

kmashint commented Apr 7, 2019 •

edited

Loading