feat: mock data compress-decompress #209

Shivansh-yadav13 · 2022-10-09T15:38:14Z

Related Issue

Closes: [feature]: Make mock data compress-decompress while writing it into yaml #184
Compressed the data on Encode wiriting mocks into yaml and decompressed on Decode

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Code style update (formatting, local variables)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project.
I have performed a self-review of my own code.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have added tests that prove my fix is effective or that my feature works.
New and existing unit tests pass locally with my changes.

Screenshots (if any)

github-actions · 2022-10-09T15:38:29Z

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

github-actions

Thank you and congratulations 🎉 for opening your very first pull request in keploy

Shivansh-yadav13 · 2022-10-09T15:39:42Z

I have read the CLA Document and I hereby sign the CLA

slayerjain · 2022-10-10T17:55:08Z

@Sarthak160 @re-Tick please review

re-Tick · 2022-10-13T16:50:11Z

Hii @Shivansh-yadav13, I have tried the changes locally for recording testcases in samples-go/gin-mongo app. The server is crashing with error-stack:

go.keploy.io/server/pkg/service/regression.(*Regression).deNoiseYaml
        /Users/ritikjain/Desktop/go-workspace/github.com/keploy/keploy/pkg/service/regression/regression.go:542
go.keploy.io/server/pkg/service/regression.(*Regression).DeNoise
        /Users/ritikjain/Desktop/go-workspace/github.com/keploy/keploy/pkg/service/regression/regression.go:605
go.keploy.io/server/http/regression.(*regression).DeNoise
        /Users/ritikjain/Desktop/go-workspace/github.com/keploy/keploy/http/regression/regression.go:317
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/cors.(*Cors).Handler.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/cors@v1.2.0/cors.go:228
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/keploy/go-sdk/integrations/kchi.(*chiV5).Next
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/integrations/kchi/chi-v5.go:64
github.com/keploy/go-sdk/keploy.Middleware
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/keploy/router.go:30
github.com/keploy/go-sdk/integrations/kchi.ChiMiddlewareV5.func1.1
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/integrations/kchi/chi-v5.go:13
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:87
net/http.serverHandler.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2916
net/http.(*conn).serve
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:1966

2022-10-13T15:54:28.137+0530    ERROR   regression/regression.go:319    error putting testcase  {"error": "EOF"}
go.keploy.io/server/http/regression.(*regression).DeNoise
        /Users/ritikjain/Desktop/go-workspace/github.com/keploy/keploy/http/regression/regression.go:319
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/cors.(*Cors).Handler.func1
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/cors@v1.2.0/cors.go:228
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/keploy/go-sdk/integrations/kchi.(*chiV5).Next
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/integrations/kchi/chi-v5.go:64
github.com/keploy/go-sdk/keploy.Middleware
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/keploy/router.go:30
github.com/keploy/go-sdk/integrations/kchi.ChiMiddlewareV5.func1.1
        /Users/ritikjain/go/pkg/mod/github.com/keploy/go-sdk@v0.6.4/integrations/kchi/chi-v5.go:13
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2084
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/ritikjain/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:87
net/http.serverHandler.ServeHTTP
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:2916
net/http.(*conn).serve
        /opt/homebrew/Cellar/go/1.18/libexec/src/net/http/server.go:1966```

Sarthak160 · 2022-10-17T12:22:20Z

Hi @Shivansh-yadav13 . Are you still working on this part ?? I was trying to run my application on test mode after recording few tests and It was showing below stacktrace -

 ERROR   regression/regression.go:145    unexpected EOF
go.keploy.io/server/pkg/service/regression.(*Regression).ReadTCS
        /Users/sarthak_1/Documents/Keploy/final/keploy/pkg/service/regression/regression.go:145
go.keploy.io/server/http/regression.(*regression).GetTCS
        /Users/sarthak_1/Documents/Keploy/final/keploy/http/regression/regression.go:177
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/cors.(*Cors).Handler.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/cors@v1.2.0/cors.go:228
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/keploy/go-sdk/integrations/kchi.(*chiV5).Next
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/integrations/kchi/chi-v5.go:64
github.com/keploy/go-sdk/keploy.Middleware
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/keploy/router.go:30
github.com/keploy/go-sdk/integrations/kchi.ChiMiddlewareV5.func1.1
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/integrations/kchi/chi-v5.go:13
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:87
net/http.serverHandler.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2947
net/http.(*conn).serve
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:1991
2022-10-17T15:20:11.879+0530    ERROR   regression/regression.go:128    unexpected EOF
go.keploy.io/server/pkg/service/regression.(*Regression).StartTestRun
        /Users/sarthak_1/Documents/Keploy/final/keploy/pkg/service/regression/regression.go:128
go.keploy.io/server/http/regression.(*regression).Start
        /Users/sarthak_1/Documents/Keploy/final/keploy/http/regression/regression.go:94
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:70
github.com/go-chi/chi.(*Mux).Mount.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:311
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).routeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:436
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/cors.(*Cors).Handler.func1
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/cors@v1.2.0/cors.go:228
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/keploy/go-sdk/integrations/kchi.(*chiV5).Next
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/integrations/kchi/chi-v5.go:64
github.com/keploy/go-sdk/keploy.Middleware
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/keploy/router.go:30
github.com/keploy/go-sdk/integrations/kchi.ChiMiddlewareV5.func1.1
        /Users/sarthak_1/go/pkg/mod/github.com/keploy/go-sdk@v0.6.3/integrations/kchi/chi-v5.go:13
net/http.HandlerFunc.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2109
github.com/go-chi/chi.(*Mux).ServeHTTP
        /Users/sarthak_1/go/pkg/mod/github.com/go-chi/chi@v1.5.4/mux.go:87
net/http.serverHandler.ServeHTTP
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:2947
net/http.(*conn).serve
        /opt/homebrew/Cellar/go/1.19.1/libexec/src/net/http/server.go:1991

It's unable to fetch testcases , I tried to debug and found that it is unable to read data during unzipping
data, err := ioutil.ReadAll(gzr)

Sarthak160 · 2022-10-27T12:14:08Z

Hey @Shivansh-yadav13 I have fixed this for test mode you can refer that to make changes in your branch

Shivansh-yadav13 · 2022-10-27T15:27:36Z

@Sarthak160 should I close this PR?

Sarthak160 · 2022-10-27T15:53:38Z

@Sarthak160 should I close this PR?

Or you can just add changes in your branch , I made a separate branch coz I didn't have access to your repo 🙂

Signed-off-by: Shivansh-yadav13 <yadavshivansh@gmail.com>

slayerjain

LGTM. Thanks for the contribution @Shivansh-yadav13

github-actions bot reviewed Oct 9, 2022

View reviewed changes

slayerjain added a commit that referenced this pull request Oct 9, 2022

@Shivansh-yadav13 has signed the CLA from Pull Request #209

4499835

slayerjain requested review from re-Tick and Sarthak160 October 10, 2022 17:55

Shivansh-yadav13 force-pushed the fix_184 branch from 19256cb to 5e22eaf Compare October 16, 2022 10:43

Sarthak160 force-pushed the fix_184 branch from de3264a to 6356102 Compare October 18, 2022 08:31

Shivansh-yadav13 force-pushed the fix_184 branch from 6356102 to c6f1546 Compare October 19, 2022 06:43

Sarthak160 mentioned this pull request Oct 27, 2022

feat: mock data compress-decompress #225

Closed

12 tasks

Sarthak160 requested a review from slayerjain October 28, 2022 11:58

Shivansh-yadav13 added 2 commits November 2, 2022 15:53

feat: mock data compress-decompress

9a2e394

Signed-off-by: Shivansh-yadav13 <yadavshivansh@gmail.com>

Update mock.go

2ae4ae3

Sarthak160 force-pushed the fix_184 branch from 4f60f6c to 2ae4ae3 Compare November 2, 2022 10:23

slayerjain approved these changes Nov 3, 2022

View reviewed changes

slayerjain merged commit e55df0a into keploy:main Nov 3, 2022

github-actions bot locked and limited conversation to collaborators Nov 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: mock data compress-decompress #209

feat: mock data compress-decompress #209

Shivansh-yadav13 commented Oct 9, 2022 •

edited

Loading

github-actions bot commented Oct 9, 2022 •

edited

Loading

github-actions bot left a comment

Shivansh-yadav13 commented Oct 9, 2022

slayerjain commented Oct 10, 2022

re-Tick commented Oct 13, 2022

Sarthak160 commented Oct 17, 2022

Sarthak160 commented Oct 27, 2022 •

edited

Loading

Shivansh-yadav13 commented Oct 27, 2022

Sarthak160 commented Oct 27, 2022

slayerjain left a comment

feat: mock data compress-decompress #209

feat: mock data compress-decompress #209

Conversation

Shivansh-yadav13 commented Oct 9, 2022 • edited Loading

Related Issue

Type of change

Checklist:

Screenshots (if any)

github-actions bot commented Oct 9, 2022 • edited Loading

github-actions bot left a comment

Choose a reason for hiding this comment

Shivansh-yadav13 commented Oct 9, 2022

slayerjain commented Oct 10, 2022

re-Tick commented Oct 13, 2022

Sarthak160 commented Oct 17, 2022

Sarthak160 commented Oct 27, 2022 • edited Loading

Shivansh-yadav13 commented Oct 27, 2022

Sarthak160 commented Oct 27, 2022

slayerjain left a comment

Choose a reason for hiding this comment

Shivansh-yadav13 commented Oct 9, 2022 •

edited

Loading

github-actions bot commented Oct 9, 2022 •

edited

Loading

Sarthak160 commented Oct 27, 2022 •

edited

Loading