Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

release 4.3 #4449

Closed
wants to merge 340 commits into from
Closed

release 4.3 #4449

wants to merge 340 commits into from

Conversation

buger
Copy link
Member

@buger buger commented Nov 29, 2022 •
edited by jira bot

furkansenharputlu and others added 30 commits March 8, 2022 12:09
@furkansenharputlu
Fix the case where multiple securitySchemes exist (#3922)
825e72d 
(cherry picked from commit 773f702)
@buraksezer
[TT-4655] Federation crashes gateway during performance tests (#3910)
b9cc12f 
PR for [TT-4655](https://tyktech.atlassian.net/browse/TT-4655)

Basically, Tyk gateway creates a `memconn.Provider` to handle subgraphs for every request. it also creates a new HTTP client with a fresh `http.Transport`. 

It repeats this flow for every request and the dialer implementation in the Go HTTP library allocates new buffers to dial the upstream server. This creates a massive memory leak in the runtime.  I implemented a simple caching approach to overcome that problem. Now it uses a global `http.Client` instance and the dialer function gets a `memconn.Provider` for the subgraph. Before the fix, Tyk gateway had almost 12GB of memory on my local setup. With the fix, it’s stable at around 140 MB.

Users can rename the subgraph or delete it completely. If a subgraph is renamed, a new `memconn.Provider` will be created and cached.

From memory profiler:

![Screen Shot 2022-02-25 at 12 20 51](https://user-images.githubusercontent.com/33702537/155992064-dde90b72-77bf-45f4-9b75-86f40d0a4cd6.png)

(cherry picked from commit b8496b3)
@pvormste
update graphql-go-tools (#3926)
ee54fe8 
(cherry picked from commit cc952fe)
@furkansenharputlu
Auto-enable auth-source in security scheme (#3924)
3dcb459 
(cherry picked from commit 3e23111)
@furkansenharputlu
Move auth type constants to apidef package (#3927)
bd851af 
(cherry picked from commit 4edf523)
@ermirizio
TD-876/release-4-cleanup (#3918)
101b301 
* internal: unused moved files cleanup

* internal: fix smoke-test readme

* internal: change smoke-tests path
@furkansenharputlu
Iterate auth configs map in alphabetical order in auth source validat…
2737cc2 
…ion (#3928)

* Iterate auth configs map in alphabetical order in auth source validation

* Preallocate sorting slice

(cherry picked from commit 1bdd59c)
@furkansenharputlu
Refactor jwt OAS conversion (#3934)
25700d1 
(cherry picked from commit b405a7d)
@furkansenharputlu
Refactor basic auth OAS conversion (#3935)
7b0cc51 
(cherry picked from commit f9b132e)
@furkansenharputlu
[TT-4758] Implement OAS oauth2 scheme conversion (#3936)
e5b229c 
* Refactor basic auth OAS conversion

* Implement OAS oauth2 scheme conversion

(cherry picked from commit 92655ed)
@furkansenharputlu
[TT-4869] Implement base of paths and operations (#3940)
da49ad8 
* Implement base of paths and operations

* Add request changes

* Add requested change

(cherry picked from commit ceccf70)
@furkansenharputlu
Add TransformRequestMethod operation to OAS (#3941)
67645f4 
(cherry picked from commit 2d13500)
@furkansenharputlu
Add Cache operation to OAS (#3942)
480daa2 
(cherry picked from commit 29c3f7d)
@furkansenharputlu
Add EnforceTimeout operation to OAS (#3943)
7c1650d 
(cherry picked from commit e5cd319)
@furkansenharputlu
Refactor custom plugin OAS conversion (#3933)
d623062 
(cherry picked from commit 27c54b3)
@furkansenharputlu
Refactor go plugin OAS conversion (#3951)
76a130d 
(cherry picked from commit 9a68483)
@furkansenharputlu
Refactor oidc auth OAS conversion (#3950)
6d9b4a4
@ermirizio
internal: relocate and fix legacy scripts (#3952)
7073f9f 
* internal: relocate and fix legacy scripts

(cherry picked from commit e6dbfa2)
@buraksezer
update graphql-to-tools TT-4627 (#3962)
2abbcf5 
(cherry picked from commit 81638fe)
@furkansenharputlu
Cleanup OAS authentication code (#3963)
47565c0 
(cherry picked from commit 3e5d3e7)
@sredxny
TT-4934 make ctx.SetSession backwards compatible (#3967)
48b2825 
<!-- Provide a general summary of your changes in the Title above -->

## Description
In order to be fully backward compatible, then updated the signature of `ctx.SetSession` function, now it will infer from where to get the configuration if keys should be hashed or not. When a gw starts then it will override the function `Global func() Config` that resides inside the config package, in this way it will take automatically the configurtion of the current gateway. 

## Related Issue
[ticket](https://tyktech.atlassian.net/browse/TT-4934)

## Motivation and Context
Unblock release

## How This Has Been Tested
- Ran GW
- Created a key and consume api

## Screenshots (if appropriate)

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes that apply -->
<!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->
- [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own
      fork, don't request your `master`!
- [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start
      *your branch* off *our latest `master`*.
- [ ] My change requires a change to the documentation.
  - [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  - [ ] If new config option added, ensure that it can be set via ENV variable
- [ ] I have updated the documentation accordingly.
- [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor`
- [ ] When updating library version must provide reason/explanation for this update.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.
- [x] Check your code additions will not fail linting checks:
  - [x] `go fmt -s`
  - [ ] `go vet`

(cherry picked from commit 61ca289)
@jeffy-mathew
[TT-4838] Add ValidateOASObject method (#3945)
bf6c8b7 
* Add ValidateOASObject method

[changelog]
added: Add ValidateOASObject method to validate OAS definition against mulitple OAS versions

(cherry picked from commit 67a3af6)
@jeffy-mathew
Revert "[TT-4838] Add ValidateOASObject method (#3945)" (#3968)
52ff9cb 
This reverts commit 67a3af6.

(cherry picked from commit 4637fac)
@furkansenharputlu
Fix HMAC OAS conversion (#3970)
a7c60bb 
(cherry picked from commit 2c62163)
@furkansenharputlu
Implement OAS upstream certificates conversion (#3964)
e1e736e 
(cherry picked from commit 776e6db)
@jay-deshmukh
fix graphiql to use default theme (#3966)
ce75e7f 
(cherry picked from commit dd7b79e)
@furkansenharputlu
Implement OAS pinned public keys conversion (#3971)
f0304cd 
(cherry picked from commit 1d0190d)
@furkansenharputlu
Fix upstream certificate OAS conversion (#3982)
ff7dd1e 
(cherry picked from commit 0488478)
@jeffy-mathew
[TT-4838] ValidateOASObject method (#3973)
c39dfe3 
* added ValidateOASObject to validate OAS object based on parameter OAS version

[changelog]

added: validate OAS object against different OAS versions
@jeffy-mathew
[TT-4874] OAS schema endpoint (#3974)
d7ca5fd 
* Add schema endpoint to return quried OAS schema, return the latest patch version if requested version is not found

[changlog]
added: endpoint `/tyk/schema` to return quried OAS schema

(cherry picked from commit 16ad4f6)
furkansenharputlu and others added 29 commits October 17, 2022 15:39
@furkansenharputlu
Implement external OAuth OAS conversions (#4352)
fbef2ce 
(cherry picked from commit 0a5f8cb)
@vaske @jeffy-mathew
TT-1799 - in case of error in signing method show proper error message (
0704960 
#3751)

* TT-1799 - in case of error in signing method show proper error message

* TT-1799 - signature test

* TT-1799 - moved static text to constants

* TT-1799 - fixed linting issue

* TT-1799 - moved const to be inside test since it's test dependent
@pvormste
add configurations for new subscription protocols (#4351)
0d8b1cb
@jeffy-mathew
fix chaining external native API to internal OAS API. (#4358)
f2e01a0 
[changelog]
fixed: chaining external native API def to internal OAS API def.

(cherry picked from commit 425d9e1)
@buraksezer
[TT-6737] [TT-6798] Updated graphql-go-tools commit hash (#4362)
5cbd371 
[changelog]
internal: Updated graphql-go-tools commit hash.

(cherry picked from commit 3b8bc89)
@titpetric
[TT-6502] working around memory allocations in restricted memory envi…
e1faf54 
…ronments, rewriting nopCloser (#4299)

* Add flusher from #4021 PR
* Reuse bytes.Buffer for our ReaderWriterSeeker
* Only close/clear the reader when we copy it into the buffer without error
* Remove superflous readBody code from mw_redis_cache

(cherry picked from commit 9202894)
@furkansenharputlu
Implement OAuth introspection redis cache (#4356)
bd70729 
(cherry picked from commit 0f0b158)
@furkansenharputlu
[TT-6933] Implement introspection cache OAS conversion (#4369)
4e230bf 
* Implement introspection cache OAS conversion

* Make caching omittable

(cherry picked from commit 2dd015b)
@furkansenharputlu
Add GoDoc for OAS OAuth introspection fields (#4370)
181d878 
(cherry picked from commit 7d336b5)
@pvormste
add subscription type cfg for proxy-only and fix federation cfg gen (#…
3cefe73 
…4367)

* add subscription type cfg for proxy-only and fix federation cfg gen

* remove unused code

(cherry picked from commit 0d4a960)
@kofoworola
[TT-6575] feat: persisted queries (#4359)
4a88372 
* setting up the base middleware

* dynamically injecting variables

* added tests for default version and tests for context variables

* added test for version

* added path variable test, fixed bug with variable replace

* modified tests

* moved location of middleware

* handled unhandled errors.

Co-authored-by: asoorm <ahmet@mangomm.co.uk>
(cherry picked from commit eac6e1a)
@titpetric
[TT-6927] Improving DX with make lint, add faillint, PR template upda…
72c8a36 
…te (#4314)

* Add make lint command, update PR template, update ci-tests to use make lint

* Add -count 1 lines to prevent caching

* Fix goimports for analytics_test.go, broken on master

* Move go mod tidy before running linters

* Print full git diff for git-state

* Fix goimports issues on master
@titpetric
[TT-5744] OAS docs updates (#4327)
85afeaa 
* Updated apidef/oas to follow godoc, added docs for OAS doc generator, generator improvements
* Updated generated x-tyk-gateway schema markdown docs to latest
* Updated releated tests to doc generator

(cherry picked from commit c19518b)
@titpetric
[TT-6927] Improvement/goimports local (#4329)
0f2378e 
* Use goimports with -local to group tyk imports together
* Import changes made by goimports -local
@jeffy-mathew
[TT-6454] modify OAS servers methods. (#4373)
ac0f040 
* modify OAS servers methods.

[changelog]
added: OAS ReplaceServers.
changed: OAS AddServers.

* retain user added servers in replace OAS servers

(cherry picked from commit 4f04f5f)
@jeffy-mathew
change OAS AddServers signature to return error. (#4375)
3fe5f57 
[changelog]
changed: OAS AddServers.

(cherry picked from commit 501e860)
@furkansenharputlu
[TT-6919] Feature/introspection exp check (#4376)
9f4113b 
* Check the expiration of cached introspection response

* Added a comment to explain the trick in the test cases

(cherry picked from commit 3914295)
@furkansenharputlu
Check whether introspection has exp val (#4381)
715f5a7 
(cherry picked from commit 7ebf644)
@pvormste
add omitempty to SubscriptionType (#4382)
64f9bf2 
(cherry picked from commit be0b657)
@furkansenharputlu
Add hidden version base id (#4383)
2136e04 
(cherry picked from commit f86b66b)
@furkansenharputlu
Remove version change actions from apidef (#4386)
2ea6002 
(cherry picked from commit e1eeaf6)
@jeffy-mathew
Revert "change OAS AddServers signature to return error. (#4375)" (#4387
0a415e9 
)

This reverts commit 501e860.
Allow empty servers in OAS AddServers.

(cherry picked from commit 550850b)
@pvormste
fix non-working single flight (#4402) (#4404)
433daa0
@furkansenharputlu
Convert Error log to Debug in JWT API while searching OAuth client (#…
9cf56fd 
…4406) (#4410)
@furkansenharputlu
Change swagger version to 4.3 (#4423)
6614ce5
@furkansenharputlu
Change swagger version to 4.3.0 (#4426)
22ab28f 
[changelog]
internal: Changed swagger version to 4.3.0.
@sredxny @maciejwojciechowski @furkansenharputlu
Ensure that version used my plugin loader has the correct syntax (#4417…
377c6b7 
…) (#4427)


* ensure that version used my plugin loader has the correct syntax

* allow goplugin loader to load plugin without the v prefix in version

Co-authored-by: maciej <39672152+maciejwojciechowski@users.noreply.github.com>
Co-authored-by: Furkan Senharputlu <furkan_senharputlu@hotmail.com>
@titpetric
Fix middleware order, enabling virtual endpoint caching and cache con…
747130c 
…trol from jsvm (#4397) (#4431)

Co-authored-by: Tit Petric <tit@tyk.io>

Co-authored-by: Tit Petric <tit@tyk.io>
@furkansenharputlu
Fix the panic while loading Go-plugin (#4416) (#4435)
30881fd
@buger buger closed this Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet