-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Properly handling big request bodies in Admin API #702
Conversation
7351c2b
to
0cf1e4c
Compare
@@ -0,0 +1 @@ | |||
27.112.32.0/19,31.220.46.0/23,37.9.53.0/24,41.71.144.0/21,41.71.171.0/24,41.71.176.0/23,41.71.178.0/24,41.71.184.0/22,41.71.188.0/23,41.71.190.0/24,41.71.216.0/23,41.138.164.0/22,41.138.168.0/23,41.138.171.0/24,41.138.172.0/23,41.138.175.0/24,41.138.176.0/20,41.198.80.0/20,41.198.224.0/20,41.203.64.0/24,41.203.67.0/24,41.203.69.0/24,42.140.0.0/17,43.57.0.0/16,43.181.0.0/16,46.151.48.0/22,46.151.52.0/22,46.166.131.0/24,46.244.10.0/24,58.2.0.0/17,62.112.16.0/21,80.114.192.0/18,81.94.43.0/24,85.239.149.0/24,91.207.7.0/24,91.240.128.0/24,103.37.86.0/24,103.235.174.0/23,119.227.224.0/19,120.46.0.0/15,120.64.0.0/16,120.67.0.0/16,120.128.128.0/17,120.128.192.0/18,120.129.0.0/17,120.129.128.0/17,120.130.0.0/16,120.130.128.0/18,153.85.0.0/16,177.47.126.0/24,178.212.56.0/22,181.118.32.0/24,188.247.232.0/24,193.86.21.0/24,195.214.249.0/24,195.238.180.0/24,195.238.181.0/24,195.238.182.0/24,196.196.8.0/22,198.89.232.0/24,198.89.233.0/24,199.15.232.0/21,203.119.116.0/22,204.8.87.0/24,208.38.135.0/24,210.57.0.0/19,210.57.128.0/18,210.57.192.0/20,212.95.144.0/22,1.4.0.0/17,1.10.16.0/20,1.116.0.0/14,5.8.62.0/24,5.34.242.0/23,5.72.0.0/14,5.134.128.0/19,14.4.0.0/14,14.245.0.0/16,23.235.48.0/20,27.111.48.0/20,27.122.32.0/20,27.126.160.0/20,31.11.43.0/24,31.222.200.0/21,36.0.8.0/21,36.37.48.0/20,37.139.49.0/24,37.148.216.0/21,37.246.0.0/16,41.72.64.0/19,42.0.32.0/19,42.1.56.0/22,42.1.128.0/17,42.52.0.0/14,42.83.80.0/22,42.96.0.0/18,42.123.36.0/22,42.128.0.0/12,42.160.0.0/12,42.194.8.0/22,42.194.12.0/22,42.194.128.0/17,43.229.52.0/22,43.236.0.0/16,43.250.64.0/22,43.250.116.0/22,43.252.80.0/22,43.252.152.0/22,43.252.180.0/22,43.255.188.0/22,45.64.88.0/22,45.117.208.0/22,45.121.144.0/22,46.29.248.0/22,46.29.248.0/21,46.148.112.0/20,46.151.48.0/21,46.232.192.0/21,46.243.140.0/24,46.243.142.0/24,49.8.0.0/14,49.128.104.0/22,58.87.64.0/18,59.254.0.0/15,60.233.0.0/16,61.11.224.0/19,61.13.128.0/17,61.45.251.0/24,62.182.152.0/21,64.234.224.0/20,66.231.64.0/20,67.211.208.0/20,67.213.128.0/20,67.218.208.0/20,72.13.16.0/20,78.31.184.0/21,78.31.211.0/24,79.173.104.0/21,80.76.8.0/21,81.22.152.0/23,83.175.0.0/18,85.121.39.0/24,86.55.40.0/23,86.55.42.0/23,88.135.16.0/20,91.194.254.0/23,91.195.254.0/23,91.198.127.0/24,91.200.248.0/22,91.203.20.0/22,91.207.4.0/22,91.207.116.0/23,91.209.12.0/24,91.212.104.0/24,91.212.198.0/24,91.212.201.0/24,91.212.220.0/24,91.213.126.0/24,91.213.172.0/24,91.216.3.0/24,91.217.10.0/23,91.220.35.0/24,91.220.62.0/24,91.220.163.0/24,91.223.89.0/24,91.226.97.0/24,91.229.210.0/24,91.230.110.0/24,91.230.252.0/23,91.234.36.0/24,91.235.2.0/24,91.236.74.0/23,91.236.120.0/24,91.237.198.0/24,91.238.82.0/24,91.239.24.0/24,91.239.238.0/24,91.240.163.0/24,91.240.165.0/24,91.242.217.0/24,91.243.115.0/24,93.175.240.0/20,94.26.112.0/20,94.154.128.0/18,95.216.0.0/15,101.192.0.0/14,101.199.0.0/16,101.202.0.0/16,101.203.128.0/19,101.248.0.0/15,101.252.0.0/15,103.2.44.0/22,103.10.68.0/22,103.12.216.0/22,103.16.76.0/24,103.20.36.0/22,103.23.8.0/22,103.36.64.0/22,103.41.124.0/22,103.41.180.0/22,103.42.115.0/24,103.55.28.0/22,103.57.248.0/22,103.61.4.0/22,103.228.60.0/22,103.229.36.0/22,103.230.144.0/22,103.231.84.0/22,103.232.136.0/22,103.232.172.0/22,103.236.32.0/22,103.242.184.0/22,104.143.112.0/20,104.255.136.0/21,106.96.0.0/14,108.166.224.0/19,109.94.208.0/20,110.44.128.0/20,110.232.160.0/20,113.20.160.0/19,114.8.0.0/16,115.85.133.0/24,116.78.0.0/15,116.128.0.0/10,116.144.0.0/15,116.146.0.0/15,117.100.0.0/15,118.177.0.0/16,118.185.0.0/16,119.232.0.0/16,120.48.0.0/15,120.92.0.0/17,120.92.128.0/18,120.92.192.0/19,120.92.224.0/20,121.100.128.0/18,122.129.0.0/18,122.202.96.0/19,123.136.80.0/20,124.68.0.0/15,124.70.0.0/15,124.157.0.0/18,124.242.0.0/16,124.245.0.0/16,125.31.192.0/18,125.58.0.0/18,125.169.0.0/16,128.13.0.0/16,128.168.0.0/16,128.191.0.0/16,129.47.0.0/16,129.76.64.0/18,130.148.0.0/16,130.196.0.0/16,130.201.0.0/16,130.222.0.0/16,131.100.148.0/22,132.145.0.0/16,132.232.0.0/16,132.240.0.0/16,134.18.0.0/16,134.22.0.0/16,134.23.0.0/16,134.33.0.0/16,134.73.0.0/16,134.127.0.0/16,134.172.0.0/16,134.209.0.0/16,136.228.0.0/16,136.230.0.0/16,137.76.0.0/16,137.105.0.0/16,137.171.0.0/16,138.36.148.0/22,138.43.0.0/16,138.128.224.0/19,138.200.0.0/16,138.216.0.0/16,138.249.0.0/16,139.47.0.0/16,139.167.0.0/16,139.188.0.0/16,140.143.128.0/17,140.167.0.0/16,140.204.0.0/16,141.136.16.0/24,141.136.22.0/24,141.136.27.0/24,141.178.0.0/16,141.253.0.0/16,143.49.0.0/16,143.64.0.0/16,143.135.0.0/16,143.189.0.0/16,144.207.0.0/16,145.231.0.0/16,146.3.0.0/16,147.7.0.0/16,147.119.0.0/16,147.220.0.0/16,148.154.0.0/16,148.178.0.0/16,148.185.0.0/16,148.248.0.0/16,149.109.0.0/16,149.114.0.0/16,149.118.0.0/16,149.143.64.0/18,150.10.0.0/16,150.22.128.0/17,150.25.0.0/16,150.40.0.0/16,150.107.106.0/23,150.107.220.0/22,150.121.0.0/16,150.126.0.0/16,150.129.136.0/22,150.141.0.0/16,150.230.0.0/16,150.242.36.0/22,151.123.0.0/16,151.192.0.0/16,151.212.0.0/16,151.237.184.0/22,152.136.0.0/16,152.147.0.0/16,153.14.0.0/16,153.93.0.0/16,155.40.0.0/16,155.66.0.0/16,155.73.0.0/16,155.108.0.0/16,155.204.0.0/16,155.249.0.0/16,157.115.0.0/16,157.162.0.0/16,157.186.0.0/16,157.195.0.0/16,157.231.0.0/16,157.232.0.0/16,158.54.0.0/16,158.58.0.0/17,158.90.0.0/17,159.85.0.0/16,159.100.0.0/18,159.111.0.0/16,159.135.0.0/16,159.151.0.0/16,159.219.0.0/16,159.223.0.0/16,159.229.0.0/16,160.14.0.0/16,160.21.0.0/16,160.180.0.0/16,160.181.0.0/16,160.200.0.0/16,160.222.0.0/16,160.235.0.0/16,160.240.0.0/16,160.255.0.0/16,161.59.0.0/16,161.66.0.0/16,161.71.0.0/16,161.189.0.0/16,161.232.0.0/16,162.211.236.0/22,163.47.19.0/24,163.50.0.0/16,163.58.0.0/16,163.59.0.0/16,163.182.0.0/16,163.227.128.0/21,163.254.0.0/16,164.6.0.0/16,164.60.0.0/16,164.137.0.0/16,165.102.0.0/16,165.192.0.0/16,165.205.0.0/16,165.209.0.0/16,167.74.0.0/18,167.87.0.0/16,167.97.0.0/16,167.103.0.0/16,167.162.0.0/16,167.175.0.0/16,167.224.0.0/19,168.129.0.0/16,170.67.0.0/16,170.113.0.0/16,170.114.0.0/16,170.120.0.0/16,170.179.0.0/16,171.22.0.0/16,171.25.0.0/17,171.26.0.0/16,172.103.64.0/18,175.103.64.0/18,176.47.0.0/16,176.61.136.0/22,176.61.136.0/21,176.65.128.0/17,176.97.116.0/22,176.97.152.0/22,177.36.16.0/20,177.74.160.0/20,178.159.176.0/20,178.216.48.0/21,180.178.192.0/18,180.236.0.0/14,181.118.32.0/19,185.3.132.0/22,185.11.140.0/24,185.11.143.0/24,185.68.156.0/22,185.72.68.0/22,185.75.56.0/22,185.93.187.0/24,186.1.128.0/19,186.96.96.0/19,186.148.160.0/19,186.195.224.0/20,188.239.128.0/18,188.247.135.0/24,188.247.230.0/24,190.2.208.0/21,190.9.48.0/21,190.13.80.0/21,192.5.103.0/24,192.12.131.0/24,192.26.25.0/24,192.31.212.0/23,192.40.29.0/24,192.43.153.0/24,192.43.154.0/23,192.43.156.0/22,192.43.160.0/24,192.43.175.0/24,192.43.176.0/21,192.43.184.0/24,192.54.39.0/24,192.54.73.0/24,192.54.110.0/24,192.67.16.0/24,192.67.160.0/22,192.84.243.0/24,192.86.85.0/24,192.88.74.0/24,192.100.142.0/24,192.101.44.0/24,192.101.181.0/24,192.101.200.0/21,192.101.240.0/21,192.101.248.0/23,192.125.0.0/17,192.133.3.0/24,192.152.0.0/24,192.152.194.0/24,192.154.11.0/24,192.158.51.0/24,192.160.44.0/24,192.171.64.0/19,192.189.25.0/24,192.190.49.0/24,192.190.97.0/24,192.195.150.0/24,192.197.87.0/24,192.203.252.0/24,192.206.114.0/24,192.219.120.0/21,192.219.128.0/18,192.219.192.0/20,192.219.208.0/21,192.226.16.0/20,192.229.32.0/19,192.231.66.0/24,192.234.189.0/24,192.245.101.0/24,193.0.129.0/24,193.23.126.0/24,193.25.48.0/20,193.26.64.0/19,193.43.134.0/24,193.104.41.0/24,193.104.94.0/24,193.104.110.0/24,193.105.207.0/24,193.105.245.0/24,193.107.16.0/22,193.138.244.0/22,193.139.0.0/16,193.150.120.0/24,193.164.11.0/24,193.177.64.0/18,193.189.116.0/23,193.222.50.0/24,193.243.0.0/17,194.0.177.0/24,194.1.152.0/24,194.29.185.0/24,194.38.0.0/18,194.50.116.0/24,194.54.156.0/22,194.110.160.0/22,195.20.141.0/24,195.68.222.0/23,195.78.108.0/23,195.88.190.0/23,195.182.57.0/24,195.190.13.0/24,195.191.56.0/23,195.191.102.0/23,195.225.176.0/22,195.238.180.0/22,196.1.109.0/24,196.42.128.0/17,196.63.0.0/16,196.188.0.0/14,196.193.0.0/16,196.247.0.0/16,197.154.0.0/16,198.13.0.0/20,198.14.128.0/19,198.14.160.0/19,198.20.16.0/20,198.23.32.0/20,198.45.32.0/20,198.45.64.0/19,198.48.16.0/20,198.56.64.0/18,198.57.64.0/20,198.62.70.0/24,198.62.76.0/24,198.96.224.0/20,198.99.117.0/24,198.102.222.0/24,198.148.212.0/24,198.151.16.0/20,198.151.64.0/18,198.151.152.0/22,198.160.205.0/24,198.162.208.0/20,198.169.201.0/24,198.177.175.0/24,198.177.176.0/22,198.177.180.0/24,198.177.214.0/24,198.178.64.0/19,198.179.22.0/24,198.181.32.0/20,198.181.64.0/19,198.183.32.0/19,198.184.193.0/24,198.184.208.0/24,198.186.25.0/24,198.186.208.0/24,198.187.64.0/18,198.187.192.0/24,198.190.173.0/24,198.199.212.0/24,198.202.237.0/24,198.204.0.0/21,198.205.64.0/19,198.212.132.0/24,199.5.152.0/23,199.5.229.0/24,199.26.96.0/19,199.26.137.0/24,199.26.207.0/24,199.26.251.0/24,199.33.145.0/24,199.33.222.0/24,199.34.128.0/18,199.46.32.0/19,199.58.248.0/21,199.60.102.0/24,199.71.56.0/21,199.71.192.0/20,199.84.55.0/24,199.84.56.0/22,199.84.60.0/24,199.84.64.0/19,199.87.208.0/21,199.88.32.0/20,199.88.48.0/22,199.89.16.0/20,199.89.198.0/24,199.120.163.0/24,199.165.32.0/19,199.166.200.0/22,199.184.82.0/24,199.185.192.0/20,199.196.192.0/19,199.198.160.0/20,199.198.176.0/21,199.198.184.0/23,199.198.188.0/22,199.200.64.0/19,199.212.96.0/20,199.223.0.0/20,199.230.64.0/19,199.230.96.0/21,199.233.85.0/24,199.233.96.0/24,199.245.138.0/24,199.246.137.0/24,199.246.213.0/24,199.246.215.0/24,199.248.64.0/18,199.249.64.0/19,199.253.32.0/20,199.253.48.0/21,199.253.224.0/20,199.254.32.0/20,200.3.128.0/20,200.22.0.0/16,201.169.0.0/16,201.182.0.0/16,202.0.192.0/18,202.20.32.0/19,202.21.64.0/19,202.27.96.0/23,202.27.98.0/24,202.27.99.0/24,202.27.100.0/22,202.27.120.0/22,202.27.161.0/24,202.27.162.0/23,202.27.164.0/22,202.27.168.0/24,202.39.112.0/20,202.40.32.0/19,202.40.64.0/18,202.61.108.0/24,202.68.0.0/18,202.80.152.0/21,202.148.32.0/20,202.148.176.0/20,202.183.0.0/19,202.189.80.0/20,203.0.116.0/22,203.2.200.0/22,203.9.0.0/19,203.31.88.0/23,203.34.70.0/23,203.34.71.0/24,203.86.252.0/22,203.148.80.0/22,203.149.92.0/22,203.169.0.0/22,203.189.112.0/22,203.191.64.0/18,204.19.38.0/23,204.44.32.0/20,204.44.192.0/20,204.44.224.0/20,204.48.16.0/20,204.52.255.0/24,204.57.16.0/20,204.75.147.0/24,204.75.228.0/24,204.80.198.0/24,204.86.16.0/20,204.87.199.0/24 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of having to deal with such a fixture, simply use string.rep(bytes)
to build a fixture body.
a6643c1
to
2531bc6
Compare
for i=1, 1000 do | ||
table.insert(big_value, "204.48.16.0") | ||
end | ||
local big_value_str = table.concat(big_value, ",") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@thibaultcha I opted for this way instead to build a JSON value in memory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would have been cleaner to use string.rep until a certain body size is needed. That is what the Galileo buffer tests are doing for example.
--
Thibault Charbonnier
Sent with Airmail
On November 9, 2015 at 2:43:08 PM, Marco Palladino (notifications@github.com) wrote:
In spec/integration/admin_api/kong_routes_spec.lua:
@@ -83,4 +84,42 @@ describe("Admin API", function()
assert.truthy(body.server.total_requests)
end)
end)
+
- describe("Request size", function()
- it("should properly hanlde big POST bodies < 10MB", function()
local response, status = http_client.post(spec_helper.API_URL.."/apis", { request_path = "hello.com", upstream_url = "http://mockbin.org" })
assert.equal(201, status)
local api_id = json.decode(response).id
assert.truthy(api_id)
local big_value = {}
for i=1, 1000 do
table.insert(big_value, "204.48.16.0")
end
@thibaultcha I opted for this way instead to build a JSON value in memorylocal big_value_str = table.concat(big_value, ",")
—
Reply to this email directly or view it on GitHub.
926b332
to
68e75d4
Compare
Properly handling big request bodies in Admin API
@@ -1,6 +1,7 @@ | |||
local json = require "cjson" | |||
local http_client = require "kong.tools.http_client" | |||
local spec_helper = require "spec.spec_helpers" | |||
local IO = require "kong.tools.io" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nope - will remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
damn, spotted it yesterday and forgot about it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
5 points for Gryffindor
This sets the limit of request bodies to the Admin API to 10MB.
Closes #700.