Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly handling big request bodies in Admin API #702

Merged
merged 1 commit into from
Nov 10, 2015

Conversation

subnetmarco
Copy link
Member

This sets the limit of request bodies to the Admin API to 10MB.

Closes #700.

@subnetmarco subnetmarco self-assigned this Nov 9, 2015
@subnetmarco subnetmarco force-pushed the fix/admin-request-body branch 2 times, most recently from 7351c2b to 0cf1e4c Compare November 9, 2015 22:15
@@ -0,0 +1 @@
27.112.32.0/19,31.220.46.0/23,37.9.53.0/24,41.71.144.0/21,41.71.171.0/24,41.71.176.0/23,41.71.178.0/24,41.71.184.0/22,41.71.188.0/23,41.71.190.0/24,41.71.216.0/23,41.138.164.0/22,41.138.168.0/23,41.138.171.0/24,41.138.172.0/23,41.138.175.0/24,41.138.176.0/20,41.198.80.0/20,41.198.224.0/20,41.203.64.0/24,41.203.67.0/24,41.203.69.0/24,42.140.0.0/17,43.57.0.0/16,43.181.0.0/16,46.151.48.0/22,46.151.52.0/22,46.166.131.0/24,46.244.10.0/24,58.2.0.0/17,62.112.16.0/21,80.114.192.0/18,81.94.43.0/24,85.239.149.0/24,91.207.7.0/24,91.240.128.0/24,103.37.86.0/24,103.235.174.0/23,119.227.224.0/19,120.46.0.0/15,120.64.0.0/16,120.67.0.0/16,120.128.128.0/17,120.128.192.0/18,120.129.0.0/17,120.129.128.0/17,120.130.0.0/16,120.130.128.0/18,153.85.0.0/16,177.47.126.0/24,178.212.56.0/22,181.118.32.0/24,188.247.232.0/24,193.86.21.0/24,195.214.249.0/24,195.238.180.0/24,195.238.181.0/24,195.238.182.0/24,196.196.8.0/22,198.89.232.0/24,198.89.233.0/24,199.15.232.0/21,203.119.116.0/22,204.8.87.0/24,208.38.135.0/24,210.57.0.0/19,210.57.128.0/18,210.57.192.0/20,212.95.144.0/22,1.4.0.0/17,1.10.16.0/20,1.116.0.0/14,5.8.62.0/24,5.34.242.0/23,5.72.0.0/14,5.134.128.0/19,14.4.0.0/14,14.245.0.0/16,23.235.48.0/20,27.111.48.0/20,27.122.32.0/20,27.126.160.0/20,31.11.43.0/24,31.222.200.0/21,36.0.8.0/21,36.37.48.0/20,37.139.49.0/24,37.148.216.0/21,37.246.0.0/16,41.72.64.0/19,42.0.32.0/19,42.1.56.0/22,42.1.128.0/17,42.52.0.0/14,42.83.80.0/22,42.96.0.0/18,42.123.36.0/22,42.128.0.0/12,42.160.0.0/12,42.194.8.0/22,42.194.12.0/22,42.194.128.0/17,43.229.52.0/22,43.236.0.0/16,43.250.64.0/22,43.250.116.0/22,43.252.80.0/22,43.252.152.0/22,43.252.180.0/22,43.255.188.0/22,45.64.88.0/22,45.117.208.0/22,45.121.144.0/22,46.29.248.0/22,46.29.248.0/21,46.148.112.0/20,46.151.48.0/21,46.232.192.0/21,46.243.140.0/24,46.243.142.0/24,49.8.0.0/14,49.128.104.0/22,58.87.64.0/18,59.254.0.0/15,60.233.0.0/16,61.11.224.0/19,61.13.128.0/17,61.45.251.0/24,62.182.152.0/21,64.234.224.0/20,66.231.64.0/20,67.211.208.0/20,67.213.128.0/20,67.218.208.0/20,72.13.16.0/20,78.31.184.0/21,78.31.211.0/24,79.173.104.0/21,80.76.8.0/21,81.22.152.0/23,83.175.0.0/18,85.121.39.0/24,86.55.40.0/23,86.55.42.0/23,88.135.16.0/20,91.194.254.0/23,91.195.254.0/23,91.198.127.0/24,91.200.248.0/22,91.203.20.0/22,91.207.4.0/22,91.207.116.0/23,91.209.12.0/24,91.212.104.0/24,91.212.198.0/24,91.212.201.0/24,91.212.220.0/24,91.213.126.0/24,91.213.172.0/24,91.216.3.0/24,91.217.10.0/23,91.220.35.0/24,91.220.62.0/24,91.220.163.0/24,91.223.89.0/24,91.226.97.0/24,91.229.210.0/24,91.230.110.0/24,91.230.252.0/23,91.234.36.0/24,91.235.2.0/24,91.236.74.0/23,91.236.120.0/24,91.237.198.0/24,91.238.82.0/24,91.239.24.0/24,91.239.238.0/24,91.240.163.0/24,91.240.165.0/24,91.242.217.0/24,91.243.115.0/24,93.175.240.0/20,94.26.112.0/20,94.154.128.0/18,95.216.0.0/15,101.192.0.0/14,101.199.0.0/16,101.202.0.0/16,101.203.128.0/19,101.248.0.0/15,101.252.0.0/15,103.2.44.0/22,103.10.68.0/22,103.12.216.0/22,103.16.76.0/24,103.20.36.0/22,103.23.8.0/22,103.36.64.0/22,103.41.124.0/22,103.41.180.0/22,103.42.115.0/24,103.55.28.0/22,103.57.248.0/22,103.61.4.0/22,103.228.60.0/22,103.229.36.0/22,103.230.144.0/22,103.231.84.0/22,103.232.136.0/22,103.232.172.0/22,103.236.32.0/22,103.242.184.0/22,104.143.112.0/20,104.255.136.0/21,106.96.0.0/14,108.166.224.0/19,109.94.208.0/20,110.44.128.0/20,110.232.160.0/20,113.20.160.0/19,114.8.0.0/16,115.85.133.0/24,116.78.0.0/15,116.128.0.0/10,116.144.0.0/15,116.146.0.0/15,117.100.0.0/15,118.177.0.0/16,118.185.0.0/16,119.232.0.0/16,120.48.0.0/15,120.92.0.0/17,120.92.128.0/18,120.92.192.0/19,120.92.224.0/20,121.100.128.0/18,122.129.0.0/18,122.202.96.0/19,123.136.80.0/20,124.68.0.0/15,124.70.0.0/15,124.157.0.0/18,124.242.0.0/16,124.245.0.0/16,125.31.192.0/18,125.58.0.0/18,125.169.0.0/16,128.13.0.0/16,128.168.0.0/16,128.191.0.0/16,129.47.0.0/16,129.76.64.0/18,130.148.0.0/16,130.196.0.0/16,130.201.0.0/16,130.222.0.0/16,131.100.148.0/22,132.145.0.0/16,132.232.0.0/16,132.240.0.0/16,134.18.0.0/16,134.22.0.0/16,134.23.0.0/16,134.33.0.0/16,134.73.0.0/16,134.127.0.0/16,134.172.0.0/16,134.209.0.0/16,136.228.0.0/16,136.230.0.0/16,137.76.0.0/16,137.105.0.0/16,137.171.0.0/16,138.36.148.0/22,138.43.0.0/16,138.128.224.0/19,138.200.0.0/16,138.216.0.0/16,138.249.0.0/16,139.47.0.0/16,139.167.0.0/16,139.188.0.0/16,140.143.128.0/17,140.167.0.0/16,140.204.0.0/16,141.136.16.0/24,141.136.22.0/24,141.136.27.0/24,141.178.0.0/16,141.253.0.0/16,143.49.0.0/16,143.64.0.0/16,143.135.0.0/16,143.189.0.0/16,144.207.0.0/16,145.231.0.0/16,146.3.0.0/16,147.7.0.0/16,147.119.0.0/16,147.220.0.0/16,148.154.0.0/16,148.178.0.0/16,148.185.0.0/16,148.248.0.0/16,149.109.0.0/16,149.114.0.0/16,149.118.0.0/16,149.143.64.0/18,150.10.0.0/16,150.22.128.0/17,150.25.0.0/16,150.40.0.0/16,150.107.106.0/23,150.107.220.0/22,150.121.0.0/16,150.126.0.0/16,150.129.136.0/22,150.141.0.0/16,150.230.0.0/16,150.242.36.0/22,151.123.0.0/16,151.192.0.0/16,151.212.0.0/16,151.237.184.0/22,152.136.0.0/16,152.147.0.0/16,153.14.0.0/16,153.93.0.0/16,155.40.0.0/16,155.66.0.0/16,155.73.0.0/16,155.108.0.0/16,155.204.0.0/16,155.249.0.0/16,157.115.0.0/16,157.162.0.0/16,157.186.0.0/16,157.195.0.0/16,157.231.0.0/16,157.232.0.0/16,158.54.0.0/16,158.58.0.0/17,158.90.0.0/17,159.85.0.0/16,159.100.0.0/18,159.111.0.0/16,159.135.0.0/16,159.151.0.0/16,159.219.0.0/16,159.223.0.0/16,159.229.0.0/16,160.14.0.0/16,160.21.0.0/16,160.180.0.0/16,160.181.0.0/16,160.200.0.0/16,160.222.0.0/16,160.235.0.0/16,160.240.0.0/16,160.255.0.0/16,161.59.0.0/16,161.66.0.0/16,161.71.0.0/16,161.189.0.0/16,161.232.0.0/16,162.211.236.0/22,163.47.19.0/24,163.50.0.0/16,163.58.0.0/16,163.59.0.0/16,163.182.0.0/16,163.227.128.0/21,163.254.0.0/16,164.6.0.0/16,164.60.0.0/16,164.137.0.0/16,165.102.0.0/16,165.192.0.0/16,165.205.0.0/16,165.209.0.0/16,167.74.0.0/18,167.87.0.0/16,167.97.0.0/16,167.103.0.0/16,167.162.0.0/16,167.175.0.0/16,167.224.0.0/19,168.129.0.0/16,170.67.0.0/16,170.113.0.0/16,170.114.0.0/16,170.120.0.0/16,170.179.0.0/16,171.22.0.0/16,171.25.0.0/17,171.26.0.0/16,172.103.64.0/18,175.103.64.0/18,176.47.0.0/16,176.61.136.0/22,176.61.136.0/21,176.65.128.0/17,176.97.116.0/22,176.97.152.0/22,177.36.16.0/20,177.74.160.0/20,178.159.176.0/20,178.216.48.0/21,180.178.192.0/18,180.236.0.0/14,181.118.32.0/19,185.3.132.0/22,185.11.140.0/24,185.11.143.0/24,185.68.156.0/22,185.72.68.0/22,185.75.56.0/22,185.93.187.0/24,186.1.128.0/19,186.96.96.0/19,186.148.160.0/19,186.195.224.0/20,188.239.128.0/18,188.247.135.0/24,188.247.230.0/24,190.2.208.0/21,190.9.48.0/21,190.13.80.0/21,192.5.103.0/24,192.12.131.0/24,192.26.25.0/24,192.31.212.0/23,192.40.29.0/24,192.43.153.0/24,192.43.154.0/23,192.43.156.0/22,192.43.160.0/24,192.43.175.0/24,192.43.176.0/21,192.43.184.0/24,192.54.39.0/24,192.54.73.0/24,192.54.110.0/24,192.67.16.0/24,192.67.160.0/22,192.84.243.0/24,192.86.85.0/24,192.88.74.0/24,192.100.142.0/24,192.101.44.0/24,192.101.181.0/24,192.101.200.0/21,192.101.240.0/21,192.101.248.0/23,192.125.0.0/17,192.133.3.0/24,192.152.0.0/24,192.152.194.0/24,192.154.11.0/24,192.158.51.0/24,192.160.44.0/24,192.171.64.0/19,192.189.25.0/24,192.190.49.0/24,192.190.97.0/24,192.195.150.0/24,192.197.87.0/24,192.203.252.0/24,192.206.114.0/24,192.219.120.0/21,192.219.128.0/18,192.219.192.0/20,192.219.208.0/21,192.226.16.0/20,192.229.32.0/19,192.231.66.0/24,192.234.189.0/24,192.245.101.0/24,193.0.129.0/24,193.23.126.0/24,193.25.48.0/20,193.26.64.0/19,193.43.134.0/24,193.104.41.0/24,193.104.94.0/24,193.104.110.0/24,193.105.207.0/24,193.105.245.0/24,193.107.16.0/22,193.138.244.0/22,193.139.0.0/16,193.150.120.0/24,193.164.11.0/24,193.177.64.0/18,193.189.116.0/23,193.222.50.0/24,193.243.0.0/17,194.0.177.0/24,194.1.152.0/24,194.29.185.0/24,194.38.0.0/18,194.50.116.0/24,194.54.156.0/22,194.110.160.0/22,195.20.141.0/24,195.68.222.0/23,195.78.108.0/23,195.88.190.0/23,195.182.57.0/24,195.190.13.0/24,195.191.56.0/23,195.191.102.0/23,195.225.176.0/22,195.238.180.0/22,196.1.109.0/24,196.42.128.0/17,196.63.0.0/16,196.188.0.0/14,196.193.0.0/16,196.247.0.0/16,197.154.0.0/16,198.13.0.0/20,198.14.128.0/19,198.14.160.0/19,198.20.16.0/20,198.23.32.0/20,198.45.32.0/20,198.45.64.0/19,198.48.16.0/20,198.56.64.0/18,198.57.64.0/20,198.62.70.0/24,198.62.76.0/24,198.96.224.0/20,198.99.117.0/24,198.102.222.0/24,198.148.212.0/24,198.151.16.0/20,198.151.64.0/18,198.151.152.0/22,198.160.205.0/24,198.162.208.0/20,198.169.201.0/24,198.177.175.0/24,198.177.176.0/22,198.177.180.0/24,198.177.214.0/24,198.178.64.0/19,198.179.22.0/24,198.181.32.0/20,198.181.64.0/19,198.183.32.0/19,198.184.193.0/24,198.184.208.0/24,198.186.25.0/24,198.186.208.0/24,198.187.64.0/18,198.187.192.0/24,198.190.173.0/24,198.199.212.0/24,198.202.237.0/24,198.204.0.0/21,198.205.64.0/19,198.212.132.0/24,199.5.152.0/23,199.5.229.0/24,199.26.96.0/19,199.26.137.0/24,199.26.207.0/24,199.26.251.0/24,199.33.145.0/24,199.33.222.0/24,199.34.128.0/18,199.46.32.0/19,199.58.248.0/21,199.60.102.0/24,199.71.56.0/21,199.71.192.0/20,199.84.55.0/24,199.84.56.0/22,199.84.60.0/24,199.84.64.0/19,199.87.208.0/21,199.88.32.0/20,199.88.48.0/22,199.89.16.0/20,199.89.198.0/24,199.120.163.0/24,199.165.32.0/19,199.166.200.0/22,199.184.82.0/24,199.185.192.0/20,199.196.192.0/19,199.198.160.0/20,199.198.176.0/21,199.198.184.0/23,199.198.188.0/22,199.200.64.0/19,199.212.96.0/20,199.223.0.0/20,199.230.64.0/19,199.230.96.0/21,199.233.85.0/24,199.233.96.0/24,199.245.138.0/24,199.246.137.0/24,199.246.213.0/24,199.246.215.0/24,199.248.64.0/18,199.249.64.0/19,199.253.32.0/20,199.253.48.0/21,199.253.224.0/20,199.254.32.0/20,200.3.128.0/20,200.22.0.0/16,201.169.0.0/16,201.182.0.0/16,202.0.192.0/18,202.20.32.0/19,202.21.64.0/19,202.27.96.0/23,202.27.98.0/24,202.27.99.0/24,202.27.100.0/22,202.27.120.0/22,202.27.161.0/24,202.27.162.0/23,202.27.164.0/22,202.27.168.0/24,202.39.112.0/20,202.40.32.0/19,202.40.64.0/18,202.61.108.0/24,202.68.0.0/18,202.80.152.0/21,202.148.32.0/20,202.148.176.0/20,202.183.0.0/19,202.189.80.0/20,203.0.116.0/22,203.2.200.0/22,203.9.0.0/19,203.31.88.0/23,203.34.70.0/23,203.34.71.0/24,203.86.252.0/22,203.148.80.0/22,203.149.92.0/22,203.169.0.0/22,203.189.112.0/22,203.191.64.0/18,204.19.38.0/23,204.44.32.0/20,204.44.192.0/20,204.44.224.0/20,204.48.16.0/20,204.52.255.0/24,204.57.16.0/20,204.75.147.0/24,204.75.228.0/24,204.80.198.0/24,204.86.16.0/20,204.87.199.0/24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having to deal with such a fixture, simply use string.rep(bytes) to build a fixture body.

for i=1, 1000 do
table.insert(big_value, "204.48.16.0")
end
local big_value_str = table.concat(big_value, ",")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thibaultcha I opted for this way instead to build a JSON value in memory

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would have been cleaner to use string.rep until a certain body size is needed. That is what the Galileo buffer tests are doing for example.

-- 
Thibault Charbonnier
Sent with Airmail

On November 9, 2015 at 2:43:08 PM, Marco Palladino (notifications@github.com) wrote:

In spec/integration/admin_api/kong_routes_spec.lua:

@@ -83,4 +84,42 @@ describe("Admin API", function()
assert.truthy(body.server.total_requests)
end)
end)
+

  • describe("Request size", function()
  • it("should properly hanlde big POST bodies < 10MB", function()
  •  local response, status = http_client.post(spec_helper.API_URL.."/apis", { request_path = "hello.com", upstream_url = "http://mockbin.org" })
    
  •  assert.equal(201, status)
    
  •  local api_id = json.decode(response).id
    
  •  assert.truthy(api_id)
    
  •  local big_value = {}
    
  •  for i=1, 1000 do
    
  •    table.insert(big_value, "204.48.16.0")
    
  •  end
    
  •  local big_value_str = table.concat(big_value, ",")
    
    @thibaultcha I opted for this way instead to build a JSON value in memory


Reply to this email directly or view it on GitHub.

thibaultcha added a commit that referenced this pull request Nov 10, 2015
Properly handling big request bodies in Admin API
@thibaultcha thibaultcha merged commit 330e6e2 into next Nov 10, 2015
@thibaultcha thibaultcha deleted the fix/admin-request-body branch November 10, 2015 20:27
@@ -1,6 +1,7 @@
local json = require "cjson"
local http_client = require "kong.tools.http_client"
local spec_helper = require "spec.spec_helpers"
local IO = require "kong.tools.io"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope - will remove it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

damn, spotted it yesterday and forgot about it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 points for Gryffindor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants