New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable jetty threadlimit mod if needed #491

Closed
yurem opened this Issue Oct 24, 2018 · 6 comments

Comments

Projects
None yet
3 participants
@yurem
Contributor

yurem commented Oct 24, 2018

There is jetty module threadlimit which might help to prevent DDOS attacks. It allows to limit thread count per remote IP. It works together with proxy jetty module which we enable by default.

Can you check if this module put in queue requests from remote IP or it just return HTTP error when number of threads is exceeded (we can try to open /identity which has many resources). If it put requests in queue we may need to enable it by default with optimal settings.

Here is test scenario:

service oxauth stop
/opt/jre/bin/java -jar /opt/jetty/start.jar jetty.home=/opt/jetty jetty.base=/opt/gluu/jetty/oxauth --add-to-start=threadlimit
chown -R jetty:jetty /opt/gluu/jetty/oxauth
# Modify start.ini
service oxauth start

@yurem yurem added the enhancement label Oct 24, 2018

@yurem yurem added this to the 3.1.5 milestone Oct 24, 2018

@mbaser

This comment has been minimized.

Collaborator

mbaser commented Nov 9, 2018

Here is my test program:

https://github.com/mbaser/gluu/blob/master/threadLimit_test.py

start.ini settings:

--module=threadlimit
jetty.threadlimit.forwardedHeader=Forwarded
jetty.threadlimit.enabled=true
jetty.threadlimit.threadLimit=2

When I run test program, I saw many 403 pages:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /oxauth/restv1/authorize
on this server.<br />
</p>
<hr>
<address>Apache/2.4.18 (Ubuntu) Server at c2.gluu.org Port 443</address>
</body></html>

When I increase number of threads to 10 as:
jetty.threadlimit.threadLimit=2

I never see 403 pages. It means it is working well.

@yurem

This comment has been minimized.

Contributor

yurem commented Nov 9, 2018

Can you run 2-3 test from different IP to make sure that there is no 403 with jetty.threadlimit.threadLimit=10. If there are no errors we need to enable this jetty module in CE with value 10 by default

@mbaser

This comment has been minimized.

Collaborator

mbaser commented Nov 9, 2018

What I found on my previous test is unreliable. 403 pages are generated by apache, most probably by mod_evasive

Since access to jetty is not directly from IP address (proxied by apache), threadLimit seems won't work.

@mbaser

This comment has been minimized.

Collaborator

mbaser commented Nov 9, 2018

Default settings of mod_evasive is 50 pages per second. So if I increase to 150 I never get 403 pages.

@mbaser

This comment has been minimized.

Collaborator

mbaser commented Nov 9, 2018

I sligtthy modifed test program as:

#!/usr/bin/python

import thread
import time
import requests

requests.packages.urllib3.disable_warnings()

my_url = 'https://c2.gluu.org/oxauth/restv1/authorize?response_type=code&client_id=@!F7CB.929B.4A73.EA6E!0001!EA93.FCCF!0008!CCA5.4735.7AFF.7646&redirect_uri=https://c3.gluu.org:8080/login_callback/'

counters = {'c':0,'e':0, 't':0}

def get_login_page(tn):
    global counters
    for i in range(250):
        counters['c'] += 1
        result = requests.get(my_url, verify=False)
        html = result.text
        if result.status_code == 403:
            counters['e'] += 1
            print counters['e'],'/', counters['c'] , result.status_code

        time.sleep(0.02)

    print "Thread {0} finished".format(tn)
    counters['t'] += 1

thread.start_new_thread(get_login_page, (1,))
thread.start_new_thread(get_login_page, (2,))
thread.start_new_thread(get_login_page, (3,))
thread.start_new_thread(get_login_page, (4,))

while 1:
   if counters['t'] == 4:
       break
@yurem

This comment has been minimized.

Contributor

yurem commented Nov 12, 2018

Closing it because behind proxy this module not works well.

@yurem yurem closed this Nov 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment