Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EBSI digital identity DDoS hardening using IP reputation #7481

Open
synctext opened this issue Jun 12, 2023 · 30 comments
Open

EBSI digital identity DDoS hardening using IP reputation #7481

synctext opened this issue Jun 12, 2023 · 30 comments
Assignees
@AdiDumi
Labels
type: MSc-Thesis-Work

Comments

@synctext
Copy link
Member

synctext commented Jun 12, 2023 •
edited

Survey Q1 + Starting thesis 10 jan {job next to thesis}

Also interested in the problem of online trust. General intro and overview by Bruce {rockstar of security research} Ten Risks of PKI: What You’re not Being Told about Public Key Infrastructure. Technical discussion on HackerNews on passport failures and public key directories.

{duplicate from # 7423}

Topic Title of paper and URL
strong identity A Truly Self-Sovereign Identity System
strong identity TrustVault: A privacy-first data wallet for the European Blockchain Services Infrastructure
strong identity Zero-Trust Architecture for Legal Entities
strong identity Distributed Attestation Revocation in Self-Sovereign Identity

More on passport-level digital identity. Warning given 😸 "this is complex stuff". identity is the foundation of trust in the old analog world. Please make a reading list about SSI, EBSI, EBP and eIDAS2

please note the survey @ Tribler lab methodology: https://github.com/Tribler/tribler/wiki/MasterThesis#10-ects-literature-survey-at-tribler-lab (e.g. try out all known open source wallets for suvey and grade them with stong/weak analysis in a table with screenshots { https://walt.id/ebsi ; https://github.com/walt-id })
@AdiDumi
Copy link

AdiDumi commented Jun 12, 2023 •
edited

why has public key infrastructure failed for 35 years

@synctext
Copy link
Member Author

synctext commented Aug 28, 2023 •
edited

Ask Google Scholar about PKI failure. First result:
https://www.nics.uma.es/pub/papers/JavierLopez2005.pdf
With the upcoming European passport this given a nice modern twist to 35(?) years of failure: Failure of Public Key Infrastructure for 35 years: lessons for the European passport-grade digital identity (EBSI) {putting 2 storylines into 1 title}. Shorter attempt: 35 years of failure: European Digital Identity lessons

see 2016 ideas on Self-sovereign identity (SSI). TUDelft Master student full-time on EU digital ID from 2021. Discussed the Literature Survey wiki documentation

{brainstorm in bit harsh term} A fraud-resilient authentication method is notoriously hard to realise. Academic thinkers have offered little help, academic literature extensively documents numerous ideas and design sketches. Public key cryptography was invented in the year: ... In the 35 years since this invention we have failed to utilise this invention. The European Digital Identity project (EBSI) is only the latest in a long line of failed attempts. For instance, "Overview of the German identity card project and lessons learned (2020 update)" and "The giant is lagging behind - How the German electronic ID fails to reap its potential.
This survey lists the numerous projects to provide digital identity and also highlights the decades scientific research. To date, no solution exists for the 400 million citizens within the EU. Finally, we list the vital lessons for the upcoming project to provide an EU-wide passport-grade digital identity. Contrary to numerous past projects, a refreshing amount of transparency is provided. For instance, see the detailed public EBSI node operator operational handbook.

Bit of general info news article "Digital Identity: Where We Began, Where We Are And Where We Are Going"

{early brainstorm for master thesis} The above builds expertise on wide scope of upcoming EU digital ID. Security will we essential, but EBSI server is based upon IBM Hyperledger technology. This is expected to only serve a good purpose during development. For full-scale production usage as the underpinning of the entire EU digital economy Hyperledger server probably needs replacing. Ideal outcome would be re-using the infrastructure at Delft to develop a 12-line Kotlin script to bring down such a EBSI Hyperledger server. See 2021 Delft master student who made the first open source Android EBSI v2 communications lib. Good news, save this project with application-level firewall rules????

@AdiDumi
Copy link

AdiDumi commented Aug 28, 2023 •
edited

On the risk of misbehaving RPKI authorities

REGULATION (EU) on eid
'Building trust in the online environment is key to economic and social development. Lack of trust, in particular because of a perceived lack of legal certainty, makes consumers, businesses and public authorities hesitate to carry out transactions electronically and to adopt new services.'

@AdiDumi
Copy link

AdiDumi commented Sep 20, 2023

Survey Template

@synctext
Copy link
Member Author

synctext commented Sep 20, 2023 •
edited

Lots of written documents in the past 35 years, yet no generic solution for digital identity:

@AdiDumi
Copy link

AdiDumi commented Oct 10, 2023 •
edited

Additional read:

@synctext
Copy link
Member Author

synctext commented Oct 11, 2023 •
edited

@AdiDumi
Copy link

AdiDumi commented Oct 31, 2023 •
edited

overleaf
Secure Web Client Using SPKI/SDSI
SDSI Java imple
Simple Public Key Infrastructure Analysis Protocol Analysis and Design
Attribute-Based Identity Management
Bridging the Cryptographic Design of ABCs with the Real World
IRMA
Glass

To get more from:
Blockchain-based identity management systems: A review

@synctext
Copy link
Member Author

synctext commented Nov 1, 2023 •
edited

@AdiDumi
Copy link

AdiDumi commented Nov 22, 2023 •
edited

Almost final draft
Missing abstract, conclusion and some table information
Literature_Survey_IN4306.pdf

@synctext
Copy link
Member Author

synctext commented Nov 22, 2023 •
edited

image

  • J.H. Ellis, Communications - Electronics Security Group, Government Communications Headquarters, Research Report No. 3006, The Possibility of Secure Non-Secret Digital Encryption, January 1970. Secret.. the possibility of non-secret crypto is from Jan 1970, 53 years ago

  • Just stated that it might be possible, no algorithm for one-way function, no implementation. That was RSA.

  • Missing milestone: Trust-on-first-use (TOFU, 2008) Perspectives: Improving SSH-style host authentication with multi-path probing. low-cost and simple key management model

  • small errors: "W. Diffie; M. Hellman" wrote the [27] citation. That's just key agreement, not the invention or realisation of public key crypto.

  • typos: \event{1996}{SDSI intorduction}

  • Dan Boneh, FEBRUARY 1999: twenty years of attacks on the rsa cryptosystem Essence: securely implementing RSA is a nontrivial task. We conclude: deploying public key cryptography such as RSA is also a non-trivial task!

  • "III. A HISTORY OF ALTERNATIVES", more like "evolution of PKI"?

  • "TABLE I: Overview of the ranking countries eID." the core taxonomy table! Facinating stuff, please at least double the number of countries+failures

  • EU on ID. "Every time an App or website asks us to create a new digital identity or to easily log on via a big platform, we have no idea what happens to our data in reality. That is why the Commission will propose a secure European e-identity. One that we trust and that any citizen can use anywhere in Europe to do anything from paying your taxes to renting a bicycle. A technology where we can control ourselves what data is used and how." Ursula von der Leyen, President of the European Commission, in her State of the Union address, 16 September 2020

  • {early brainstorm for master thesis - PART II} EBSI wallet, wallet-to-wallet communication (known IPv4), overlay (key lookup),data sharing (EBSI-to-EBSI), server hardening!
    (repeating)The above builds expertise on wide scope of upcoming EU digital ID. Security will we essential, but EBSI server is based upon IBM Hyperledger technology. This is expected to only serve a good purpose during development. For full-scale production usage as the underpinning of the entire EU digital economy Hyperledger server probably needs replacing. Ideal outcome would be re-using the infrastructure at Delft to develop a 12-line Kotlin script to bring down such a EBSI Hyperledger server. See 2021 Delft master student who made the first open source Digital Basic Identity - EU EBSI gateway and "regie op gegevens" #6023 (comment). Good news, save this project with application-level firewall rules????

    • time for coding again!
    • Data-vault code + wallet v2 interfacing with EBSI.
    • big milestone: talk to live EBSI server
    • Check out the code: https://ec.europa.eu/digital-building-blocks/code/projects/EBSI

@AdiDumi
Copy link

AdiDumi commented Nov 22, 2023

Ursula von der Leyen statement

@AdiDumi
Copy link

AdiDumi commented Dec 12, 2023 •
edited

Finished final version of the Survey.
Worked on the abstract, conclusions, history of PKI evolution, added countries to the survey and completed the taxonomy table. Rephrasing some paragraphs and added more literature about PKI failures and EU eIDs.
Literature_Survey_IN4306_final.pdf

@synctext
Copy link
Member Author

synctext commented Dec 13, 2023 •
edited

  • Solid work 🥇
  • Not explicit what PKI offers
  • Difficult to read academic writing style with section opening sentences covering 6 lines sometimes: The core element of the Public Key Infrastructures, key exchange with the RSA cryptosystem, has been the subject of different attacks from its introduction and securely implementing RSA is a nontrivial task, concluding that deploying public key cryptography such as RSA is also a non-trivial task
  • Would love to hear even a small success story, I get depressed from these 53 years of failings; However, only a few have succeeded, and there are several reasons behind the failure of PKIs, which can be categorized into technical, economical, legal, and social factors [37].. See wise people of Reddit: is Keybase dead?. That was the best startup we had in PKI space.
  • cultural reason is missing, digital authentication infrastructure has overlap with the "identity wars". Storage of government attributes like gender 👯‍♂️ with cryptographic authentication are notorious, see the "gender X" UK petition for border-crossing identity documents.
  • Leading company today seems: https://idk.digidentity.eu/Overview/
  • Would be great work for an arXiv upload !!!
  • First steps of master thesis: study code our our existing EBSI wallet, study the server code.

@AdiDumi
Copy link

AdiDumi commented Jan 10, 2024

The survey is uploaded to arXiv with the lastest version.
Literature_Survey_IN4306 (2).pdf

@synctext
Copy link
Member Author

synctext commented Jan 10, 2024 •
edited

Completed! Today the master thesis work starts 🎉

@AdiDumi
Copy link

AdiDumi commented Jan 24, 2024 •
edited

Studied the code for existing EBSI wallet in the SuperApp
Studied the server code and also on the gitlab with the APIs and wallets available now
and wallets

@synctext
Copy link
Member Author

synctext commented Jan 24, 2024 •
edited

Concrete tasks to further explore your thesis focus.

  • Try to find an EBSI (pilot) server to talk to .
    • try for at least days to find one, backup plan: study, compile the EBSI data vault superapp
    • Kotlin tooling
  • goal: build command-line tooling for performance analysis
  • outcome: this server crashes with a workload of 3k `did_write` requests per second. Stable with 2.5k tps.
    • measure response time, jitter
    • bandwidth usage
  • passkeys, FIDO2, webAuth are heavily pushed by Big Tech.
    • only low-level TOFU for email and e-commerce
    • no bank KYC, passport-grade legal signature, or air-plane boarding
    • However, this is usable and will be HUGE. Deployed to all Apple fans
    • authenticator apps with: time-based one-time password (TOTP; specified in RFC 6238) and HMAC-based one-time password (HOTP; specified in RFC 4226)
    • EBSI will almost be forced to interact with this or be compatible somewhat ??? (or given the EU temperature: "EU passkeys Regulation Act", forcing them to change)
  • mental note: this master student working on PKI, athentication and digital identity does not know backup codes
  • Next sprint: 1-page "Problem Description" as MARE.tudelft.nl first stage evaluation doc + also first sample master thesis chapter.
  • Either Rowdy or Egbert can help with EBSI daily details.

@synctext
Copy link
Member Author

Please read all these details:
https://medium.com/@schwalm.steffen/qualified-ledgers-bridging-the-gap-between-blockchain-technology-and-legal-compliance-c08d24a68db9

@synctext synctext changed the title msc placeholder: solid coding specialist msc placeholder: PKI, EBSI, and SSI Feb 7, 2024
@synctext
Copy link
Member Author

synctext commented Feb 7, 2024 •
edited

Next week we should get an EBSI server image to install, so there is movement...

@AdiDumi
Copy link

AdiDumi commented Feb 19, 2024 •
edited

Update sprint:

  • key takes from Qualified Ledgers: Bridging the Gap between Blockchain Technology and Legal Compliance:
    - eIDAS 2.0 shaping the regulatory framework for digital identity and trust services within the European Union with an accent on Decentralized digital identities and ecosystems and Self Sovereign identities
    - qualified electronic ledgers aligns with the principles of eIDAS 2.0
    - (EBSI) and its transition into the European Digital Infrastructure Consortium (EDIC) for decentralized infrastructure for public services across Europe
    - EBSI could evolve into a qualified electronic ledger, enhancing security and reliability for various trust services and with DLT intersect with the requirements of eIDAS 2.0, such as digital credentials, cross-border cooperation, and regulatory compliance.
    - German government's decision not to support DLT for their national EUDI wallet because provable security standards, addressed through certification requirements maybe?

  • Challenges in Stopping Application Layer DDoS Attacks:
    - distinguishing between attack traffic and normal traffic is challenging, especially in the case of botnets performing HTTP Flood attacks.
    - each bot in a botnet makes seemingly legitimate network requests, making the traffic appear "normal" in origin.

  • Solution?
    - tools like a properly configured Web Application Firewall (WAF), are necessary for mitigation(ModSecurity).
    - filter traffic through an IP reputation database. LIKE THIS

  • Could not talk to a EBSI pilot server - need a "Verifiable Authorisation To Onboard"

  • Found and compiled the DataVault app from old master thesis. Spend days fixing compatibility and version errors. Updated libraries to match the newer kotlin version => working on my phone(except the performance test).

@synctext
Copy link
Member Author

synctext commented Feb 19, 2024 •
edited

  • 🎉 🎉 🎉
    • Awesome core thesis idea
    • filter traffic through an IP reputation database.
    • draft thesis title: "IP reputation protection for passport-grade EU SSI ID server" (todo: less abbreviation)
  • Related work
  • fundamental issue is that reputation is tied to your passport-grade ID versus IPv4 address
    • not really totally ambitious, just fundamentally fix The Internet 🙄
    • ideas:
      • for every successful login, leave IPv4 in table (Goodness table)
      • for all irrefutable attack patterns, leave IPv4 in table (Bad table)
      • Are you a bot? Solve this Captcha or call your local EU municipality
      • You are a bot, you now go straight to bot jail
      • For each /24, /16, and /8 keep a block reputation
        • Multiple public keys of logins (or hashed IDs)
        • Multiple irrefutable attack patterns
        • Mixed 😲 😮 😨
        • Keep this smaller then complete BGP routing table (AS count 74,200)
  • We completely safe, except for "server-side request forgery" attacks.
  • Upcoming sprint. Read related work:
  • Future: top-down analysis, fixes problem?

@synctext synctext mentioned this issue Feb 21, 2024
Open
@AdiDumi
Copy link

AdiDumi commented Mar 7, 2024 •
edited

Sprint update:

  • not as much progress as hoped due to house hunting and stomach flu
  • studied more the ModSecurity with Core Rule Set:
    - combination of anomaly scoring = each rule that matches a part of the request contributes to a cumulative score. The more rules a request triggers, the higher its overall score becomes until a threshold set by user
    - and paranoia level the sensitivity or aggressiveness of the rule set, depending on the type of data accessed, problems with false-positives
  • working on docker files to have containers running the ModSecurity + SlowHttpTest with docker compose:
    - SlowHttpTest
    - ModSecurity
  • Changed dockerfiles from repos, work in progress to make them build, had to add versions, delete packages..
  • Paper shows that ModSecurity is vulnerable especially to DoS attacks(multiple types: slow header, slow body, range attacks)
  • ModSecurity has been passed even from 2014, presented in Analysis of Slow Read DoS Attack and Countermeasures
Full draft docker magic 

ARG HTTPD_VERSION="1"

FROM httpd:${HTTPD_VERSION} as build

ARG MODSEC2_VERSION="1"
ARG LUA_VERSION="1"

RUN set -eux; \
    echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections; \
    apt-get update -qq; \
    apt-get install -y -qq --no-install-recommends --no-install-suggests \
        automake \
        ca-certificates \
        g++ \
        git \
        libapr1-dev \
        libaprutil1-dev \
        libcurl4-gnutls-dev \
        libfuzzy-dev \
        libpcre3-dev \
        libtool \
        libxml2-dev \
        libyajl-dev \
        lua${LUA_VERSION}-dev \
        make \
        pkgconf \
        wget

RUN set -eux; \
    wget --quiet https://github.com/owasp-modsecurity/ModSecurity/archive/refs/tags/v${MODSEC2_VERSION}.tar.gz; \
    tar -zxvf v${MODSEC2_VERSION}.tar.gz; \
    cd ModSecurity-${MODSEC2_VERSION}; \
    ./autogen.sh; \
    ./configure --with-yajl --with-ssdeep; \
    make; \
    make install; \
    make clean

FROM httpd:${HTTPD_VERSION} as crs_release

ARG CRS_RELEASE

# hadolint ignore=DL3008,SC2016
RUN set -eux; \
    apt-get update; \
    apt-get -y install --no-install-recommends \
      ca-certificates \
      curl \
      gnupg; \
    mkdir /opt/owasp-crs; \
    curl -SL https://github.com/coreruleset/coreruleset/archive/v${CRS_RELEASE}.tar.gz -o v${CRS_RELEASE}.tar.gz; \
    curl -SL https://github.com/coreruleset/coreruleset/releases/download/v${CRS_RELEASE}/coreruleset-${CRS_RELEASE}.tar.gz.asc -o coreruleset-${CRS_RELEASE}.tar.gz.asc; \
    gpg --fetch-key https://coreruleset.org/security.asc; \
    gpg --verify coreruleset-${CRS_RELEASE}.tar.gz.asc v${CRS_RELEASE}.tar.gz; \
    tar -zxf v${CRS_RELEASE}.tar.gz --strip-components=1 -C /opt/owasp-crs; \
    rm -f v${CRS_RELEASE}.tar.gz coreruleset-${CRS_RELEASE}.tar.gz.asc; \
    mv -v /opt/owasp-crs/crs-setup.conf.example /opt/owasp-crs/crs-setup.conf

FROM httpd:${HTTPD_VERSION}

ARG MODSEC2_VERSION
ARG LUA_VERSION
ARG LUA_MODULES

ENV APACHE_ALWAYS_TLS_REDIRECT=off \
    APACHE_LOGFORMAT='"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""' \
    APACHE_ERRORLOG_FORMAT='"[%{u}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% ,\ referer\ %{Referer}i"' \
    APACHE_METRICS_LOGFORMAT='"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""' \
    ACCESSLOG=/var/log/apache2/access.log \
    BACKEND=http://localhost:80 \
    BACKEND_WS=ws://localhost:8080 \
    ERRORLOG='/proc/self/fd/2' \
    H2_PROTOCOLS='h2 http/1.1' \
    LOGLEVEL=warn \
    METRICS_ALLOW_FROM='127.0.0.0/255.0.0.0 ::1/128' \
    METRICS_DENY_FROM='All' \
    MUTEX='default' \
    METRICSLOG='/dev/null' \
    MODSEC_AUDIT_ENGINE="RelevantOnly" \
    MODSEC_AUDIT_LOG_FORMAT=JSON \
    MODSEC_AUDIT_LOG_TYPE=Serial \
    MODSEC_AUDIT_LOG=/dev/stdout \
    MODSEC_AUDIT_LOG_PARTS='ABIJDEFHZ' \
    MODSEC_AUDIT_STORAGE=/var/log/modsecurity/audit/ \
    MODSEC_DATA_DIR=/tmp/modsecurity/data \
    MODSEC_DEBUG_LOG=/dev/null \
    MODSEC_DEBUG_LOGLEVEL=0 \
    MODSEC_DEFAULT_PHASE1_ACTION="phase:1,pass,log,tag:'\${MODSEC_TAG}'" \
    MODSEC_DEFAULT_PHASE2_ACTION="phase:2,pass,log,tag:'\${MODSEC_TAG}'" \
    MODSEC_DISABLE_BACKEND_COMPRESSION="On" \
    MODSEC_PCRE_MATCH_LIMIT_RECURSION=100000 \
    MODSEC_PCRE_MATCH_LIMIT=100000 \
    MODSEC_REQ_BODY_ACCESS=on \
    MODSEC_REQ_BODY_LIMIT=13107200 \
    MODSEC_REQ_BODY_LIMIT_ACTION="Reject" \
    MODSEC_REQ_BODY_JSON_DEPTH_LIMIT=512 \
    MODSEC_REQ_BODY_NOFILES_LIMIT=131072 \
    MODSEC_RESP_BODY_ACCESS=on \
    MODSEC_RESP_BODY_LIMIT=1048576 \
    MODSEC_RESP_BODY_LIMIT_ACTION="ProcessPartial" \
    MODSEC_RESP_BODY_MIMETYPE="text/plain text/html text/xml" \
    MODSEC_RULE_ENGINE=on \
    MODSEC_SERVER_SIGNATURE="Apache" \
    MODSEC_STATUS_ENGINE="Off" \
    MODSEC_TAG=modsecurity \
    MODSEC_TMP_DIR=/tmp/modsecurity/tmp \
    MODSEC_TMP_SAVE_UPLOADED_FILES="on" \
    MODSEC_UPLOAD_DIR=/tmp/modsecurity/upload \
    PORT=80 \
    PROXY_ERROR_OVERRIDE=on \
    PROXY_PRESERVE_HOST=on \
    PROXY_SSL=off \
    PROXY_SSL_CA_CERT=/etc/ssl/certs/ca-certificates.crt \
    PROXY_SSL_CERT=/usr/local/apache2/conf/proxy.crt \
    PROXY_SSL_CERT_KEY=/usr/local/apache2/conf/proxy.key \
    PROXY_SSL_CHECK_PEER_NAME=off \
    PROXY_SSL_CIPHERS="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
    PROXY_SSL_PROTOCOLS="all -SSLv3 -TLSv1 -TLSv1.1" \
    PROXY_SSL_VERIFY=none \
    PROXY_TIMEOUT=60 \
    REMOTEIP_INT_PROXY='10.1.0.0/16' \
    REQ_HEADER_FORWARDED_PROTO='https' \
    SERVER_ADMIN=root@localhost \
    SERVER_NAME=localhost \
    SERVER_SIGNATURE=Off \
    SERVER_TOKENS=Full \
    SSL_CERT=/usr/local/apache2/conf/server.crt \
    SSL_CERT_KEY=/usr/local/apache2/conf/server.key \
    SSL_CIPHERS="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
    SSL_ENGINE=on \
    SSL_HONOR_CIPHER_ORDER=off \
    SSL_PORT=443 \
    SSL_PROTOCOLS="all -SSLv3 -TLSv1 -TLSv1.1" \
    SSL_SESSION_TICKETS=off \
    SSL_OCSP_STAPLING=On \
    TIMEOUT=60 \
    WORKER_CONNECTIONS=400 \
    # CRS specific variables
    PARANOIA=1 \
    ANOMALY_INBOUND=5 \
    ANOMALY_OUTBOUND=4 \
    BLOCKING_PARANOIA=1

COPY --from=build /usr/local/apache2/modules/mod_security2.so                  /usr/local/apache2/modules/mod_security2.so
COPY --from=build /usr/local/apache2/ModSecurity-${MODSEC2_VERSION}/modsecurity.conf-recommended  /etc/modsecurity.d/modsecurity.conf
COPY --from=build /usr/local/apache2/ModSecurity-${MODSEC2_VERSION}/unicode.mapping               /etc/modsecurity.d/unicode.mapping
COPY --from=crs_release /opt/owasp-crs /opt/owasp-crs
COPY src/etc/modsecurity.d/*.conf /etc/modsecurity.d/
COPY src/bin/* /usr/local/bin/
COPY apache/conf/extra/*.conf /usr/local/apache2/conf/extra/
COPY src/etc/modsecurity.d/*.conf /etc/modsecurity.d/
COPY src/opt/modsecurity/activate-*.sh /opt/modsecurity/
COPY apache/docker-entrypoint.sh /

RUN set -eux; \
    echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections; \
    apt-get update -qq; \
    apt-get install -qq -y --no-install-recommends --no-install-suggests \
        ca-certificates \
        curl \
        gnupg \
        iproute2 \
        libcurl3-gnutls \
        libfuzzy2 \
        liblua${LUA_VERSION} \
        ${LUA_MODULES} \
        libxml2 \
        libyajl2; \
    update-ca-certificates -f; \
    apt-get clean; \
    rm -rf /var/lib/apt/lists/*

RUN set -eux; \
    mkdir -p /etc/modsecurity.d/; \
    mkdir -p /tmp/modsecurity/data; \
    mkdir -p /tmp/modsecurity/upload; \
    mkdir -p /tmp/modsecurity/tmp; \
    chown -R $(awk '/^User/ { print $2;}' /usr/local/apache2/conf/httpd.conf) /tmp/modsecurity; \
    mkdir -p /var/log/apache2/; \
    ln -s /opt/owasp-crs /etc/modsecurity.d/; \
    sed -i -E 's|(Listen) [0-9]+|\1 ${PORT}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(ServerTokens) Full|\1 ${SERVER_TOKENS}|' /usr/local/apache2/conf/extra/httpd-default.conf; \
    sed -i -E 's|(ServerSignature) Off|\1 ${SERVER_SIGNATURE}|' /usr/local/apache2/conf/extra/httpd-default.conf; \
    sed -i -E 's|#(ServerName) www.example.com:80|\1 ${SERVER_NAME}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(ServerAdmin) you@example.com|\1 ${SERVER_ADMIN}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|^(\s*CustomLog)(\s+\S+)+|\1 ${ACCESSLOG} modsec "env=!nologging"|g' /usr/local/apache2/conf/httpd.conf; \
	sed -i -E 's|^(\s*ErrorLog)\s+\S+|\1 ${ERRORLOG}|g' /usr/local/apache2/conf/httpd.conf; \
	sed -i -E 's|^(\s*TransferLog)\s+\S+|\1 ${ACCESSLOG}|g' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule unique_id_module)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule rewrite_module modules/mod_rewrite.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule proxy_module modules/mod_proxy.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule proxy_http_module modules/mod_proxy_http.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule remoteip_module modules/mod_remoteip.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule socache_shmcb_module modules/mod_socache_shmcb.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule ssl_module modules/mod_ssl.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule http2_module modules/mod_http2.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-default.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-proxy.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-ssl.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-vhosts.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    echo 'Include conf/extra/httpd-locations.conf' >> /usr/local/apache2/conf/httpd.conf; \
    echo 'Include conf/extra/httpd-modsecurity.conf' >> /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(MaxRequestWorkers[ ]*)[0-9]*|\1${WORKER_CONNECTIONS}|' /usr/local/apache2/conf/extra/httpd-mpm.conf; \
    chgrp -R 0 /var/log/ /usr/local/apache2/; \
    chmod -R g=u /var/log/ /usr/local/apache2/

ENTRYPOINT ["/docker-entrypoint.sh"]

FROM alpine:3.17 as builder

RUN apk add --no-cache build-base git openssl-dev autoconf automake
WORKDIR /build
COPY . /build
RUN ./configure && make


FROM alpine:3.17
RUN apk add --no-cache libstdc++
COPY --from=builder /build/src/slowhttptest /usr/local/bin/
ENTRYPOINT ["slowhttptest"]

services:
  slowhttp:
    build:
      context: ./slowhttp

  modsecurity:
    build:
      context: ./modsecurity

@synctext synctext changed the title msc placeholder: PKI, EBSI, and SSI EBSI digital identity DDoS hardening using IP reputation Mar 8, 2024
@synctext
Copy link
Member Author

synctext commented Mar 8, 2024 •
edited

@AdiDumi
Copy link

AdiDumi commented Mar 26, 2024 •
edited

Update sprint:

  • ModSecurity can be deployed as a reverse proxy for the server so the traffic is not encrypted

image

  • by default, ModSecurity operates at the application layer and is not designed to directly manage login sessions or track successful login attempts. But, it can be configured to work in conjunction with other security mechanisms, such as rate limiting or custom rules, to detect and mitigate potential threats, including brute-force login attacks:
    - Rate Limiting: implement rate limiting rules that throttle the number of login attempts from specific IP netranges. This can help mitigate brute-force attacks by limiting the number of requests allowed within a certain time frame.
    - Custom Rules: custom ModSecurity rules to detect and block login attempts from specific IP netranges. These rules could analyze patterns in the request headers or parameters to identify login attempts and then take appropriate action based on the originating IP address.
  • slowhttptest container build and run ✔️
  • modsecurity container build and run ✔️
  • still problems in connecting the containers but docker-compose and Dockerfile clean up, short and simple commands
  • update - connection established between containers ✔️
  • working in repo

image

@synctext
Copy link
Member Author

synctext commented Mar 27, 2024 •
edited

@AdiDumi
Copy link

AdiDumi commented Apr 14, 2024 •
edited

Short sprint update:

Problem_Description_thesis (1).pdf

@synctext
Copy link
Member Author

synctext commented Apr 15, 2024 •
edited

  • as usual, solid progress 🚀
    • start to look like a thesis
    • Only 5-6 experimental result figures, more text, DONE!
  • Good related work section
  • Fundamental security problem with Internet architecture (e.g. Problem Description)
  • Sprint proposal:
    • given IP reputation focus, IETF Dots, Trustguard, and Modsecurity
    • come up with 3 thesis direction for implementation and performance analysis 😄 Easy sprint
    • plus needs to evolve in a generic Fix-The-Internet 👍
    • {idea III: login-once, light-weight token check for every API call, large-scale zero-trust tracking {epic table of reputations with O(1) in 1 TByte}} { ⚡ fails to use coordination by network intermediaries}
    • Zero trust is based on the principle of “never trust, always verify.”

@AdiDumi
Copy link

AdiDumi commented Apr 30, 2024 •
edited

Brainstorming sprint:

  • ideas for directions:
    1. Design smart contracts to manage reputation scores and updates on the blockchain. Each reputation update (e.g., positive or negative feedback) is recorded as a transaction on the blockchain, ensuring transparency and immutability. Extend ModSecurity to interact with the blockchain for reputation validation. When processing incoming requests, ModSecurity queries the blockchain to verify the reputation score of the requesting IP address and incorporate a "TrustGuard's like" reputation management techniques to dynamically adjust reputation scores based on network interactions.
    2. Deploy the IETF DOTS protocol with blockchain-based IP reputation management for DDoS mitigation in distributed server environments, real-time threat signaling and coordination during attacks. Manage IP reputation data securely across distributed DOTS agents and servers. Configure DOTS agents to interact with the blockchain for reputation validation and retrieval during DDoS attack mitigation
    3. Zero-Trust Reputation Management for API security, login-once mechanism for API -> authentication tokens -> lightweight token checks into API endpoints to validate the authenticity and permissions of incoming requests -> IP reputation check
    4. Simple reputation scores and updates on the blockchain for every IP or range/16. Define a harsh reputation computation if heavy traffic comes from IP, users usually have something that DDos attackers do not have -> time to get back the reputation -> time for the server to respond to an attack. The important part: to define a good calculation of reputation based on factors like nr request/second + history over time + successful accesses + bad requests etc.
  • performance analysis:
    - blockchain transaction throughput
    - overall attack detection rate
    - mitigation success rate
    - system overhead during attack
    - attack detection accuracy
    - response time for blocking malicious traffic

@synctext
Copy link
Member Author

synctext commented May 1, 2024 •
edited

😲 😲 😲 You have a viable roadmap to fix The Internet.

P2P Modsecurity is a brilliant idea. With a bit of tweaking it is also incrementally expandable. However, no smart contracts, gas payments, and Turing incompleteness please.

Re-architecting The Internet using zero-trust principle, Modsecurity, and PUFs

We create a layer of trust The Internet always needed, but never had. In 1962 the architecture of The Internet for the thermonuclear era was written down in report P2626. The highly survivable system structure is fundamentally unsuitable for today's world. For instance, Internet address 180.101.88.232 owned by ISP ChinaNet Jiangsu Province has been launching SSH login attacks for multiple years.
First we create a universal trust token. It consists of a non-revocable self-sovereign identity with list of trust attestations. Second, we instrument Modsecurity with trust scoring, real-time threat signalling, coordination with others, and automatic formation of a global web-of-trust. Third, we present a trust model which is grounded in the laws of physics and mathematical axioms. By combing zero-trust principle and physical unclonable functions we create strong identity and web-of-trust framework which can serve as a mid-life upgrade of The Internet.

Roadmap till 31 Oct graduation

  • priority is realisation of zero-trust architecture. All performance or specific features are secondary. Proof-of-principle running code.
  • Modsecurity IPv4 scoring for just two trivial attack types (SSH login FAIL, SlowHHTPtest).
    • exchange of universal mistrust tokens about attacks
    • both IPv4 mistrust or self-sovereign identity abuse.
    • web-of-trust creation
    • re-usage of either our mature codebase of Python-based IPv8, trustchain, and EVA. Or compiled Kotlin, trustchain, and uTP.
  • Modsecurity understanding of the universal trust token
    • add trust attestations to universal trust token
    • trust anchors can sign the self-sovereign ID.
  • PUF part: only in text and pictures of prior work, no coding
  • classical thesis layout: problem, design, implementation and performance analysis
    • first experiment: server hammering. modsecurity login fails. show creation of mistrust records. Repeated offenders get blocked faster using web-of-trust.
    • second experiment: SlowHHTPtest
    • third experiment: SSI wallet collecting trust signatures
    • final experiment: Attack of the stolen wallets. numerous wallet abusive repeated logins, real-time threat signalling, web-of-trust prevents repeated attacks on other Modsecurity instances

update: strong related work of a -simulation- of attack info sharing https://github.com/LukasForst/fides + proper code: https://github.com/stratosphereips/StratosphereLinuxIPS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AdiDumi AdiDumi

Labels
type: MSc-Thesis-Work
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@synctext @AdiDumi