Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: pypi/inspector
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: rubygems/inspector
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 9 commits
  • 32 files changed
  • 1 contributor

Commits on Feb 25, 2025

  1. Rewrite for rubygems.org support

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 25, 2025
    Copy the full SHA
    7f17673 View commit details

Commits on Feb 26, 2025

  1. Remove deob

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 26, 2025
    Copy the full SHA
    cdb063e View commit details
  2. Add ddtrace

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 26, 2025
    Copy the full SHA
    33bec20 View commit details

Commits on Feb 27, 2025

  1. Shipit & k8s config

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 27, 2025
    Copy the full SHA
    4d29e23 View commit details
  2. Use ruff for formatting

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 27, 2025
    Copy the full SHA
    1d34141 View commit details

Commits on Feb 28, 2025

  1. Only build for amd64

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Feb 28, 2025
    Copy the full SHA
    74d5a9d View commit details

Commits on Mar 5, 2025

  1. Merge pull request #2 from rubygems/segiddins/rubygems

    Configure for inspector.rubygems.info
    segiddins authored Mar 5, 2025
    Copy the full SHA
    1b73b09 View commit details
  2. Fix push criteria

    Signed-off-by: Samuel Giddins <segiddins@segiddins.me>
    segiddins committed Mar 5, 2025
    Copy the full SHA
    29ca3d7 View commit details
  3. Copy the full SHA
    9f7e062 View commit details
38 changes: 38 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Docker
on:
pull_request:
push:
branches:
- main
permissions:
contents: read

jobs:
build:
name: Docker build (and optional push)
runs-on: ubuntu-24.04
env:
RUBYGEMS_VERSION: "3.6.4"
RUBY_VERSION: "3.4.2"
permissions:
id-token: write
steps:
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@f7ce87c1d6bead3e36075b2ce75da1f6cc28aaca # master
- name: Configure AWS credentials from Production account
uses: aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722 # v4.1.0
if: github.secret_source != 'None'
with:
role-to-assume: arn:aws:iam::048268392960:role/rubygems-ecr-pusher
aws-region: us-west-2
- name: Login to Amazon ECR
if: github.secret_source != 'None'
uses: aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076 # v2.0.1
- name: Build and push
uses: docker/build-push-action@v6
with:
platforms: linux/amd64
push: "${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'true' }}"
tags: |
048268392960.dkr.ecr.us-west-2.amazonaws.com/rubygems/inspector:${{ github.sha }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
.state/
__pycache__/
.venv
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"makefile.configureOnOpen": false
}
18 changes: 4 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -21,30 +21,20 @@ ARG DEVEL=no
RUN set -x \
&& apt-get update \
&& apt-get install -y \
git cmake g++ \
# $(if [ "$DEVEL" = "yes" ]; then echo 'bash postgresql-client'; fi) \
git cmake g++ \
# $(if [ "$DEVEL" = "yes" ]; then echo 'bash postgresql-client'; fi) \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Install pycdc and pycdas...
WORKDIR /tmp
RUN git clone "https://github.com/zrax/pycdc.git" && \
cd pycdc && \
cmake . && \
cmake --build . --config release && \
mv ./pycdc /usr/local/bin && \
mv ./pycdas /usr/local/bin && \
cd .. && rm -rf ./pycdc

# Copy local code to the container image.
WORKDIR /app
# Copy in requirements files
COPY ./requirements ./requirements

# Install production dependencies.
RUN pip install \
-r requirements/main.txt \
-r requirements/deploy.txt
-r requirements/main.txt \
-r requirements/deploy.txt

# Install development dependencies
RUN if [ "$DEVEL" = "yes" ]; then pip install -r requirements/lint.txt; fi
3 changes: 1 addition & 2 deletions bin/lint
Original file line number Diff line number Diff line change
@@ -5,7 +5,6 @@ set -e
set -x

# Actually run our tests.
python -m flake8 --max-line-length 88 .
python -m black --check inspector/ tests/
python -m ruff check .
python -m isort --check inspector/ tests/
#python -m curlylint ./inspector/templates
23 changes: 23 additions & 0 deletions config/deploy/ingress.yaml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: <%= environment %>
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-2-2017-01
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/target-group-attributes: load_balancing.algorithm.type=least_outstanding_requests
alb.ingress.kubernetes.io/tags: Env=<%= environment %>,Service=inspector.rubygems.info
alb.ingress.kubernetes.io/healthcheck-path: /_health/
alb.ingress.kubernetes.io/load-balancer-name: <%= environment %>-inspector-rubygems-info
spec:
tls:
- hosts:
- <%= environment %>-origin.inspector.rubygems.info
defaultBackend:
service:
name: web
port:
number: 80
140 changes: 140 additions & 0 deletions config/deploy/nginx-configmap.yaml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
default: |
user nginx;
worker_processes 3;
events {
worker_connections 5120;
}
http {
# Ensure buffer is large enough to contend with a redirect to a URL that is at the request line length limit
proxy_buffers 4 16k;
proxy_buffer_size 16k;

server {
listen 80;
server_name _;
location /nginx_health {
access_log off;
return 200 "healthy\n";
}
}

map $http_x_edge_proto $scheme_from_fastly {
"" $scheme;
default $http_x_edge_proto;
}

limit_req_status 429;
limit_req_zone $binary_remote_addr zone=ui:10m rate=300r/m;
limit_req_zone $binary_remote_addr zone=abusers:10m rate=30r/s;
limit_req_zone $binary_remote_addr zone=dependencyapi:10m rate=100r/s;
limit_req_zone $binary_remote_addr zone=minutely:10m rate=100r/m;

proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=cache-versions:10m inactive=24h max_size=100m;

include /etc/nginx/conf.d/logging.conf;
include /etc/nginx/sites-enabled/*;
}
rubygems: |
server {
listen 8080;
server_name _;

real_ip_recursive on;

# Fastly (https://api.fastly.com/public-ip-list)
set_real_ip_from '23.235.32.0/20';
set_real_ip_from '43.249.72.0/22';
set_real_ip_from '103.244.50.0/24';
set_real_ip_from '103.245.222.0/23';
set_real_ip_from '103.245.224.0/24';
set_real_ip_from '104.156.80.0/20';
set_real_ip_from '140.248.64.0/18';
set_real_ip_from '140.248.128.0/17';
set_real_ip_from '146.75.0.0/17';
set_real_ip_from '151.101.0.0/16';
set_real_ip_from '157.52.64.0/18';
set_real_ip_from '167.82.0.0/17';
set_real_ip_from '167.82.128.0/20';
set_real_ip_from '167.82.160.0/20';
set_real_ip_from '167.82.224.0/20';
set_real_ip_from '172.111.64.0/18';
set_real_ip_from '185.31.16.0/22';
set_real_ip_from '199.27.72.0/21';
set_real_ip_from '199.232.0.0/16';

set_real_ip_from '2a04:4e40::/32';
set_real_ip_from '2a04:4e42::/32';

# AWS
set_real_ip_from '172.30.0.0/16';

real_ip_header X-Forwarded-For;

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme_from_fastly;
proxy_set_header Forwarded "";
proxy_set_header X-Forwarded-Host "";
proxy_set_header Client-IP "";
proxy_set_header Host $host;
proxy_redirect off;

client_max_body_size 500M;

# Stop unsupported methods from getting to Puma,
# respond with a 405.
# (Puma responds with a 501 to unknown methods, which leads to us getting paged
# for what are essentially bad requests, not server errors)
# Can't use limit_except here because it doesn't work in server context,
# only location context, and we want this to apply to all locations.
if ( $request_method !~ ^(GET|HEAD|POST|PUT|DELETE|OPTIONS|PATCH)$ ) {
return 405;
}


location / {
limit_req zone=ui burst=5;
proxy_pass http://127.0.0.1:3000;
}
}
logging: |
log_format json escape=json '{'
'"host": "${HOSTNAME}", '
'"@timestamp": "$time_iso8601", '
'"service": "inspector.rubygems.org", '
'"env": "<%= environment %>", '
'"processing_time_milliseconds": "$request_time", '
'"forwarded_for": "$http_x_forwarded_for", '
'"upstream_x_runtime": "$upstream_http_x_runtime", '
'"upstream_processing_time": "$upstream_response_time", '
'"upstream": "$upstream_addr", '
'"http": {'
'"status_code": $status, '
'"scheme": "$scheme_from_fastly", '
'"url": "$scheme_from_fastly://$http_host$request_uri", '
'"args": "$args", '
'"dest_host": "$http_host", '
'"content_type": "$sent_http_content_type", '
'"location": "$sent_http_location", '
'"protocol": "$server_protocol", '
'"referer": "$http_referer", '
'"useragent": "$http_user_agent", '
'"method": "$request_method", '
'"request": "$request", '
'"request_id": "$upstream_http_x_request_id", '
'"ssl_protocol": "$ssl_protocol", '
'"ssl_cipher": "$ssl_cipher"'
'},'
'"network": {'
'"client": {"ip": "$remote_addr"}, '
'"bytes_written": $bytes_sent, '
'"body_bytes_written": $body_bytes_sent'
'}'
'}';
access_log /dev/stdout json;
error_log /dev/stdout;
1 change: 1 addition & 0 deletions config/deploy/production/ingress.yaml.erb
1 change: 1 addition & 0 deletions config/deploy/production/nginx-configmap.yaml.erb
12 changes: 12 additions & 0 deletions config/deploy/production/secrets.ejson
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"_public_key": "a2b3b45e4db1aaf9ee6400d3d685de7e5d761a9c4b45ab50a38c5a5403b6161d",
"kubernetes_secrets": {
"production": {
"_type": "Opaque",
"data": {
"honeybadger_api_key": "EJ[1:PdZreInTWXI1FLrFAIe4j7eLOfbTVh6EWWwBL89TMAk=:PvoZeAX52WmcFsCSWiVhSF86SNd3j9SF:z5I4XA0St0psAfod3SCcwPMWYl2d330p]",
"datadog_csp_api_key": "EJ[1:MJLOVnheQZD8mqT8Q5xlN8u6Qd9eH4sbO1kcsg/TTR4=:0eAj1g7msiS86qexeMNsWU+1vy3pXSKK:4YT1qDTiBirwkNsbNdhN90lPlF51qiMNChue7uMpEYuAiKTI3TnwOWJd7sk0PmARVsIn]",
}
}
}
}
1 change: 1 addition & 0 deletions config/deploy/production/service.yaml.erb
1 change: 1 addition & 0 deletions config/deploy/production/web.yaml.erb
16 changes: 16 additions & 0 deletions config/deploy/service.yaml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: v1
kind: Service
metadata:
name: web
labels:
name: web
annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
service.beta.kubernetes.io/aws-load-balancer-name: <%= environment %>-inspector-rubygems-info
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 8080
selector:
name: web
1 change: 1 addition & 0 deletions config/deploy/staging/ingress.yaml.erb
1 change: 1 addition & 0 deletions config/deploy/staging/nginx-configmap.yaml.erb
12 changes: 12 additions & 0 deletions config/deploy/staging/secrets.ejson
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"_public_key": "65180e1f308f283eeb6b3e05cc13d3c0b0c5cb4a35638ea85bbd5b208cea6e2a",
"kubernetes_secrets": {
"staging": {
"_type": "Opaque",
"data": {
"honeybadger_api_key": "EJ[1:jnrbdGY2s+ZVCF8BStxF4ZGp1yhxk+x/sXxH8x32qmg=:TD1e42m2hCnraaRigNZZZao9WKF9S9Nf:Myj5m7maDkOC2GbolUskZ2F+zmRscHE8]",
"datadog_csp_api_key": "EJ[1:M1xidXJHz2i/vKthLOnwYbEnkFjYneq+d6Ryj6TXdFQ=:fxMZfI3MptOYF/hXcrnrWK/SDTU1SzZC:Gn807pB1wSTTNIyTJ5cpGCvN5+vplIZUxi/a8/s+UFDJE52zeM5KyhvmK6f0J8mDa2vh]"
}
}
}
}
1 change: 1 addition & 0 deletions config/deploy/staging/service.yaml.erb
1 change: 1 addition & 0 deletions config/deploy/staging/web.yaml.erb
153 changes: 153 additions & 0 deletions config/deploy/web.yaml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
annotations:
shipit.shopify.io/restart: 'true'
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
selector:
matchLabels:
name: web
template:
metadata:
annotations:
ad.datadoghq.com/gunicorn.logs: '[{"source":"rails","service":"inspector.rubygems.info","version": <%= current_sha.dump %>}]'
labels:
name: web
tags.datadoghq.com/env: "<%= environment %>"
tags.datadoghq.com/service: inspector.rubygems.info
tags.datadoghq.com/version: <%= current_sha %>
spec:
containers:
- name: gunicorn
image: 048268392960.dkr.ecr.us-west-2.amazonaws.com/rubygems/inspector:<%= current_sha %>
args: ["ddtrace-run", "gunicorn", "-c", "gunicorn.conf", "inspector.main:app"]
ports:
- containerPort: 3000
name: http-gunicorn
readinessProbe:
httpGet:
path: /_health/
port: 3000
httpHeaders:
- name: X-Forwarded-Proto
value: https
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 5
resources:
requests:
cpu: 200m
memory: 1Gi
limits:
cpu: 1500m
memory: 3Gi
env:
- name: ENV
value: "<%= environment %>"
- name: DD_AGENT_HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: STATSD_IMPLEMENTATION
value: "datadog"
- name: STATSD_HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: STATSD_ADDR
value: $(STATSD_HOST):8125
- name: AWS_REGION
value: "us-west-2"
- name: HONEYBADGER_API_KEY
valueFrom:
secretKeyRef:
name: <%= environment %>
key: honeybadger_api_key
- name: DATADOG_CSP_API_KEY
valueFrom:
secretKeyRef:
name: <%= environment %>
key: datadog_csp_api_key
securityContext:
privileged: false
lifecycle:
preStop:
exec:
command: ["sleep", "25"]
- name: nginx
image: nginx:1.25.2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: http-nginx
protocol: TCP
resources:
<% if environment == 'production' %>
requests:
cpu: 100m
memory: 20Mi
limits:
cpu: 300m
memory: 200Mi
<% else %>
requests:
cpu: 20m
memory: 10Mi
limits:
cpu: 60m
memory: 100Mi
<% end %>
volumeMounts:
- name: nginxconfig
mountPath: /etc/nginx
readOnly: true
- name: nginxlog
mountPath: /var/log/nginx
- name: nginxcache
mountPath: /var/lib/nginx/cache
livenessProbe:
httpGet:
path: /nginx_health
port: 80
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /nginx_health
port: 80
initialDelaySeconds: 5
periodSeconds: 5
lifecycle:
# https://blog.gruntwork.io/delaying-shutdown-to-wait-for-pod-deletion-propagation-445f779a8304
preStop:
exec:
command: [
"sh", "-c",
# Introduce a delay to the shutdown sequence to wait for the
# pod eviction event to propagate. Then, gracefully shutdown
# nginx.
"sleep 20 && /usr/sbin/nginx -s quit",
]
volumes:
- name: nginxconfig
configMap:
name: nginx-configmap
items:
- key: default
path: nginx.conf
- key: logging
path: conf.d/logging.conf
- key: rubygems
path: sites-enabled/<%= environment %>.ruybgems.conf
- name: nginxlog
emptyDir: {}
- name: nginxcache
emptyDir: {}
4 changes: 2 additions & 2 deletions inspector/analysis/checks.py
Original file line number Diff line number Diff line change
@@ -3,15 +3,15 @@

from inspector.analysis.codedetails import Detail, DetailSeverity
from inspector.analysis.entropy import shannon_entropy
from inspector.distribution import TarGzDistribution, ZipDistribution
from inspector.distribution import GemDistribution


def __is_compiled(filepath: str) -> bool:
return filepath.endswith(".pyc") or filepath.endswith(".pyo")


def basic_details(
distribution: TarGzDistribution | ZipDistribution, filepath: str
distribution: GemDistribution, filepath: str
) -> Generator[Detail, Any, None]:
contents = distribution.contents(filepath)

42 changes: 0 additions & 42 deletions inspector/deob.py

This file was deleted.

77 changes: 40 additions & 37 deletions inspector/distribution.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import tarfile
import zipfile

from io import BytesIO

@@ -21,30 +20,43 @@ def read(self):
raise NotImplementedError


class ZipDistribution(Distribution):
class GemDistribution(Distribution):
def __init__(self, f):
f.seek(0)
self.zipfile = zipfile.ZipFile(f)
self.tarfile = tarfile.open(fileobj=f, mode="r")

def unpack_name(self, entry):
if entry.name.endswith(".tar.gz"):
return [
"/".join([entry.name, i.name])
for i in tarfile.open(
fileobj=self.tarfile.extractfile(entry), mode="r"
).getmembers()
]
return [entry.name]

def namelist(self):
return [i.filename for i in self.zipfile.infolist() if not i.is_dir()]
list = []
for i in self.tarfile.getmembers():
list.extend(self.unpack_name(i))
return list

def contents(self, filepath) -> bytes:
def _read_data(self, filepath):
try:
return self.zipfile.read(filepath)
except KeyError:
file = tarfile.open(
fileobj=self.tarfile.extractfile("data.tar.gz"), mode="r"
).extractfile(filepath)
if file:
return file.read()
else:
raise FileNotFoundError
except (KeyError, EOFError):
raise FileNotFoundError


class TarGzDistribution(Distribution):
def __init__(self, f):
f.seek(0)
self.tarfile = tarfile.open(fileobj=f, mode="r:gz")

def namelist(self):
return [i.name for i in self.tarfile.getmembers() if not i.isdir()]

def contents(self, filepath):
if filepath.startswith("data.tar.gz"):
return self._read_data(filepath.removeprefix("data.tar.gz/"))

try:
file_ = self.tarfile.extractfile(filepath)
if file_:
@@ -55,11 +67,16 @@ def contents(self, filepath):
raise FileNotFoundError


def _get_dist(first, second, rest, distname):
if distname in dists:
return dists[distname]
def _get_dist(project_name, version, platform):
full_name = (
f"{project_name}-{version}.gem"
if platform == "ruby"
else f"{project_name}-{version}-{platform}.gem"
)
if full_name in dists:
return dists[full_name]

url = f"https://files.pythonhosted.org/packages/{first}/{second}/{rest}/{distname}"
url = f"https://index.rubygems.org/gems/{full_name}"
try:
resp = requests_session().get(url, stream=True)
resp.raise_for_status()
@@ -68,20 +85,6 @@ def _get_dist(first, second, rest, distname):

f = BytesIO(resp.content)

if (
distname.endswith(".whl")
or distname.endswith(".zip")
or distname.endswith(".egg")
):
distfile = ZipDistribution(f)
dists[distname] = distfile
return distfile

elif distname.endswith(".tar.gz"):
distfile = TarGzDistribution(f)
dists[distname] = distfile
return distfile

else:
# Not supported
return None
distfile = GemDistribution(f)
dists[full_name] = distfile
return distfile
172 changes: 85 additions & 87 deletions inspector/main.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import gzip
import itertools
import os
import urllib.parse

@@ -9,10 +11,8 @@
from sentry_sdk.integrations.flask import FlaskIntegration

from .analysis.checks import basic_details
from .deob import decompile, disassemble
from .distribution import _get_dist
from .legacy import parse
from .utilities import pypi_report_form, requests_session
from .utilities import mailto_report_link, requests_session


def traces_sampler(sampling_context):
@@ -48,45 +48,51 @@ def handle_bad_request(e):

@app.route("/")
def index():
if project := request.args.get("project"):
if project := request.args.get("gem"):
project = project.strip()
return redirect(f"/project/{project}")
return redirect(f"/gems/{project}")
return render_template("index.html")


@app.route("/project/<project_name>/")
@app.route("/gems/<project_name>/")
def versions(project_name):
if project_name != canonicalize_name(project_name):
return redirect(
url_for("versions", project_name=canonicalize_name(project_name)), 301
)

resp = requests_session().get(f"https://pypi.org/pypi/{project_name}/json")
pypi_project_url = f"https://pypi.org/project/{project_name}"
resp = requests_session().get(
f"https://rubygems.org/api/v1/versions/{project_name}.json"
)
rubygems_url = f"https://rubygems.org/gems/{project_name}"

# Self-host 404 page to mitigate iframe embeds
if resp.status_code == 404:
return render_template("404.html")
if resp.status_code != 200:
return redirect(pypi_project_url, 307)
return redirect(rubygems_url, 307)

releases = resp.json()["releases"]
sorted_releases = {
version: releases[version]
for version in sorted(releases.keys(), key=parse, reverse=True)
}
releases = resp.json()
by_number = itertools.groupby(releases, lambda x: x["number"])
sorted_releases = {number: list(versions) for (number, versions) in by_number}

return render_template(
"releases.html",
releases=sorted_releases,
h2=project_name,
h2_link=f"/project/{project_name}",
h2_paren="View this project on PyPI",
h2_paren_link=pypi_project_url,
h2_link=f"/gems/{project_name}",
h2_paren="View this project on RubyGems.org",
h2_paren_link=rubygems_url,
)


@app.route("/project/<project_name>/<version>/")
def full_name(version):
if version["platform"] == "ruby":
return version["number"]
return f"{version['number']}-{version['platform']}"


@app.route("/gems/<project_name>/<version>/")
def distributions(project_name, version):
if project_name != canonicalize_name(project_name):
return redirect(
@@ -99,152 +105,144 @@ def distributions(project_name, version):
)

resp = requests_session().get(
f"https://pypi.org/pypi/{project_name}/{version}/json"
f"https://rubygems.org/api/v1/versions/{project_name}.json"
)
if resp.status_code != 200:
return redirect(f"/project/{project_name}/")
return redirect(f"/gems/{project_name}/")

dist_urls = [
"." + urllib.parse.urlparse(url["url"]).path + "/"
for url in resp.json()["urls"]
]
dist_urls = [f"{v['platform']}" for v in resp.json() if v["number"] == version]
return render_template(
"links.html",
links=dist_urls,
h2=f"{project_name}",
h2_link=f"/project/{project_name}",
h2_paren="View this project on PyPI",
h2_paren_link=f"https://pypi.org/project/{project_name}",
h2_link=f"/gems/{project_name}",
h2_paren="View this project on RubyGems.org",
h2_paren_link=f"https://rubygems.org/gems/{project_name}",
h3=f"{project_name}=={version}",
h3_link=f"/project/{project_name}/{version}",
h3_paren="View this release on PyPI",
h3_paren_link=f"https://pypi.org/project/{project_name}/{version}",
h3_link=f"/gems/{project_name}/{version}",
h3_paren="View this release on RubyGems.org",
h3_paren_link=f"https://rubygems.org/gems/{project_name}/versions/{version}",
)


@app.route(
"/project/<project_name>/<version>/packages/<first>/<second>/<rest>/<distname>/"
)
def distribution(project_name, version, first, second, rest, distname):
@app.route("/gems/<project_name>/<version>/<platform>/")
def distribution(project_name, version, platform):
if project_name != canonicalize_name(project_name):
return redirect(
url_for(
"distribution",
project_name=canonicalize_name(project_name),
version=version,
first=first,
second=second,
rest=rest,
distname=distname,
platform=platform,
),
301,
)

dist = _get_dist(first, second, rest, distname)
dist = _get_dist(project_name, version, platform)

h2_paren = "View this project on PyPI"
resp = requests_session().get(f"https://pypi.org/pypi/{project_name}/json")
h2_paren = "View this project on RubyGems.org"
resp = requests_session().get(
f"https://rubygems.org/api/v1/gems/{project_name}.json"
)
if resp.status_code == 404:
h2_paren = "❌ Project no longer on PyPI"
h2_paren = "❌ Project no longer on RubyGems.org"

h3_paren = "View this release on PyPI"
full_name = version if platform == "ruby" else f"{version}-{platform}"
h3_paren = "View this release on RubyGems.org"
resp = requests_session().get(
f"https://pypi.org/pypi/{project_name}/{version}/json"
f"https://rubygems.org/api/v2/rubygems/{project_name}/versions/{full_name}.json"
)
if resp.status_code == 404:
h3_paren = "❌ Release no longer on PyPI"
h3_paren = "❌ Release no longer on RubyGems.org"

if dist:
file_urls = [
"./" + urllib.parse.quote(filename) for filename in dist.namelist()
]
file_urls = [urllib.parse.quote(filename) for filename in dist.namelist()]
return render_template(
"links.html",
links=file_urls,
h2=f"{project_name}",
h2_link=f"/project/{project_name}",
h2_link=f"/gems/{project_name}",
h2_paren=h2_paren,
h2_paren_link=f"https://pypi.org/project/{project_name}",
h2_paren_link=f"https://rubygems.org/gems/{project_name}",
h3=f"{project_name}=={version}",
h3_link=f"/project/{project_name}/{version}",
h3_link=f"/gems/{project_name}/{version}",
h3_paren=h3_paren,
h3_paren_link=f"https://pypi.org/project/{project_name}/{version}",
h4=distname,
h4_link=f"/project/{project_name}/{version}/packages/{first}/{second}/{rest}/{distname}/", # noqa
h3_paren_link=f"https://rubygems.org/gems/{project_name}/versions/{version}",
h4=full_name,
h4_link=f"/gems/{project_name}/{version}/{platform}", # noqa
)
else:
return "Distribution type not supported"


@app.route(
"/project/<project_name>/<version>/packages/<first>/<second>/<rest>/<distname>/<path:filepath>" # noqa
)
def file(project_name, version, first, second, rest, distname, filepath):
@app.route("/gems/<project_name>/<version>/<platform>/<path:filepath>") # noqa
def file(project_name, version, platform, filepath):
if project_name != canonicalize_name(project_name):
return redirect(
url_for(
"file",
project_name=canonicalize_name(project_name),
version=version,
first=first,
second=second,
rest=rest,
distname=distname,
platform=platform,
filepath=filepath,
),
301,
)

h2_paren = "View this project on PyPI"
resp = requests_session().get(f"https://pypi.org/pypi/{project_name}/json")
h2_paren = "View this project on RubyGems.org"
resp = requests_session().get(
f"https://rubygems.org/api/v1/gems/{project_name}.json"
)
if resp.status_code == 404:
h2_paren = "❌ Project no longer on PyPI"
h2_paren = "❌ Project no longer on RubyGems.org"

h3_paren = "View this release on PyPI"
full_name = version if platform == "ruby" else f"{version}-{platform}"
h3_paren = "View this release on RubyGems.org"
resp = requests_session().get(
f"https://pypi.org/pypi/{project_name}/{version}/json"
f"https://rubygems.org/api/v2/rubygems/{project_name}/versions/{full_name}.json"
)
if resp.status_code == 404:
h3_paren = "❌ Release no longer on PyPI"
h3_paren = "❌ Release no longer on RubyGems.org"

dist = _get_dist(first, second, rest, distname)
dist = _get_dist(project_name, version, platform)
if dist:
try:
contents = dist.contents(filepath)
except FileNotFoundError:
return abort(404)
file_extension = filepath.split(".")[-1]
report_link = pypi_report_form(project_name, version, filepath, request.url)
report_link = mailto_report_link(
project_name, version, platform, filepath, request.url
)

details = [detail.html() for detail in basic_details(dist, filepath)]
common_params = {
"file_details": details,
"mailto_report_link": report_link,
"h2": f"{project_name}",
"h2_link": f"/project/{project_name}",
"h2_link": f"/gems/{project_name}",
"h2_paren": h2_paren,
"h2_paren_link": f"https://pypi.org/project/{project_name}",
"h2_paren_link": f"https://rubygems.org/gems/{project_name}",
"h3": f"{project_name}=={version}",
"h3_link": f"/project/{project_name}/{version}",
"h3_link": f"/gems/{project_name}/{version}",
"h3_paren": h3_paren,
"h3_paren_link": f"https://pypi.org/project/{project_name}/{version}",
"h4": distname,
"h4_link": f"/project/{project_name}/{version}/packages/{first}/{second}/{rest}/{distname}/", # noqa
"h3_paren_link": f"https://rubygems.org/gems/{project_name}/versions/{version}",
"h4": full_name,
"h4_link": f"/gems/{project_name}/{version}/{platform}/", # noqa
"h5": filepath,
"h5_link": f"/project/{project_name}/{version}/packages/{first}/{second}/{rest}/{distname}/{filepath}", # noqa
"h5_link": f"/gems/{project_name}/{version}/{platform}/{filepath}", # noqa
}

if file_extension in ["pyc", "pyo"]:
disassembly = disassemble(contents)
decompilation = decompile(contents)
return render_template(
"disasm.html",
disassembly=disassembly,
decompilation=decompilation,
**common_params,
)

if isinstance(contents, bytes):
if filepath.endswith(".gz"):
try:
contents = gzip.decompress(contents)
file_extension = filepath.split(".")[-2]
if filepath == "metadata.gz":
file_extension = "yaml"
except gzip.BadGzipFile:
return "Failed to ungzip."

try:
contents = contents.decode()
except UnicodeDecodeError:
6 changes: 3 additions & 3 deletions inspector/templates/base.html
Original file line number Diff line number Diff line change
@@ -9,9 +9,9 @@
<h1><a href="/">Inspector</a></h1>
<form action="/">
{% if h2 %}
<input type="text" name="project" placeholder="Project name" value="{{ h2 }}" autocomplete="off">
<input type="text" name="gem" placeholder="Gem name" value="{{ h2 }}" autocomplete="off">
{% else %}
<input type="text" name="project" placeholder="Project name" autocomplete="off">
<input type="text" name="gem" placeholder="Gem name" autocomplete="off">
{% endif %}
<input type="submit">
</form>
@@ -62,7 +62,7 @@ <h5>
{% block body %}{% endblock %}
</main>
<footer>
<small>Source: <a href="https://github.com/pypi/inspector">https://github.com/pypi/inspector</a></small>
<small>Source: <a href="https://github.com/rubygems/inspector">https://github.com/rubygems/inspector</a></small>
</footer>
</body>
</html>
2 changes: 1 addition & 1 deletion inspector/templates/releases.html
Original file line number Diff line number Diff line change
@@ -28,7 +28,7 @@
<tr>
{% if value|length > 0 %}
<td><a href="./{{ key }}">{{ key }}</a></td>
<td>{{ value[0]['upload_time'] }}</td>
<td>{{ value[0]['created_at'] }}</td>
<td>{{ value|length }}</td>
{% else %}
<td>{{ key }}</td>
30 changes: 8 additions & 22 deletions inspector/utilities.py
Original file line number Diff line number Diff line change
@@ -3,15 +3,16 @@
import requests


def mailto_report_link(project_name, version, file_path, request_url):
def mailto_report_link(project_name, version, platform, file_path, request_url):
"""
Generate a mailto report link for malicious code.
"""
message_body = (
"PyPI Malicious Package Report\n"
"RubyGems Malicious Package Report\n"
"--\n"
f"Package Name: {project_name}\n"
f"Gem: {project_name}\n"
f"Version: {version}\n"
f"Platform: {platform}\n"
f"File Path: {file_path}\n"
f"Inspector URL: {request_url}\n\n"
"Additional Information:\n\n"
@@ -20,30 +21,15 @@ def mailto_report_link(project_name, version, file_path, request_url):
subject = f"Malicious Package Report: {project_name}"

return (
f"mailto:security@pypi.org?"
f"mailto:security@rubygems.org?"
f"subject={urllib.parse.quote(subject)}"
f"&body={urllib.parse.quote(message_body)}"
)


def pypi_report_form(project_name, version, file_path, request_url):
"""
Generate a URL to PyPI malware report for malicious code.
"""
summary = (
f"Version: {version}\n"
f"File Path: {file_path}\n"
"Additional Information:\n\n"
)

return (
f"https://pypi.org/project/{project_name}/submit-malware-report/"
f"?inspector_link={request_url}"
f"&summary={urllib.parse.quote(summary)}"
)


def requests_session(custom_user_agent: str = "inspector.pypi.io") -> requests.Session:
def requests_session(
custom_user_agent: str = "inspector.rubygems.info",
) -> requests.Session:
"""
Custom `requests` session with default headers applied.
1 change: 1 addition & 0 deletions requirements/deploy.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/usr/bin/env -S pip-compile --allow-unsafe --generate-hashes --output-file=requirements/deploy.txt

gunicorn
ddtrace
197 changes: 193 additions & 4 deletions requirements/deploy.txt
Original file line number Diff line number Diff line change
@@ -1,14 +1,203 @@
#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --allow-unsafe --generate-hashes --output-file=requirements/deploy.txt requirements/deploy.in
# pip-compile --allow-unsafe --generate-hashes --output-file=requirements/deploy.txt ./requirements/deploy.in
#
bytecode==0.16.1 \
--hash=sha256:1d4b61ed6bade4bff44127c8283bef8131a664ce4dbe09d64a88caf329939f35 \
--hash=sha256:8fbbb637c880f339e564858bc6c7984ede67ae97bc71343379a535a9a4baf398
# via ddtrace
ddtrace==3.0.0 \
--hash=sha256:00c5eca3949072851f7d33c63952f923b0eb0b5950e0f7241e6935ea289b570a \
--hash=sha256:089fcbd57ff66c4fb0f6afa6658a712ab4d25199e144c8cf991eb49939c18744 \
--hash=sha256:0ee098c7b500926f5fe64be88269a8aa8579e454bf27c52c9cf083552b701288 \
--hash=sha256:14c01a2e247983b981a1b4a2ba7a2b4413dbc18f95b02b82e0c0f6963a790791 \
--hash=sha256:15a1915316d368fd7281d5d39b1badfda716d98172f4a8dd5a5a545a7573f260 \
--hash=sha256:176ec034d86bc22c485ab63862ffb2a767a5d69a063139c20a62fca9f633ae37 \
--hash=sha256:17dda2845ec41530457676cd1a475e6c8981ec9e05bfa39da4fa7bd2af0c11c4 \
--hash=sha256:19471e50c60f41d4003c08c0dbbdb4d51467d8fdde717368ce2979b3977ed25c \
--hash=sha256:1b1977b11312a2f810112ce45827e59f4211775a2181ac5e3c031a11571d5a7c \
--hash=sha256:1f61d492394b4b1534f3cfeba819fbe1adf3fcf4b773245fb5254645f8b37528 \
--hash=sha256:392299a8722fde8908063ed1743c3117aa21a441617a6f133a57f96296c870ee \
--hash=sha256:3b5e3835106f89940d82f4f188e599d87e53b9cf071979cf8be3cc9e830cbd06 \
--hash=sha256:3e57a4b5c62cc2c3e37b4f69e24506cc471097af9a601b14c9d252130944b48e \
--hash=sha256:3ff34424067fa6c55545319c0a024e0e264696366f847a0c9f173107b8f23b26 \
--hash=sha256:425c08367af9d848791b94bb0ea7d4f2908cb0488eb58447cadf09b0bf2c9132 \
--hash=sha256:4a7f1fd6a0069d1192afc72f83fb4347a779841d7d99c951581fbf930bfbeb86 \
--hash=sha256:4d06a5432efd28a221984dff469d5a5dee85ad56907ecd9ea5b8901998011bb2 \
--hash=sha256:4e6ec651745aa689ee57f37ecf9f0b6b63888570c48af4e1906f9709dcabf4e2 \
--hash=sha256:516d1e85da12e164df7006247577b4a416077a5188662d0bb80fe84d308e097b \
--hash=sha256:56947f44dfae0f87ef5d47836bad9b54ca95e1c95561e77d2abaa03adde6b5c7 \
--hash=sha256:6312e2790a9dbd8ed95f862ab3a82fba0fddad365067582fe744fac47609931b \
--hash=sha256:6742de5a8101e230c5c602417daaf56711b6cf52e561b121898cc51f273fa2c4 \
--hash=sha256:6a648773f555bc5c07e0793f2d1cb617b7650ddbf34b9f619ec5c9cab68bfd10 \
--hash=sha256:6c43196740abe4e138a67334ba8b08d6faabbf334868f39b5fc8a12293266380 \
--hash=sha256:6f6c4f822d2018b1955f69519b30a7ed5b0737710e06608eedf67d6a7dbda822 \
--hash=sha256:6fba529a0153c410bb79fc3ce8de4c68da8d3c40171c7f3879002c5fbf83b834 \
--hash=sha256:7f07462fe3dc030f9f7bf277950c7844cdc0f7ca4a2df4f4571b0dc10f867a43 \
--hash=sha256:85365fab771b9efb515a41083310100700c54ce25be222941dcf8c599dd2e94e \
--hash=sha256:861752fb7cb52797c7d5bb3e55d021f5b9bacee3d7057563f2dd8dbacf2a47eb \
--hash=sha256:86aa2c4224f47201ea3745f23f106e2f5baa4da900d92a05ec36835815bcd23c \
--hash=sha256:8dc0f2d854c4c396a3491a97ab0f10ff6127d392c5999c68fd636e1854a93d50 \
--hash=sha256:901dc1fe8612d855ba10aed57f85db31a40acbb60fd4112b85d185f6f670eafc \
--hash=sha256:918cdaaf1afef8f04d3e658f2d2da642e012e0e86df1d184fa23a5df19689c84 \
--hash=sha256:96fc4ef086b262f7c0906717aca346c68b9590f965302cfc414ddb0e969e065b \
--hash=sha256:9aa58ad390828dad400ad48bfafc257d9112df64695a673feeeed18a50ff6ee8 \
--hash=sha256:9c4735c9c8b35df7430919b9498990578e711a9ede3f4759283be1755480440d \
--hash=sha256:a8101edc6540cccc261a3cfa5103a2c47889b27d1fcb249ba5dd21c745de8b99 \
--hash=sha256:ae85c0bd10e443cbe8cf72cc1e6d87f941a5575980fff73c52eb8abbd822b04a \
--hash=sha256:b44562894bc879e67a99c93582c1820cb00af279de37e6e8c275112d7a94c446 \
--hash=sha256:b91f6043260c36230eb5b8f05cdf0225045d867cc14de088b59070353724c2ec \
--hash=sha256:b9669a4471af928e9ca10881b64a0133d63b9b606eae5888005b45ee3fb17b4d \
--hash=sha256:bc95675628af7053f16e6e2f8c60ad78fd016d6d8103808dd1b890477cf5a5a1 \
--hash=sha256:be01dfda0a17e223b0c17f727183a85e4b5773d91d9c7e65e4c205ee59c7069e \
--hash=sha256:c0f7c3eb64bc843c8dcc32edff3bef1a363ee20a1345223e8d018f0da33e3819 \
--hash=sha256:c8ead28c64688d45d9612573a9006f50ca9cda5282f2592a98e49bbb3605913e \
--hash=sha256:cd0ba978029a828ad0fecb615c4a997600dc587c87f8ef9b66082bdc6f364c55 \
--hash=sha256:cde24148a4d8b5909aad748435384cd1bf46f3524ef755eef63fc40435b732a3 \
--hash=sha256:d0a3876b0695fc4c17798ca15bb6db688aa341800c7a42b9d991c99b806dc4c2 \
--hash=sha256:d9af9073e0ff60d64b95fbbb519fc4201a7cb7091ee5a430f7eac47468f51a4e \
--hash=sha256:db8be7c816faad27bbf4c5580a0e37dd80ee864bc37d4cffe13b794b50d43106 \
--hash=sha256:ddb3944795b3e06e0b7f6641bee92b9a373dbec5c3400bd9f6ca822c34d1c6a0 \
--hash=sha256:e313370c8c1198df007d8ef6c448c1fecc2de0f353ae664ae9fd8e9ddf7ba06b \
--hash=sha256:e50a1a87bc4114a723d371eb75559be2e687ffe325f5eaf22f3da74fe25bc97a \
--hash=sha256:ea1fb7f33099baa530a7689407f36b426635ea5229057a4c13e8a88bd8055d97 \
--hash=sha256:ea6269f2e390986e5790d61b7e44687ef62d923e83b65148224067a7e559c8ac \
--hash=sha256:eb5713327147605a70b935891b56524a6e989e5257bbda98c0fd0217fc1ee004 \
--hash=sha256:f3498054f3821134fe5a7e53bdc0cc37859c8ca8b9867b776efe0acf5f856f41 \
--hash=sha256:f69a7994d967581bdb17919eca735c2b76470bc1039cb1bd7c059e47d12b62d2 \
--hash=sha256:fd14da22dc900edb146c489c7d7fbaeb2e516115c45887fa9e964f9931094564
# via -r ./requirements/deploy.in
deprecated==1.2.18 \
--hash=sha256:422b6f6d859da6f2ef57857761bfb392480502a64c3028ca9bbe86085d72115d \
--hash=sha256:bd5011788200372a32418f888e326a09ff80d0214bd961147cfed01b5c018eec
# via opentelemetry-api
envier==0.6.1 \
--hash=sha256:3309a01bb3d8850c9e7a31a5166d5a836846db2faecb79b9cb32654dd50ca9f9 \
--hash=sha256:73609040a76be48bbcb97074d9969666484aa0de706183a6e9ef773156a8a6a9
# via ddtrace
gunicorn==22.0.0 \
--hash=sha256:350679f91b24062c86e386e198a15438d53a7a8207235a78ba1b53df4c4378d9 \
--hash=sha256:4a0b436239ff76fb33f11c07a16482c521a7e09c1ce3cc293c2330afe01bec63
# via -r deploy.in
# via -r ./requirements/deploy.in
importlib-metadata==8.5.0 \
--hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \
--hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7
# via opentelemetry-api
opentelemetry-api==1.30.0 \
--hash=sha256:375893400c1435bf623f7dfb3bcd44825fe6b56c34d0667c542ea8257b1a1240 \
--hash=sha256:d5f5284890d73fdf47f843dda3210edf37a38d66f44f2b5aedc1e89ed455dc09
# via ddtrace
packaging==23.1 \
--hash=sha256:994793af429502c4ea2ebf6bf664629d07c1a9fe974af92966e4b8d2df7edc61 \
--hash=sha256:a392980d2b6cffa644431898be54b0045151319d1e7ec34f0cfed48767dd334f
# via gunicorn
protobuf==5.29.3 \
--hash=sha256:0a18ed4a24198528f2333802eb075e59dea9d679ab7a6c5efb017a59004d849f \
--hash=sha256:0eb32bfa5219fc8d4111803e9a690658aa2e6366384fd0851064b963b6d1f2a7 \
--hash=sha256:3ea51771449e1035f26069c4c7fd51fba990d07bc55ba80701c78f886bf9c888 \
--hash=sha256:5da0f41edaf117bde316404bad1a486cb4ededf8e4a54891296f648e8e076620 \
--hash=sha256:6ce8cc3389a20693bfde6c6562e03474c40851b44975c9b2bf6df7d8c4f864da \
--hash=sha256:84a57163a0ccef3f96e4b6a20516cedcf5bb3a95a657131c5c3ac62200d23252 \
--hash=sha256:a4fa6f80816a9a0678429e84973f2f98cbc218cca434abe8db2ad0bffc98503a \
--hash=sha256:a8434404bbf139aa9e1300dbf989667a83d42ddda9153d8ab76e0d5dcaca484e \
--hash=sha256:b89c115d877892a512f79a8114564fb435943b59067615894c3b13cd3e1fa107 \
--hash=sha256:c027e08a08be10b67c06bf2370b99c811c466398c357e615ca88c91c07f0910f \
--hash=sha256:daaf63f70f25e8689c072cfad4334ca0ac1d1e05a92fc15c54eb9cf23c3efd84
# via ddtrace
typing-extensions==4.12.2 \
--hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \
--hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8
# via ddtrace
wrapt==1.17.2 \
--hash=sha256:08e7ce672e35efa54c5024936e559469436f8b8096253404faeb54d2a878416f \
--hash=sha256:0a6e821770cf99cc586d33833b2ff32faebdbe886bd6322395606cf55153246c \
--hash=sha256:0b929ac182f5ace000d459c59c2c9c33047e20e935f8e39371fa6e3b85d56f4a \
--hash=sha256:129a150f5c445165ff941fc02ee27df65940fcb8a22a61828b1853c98763a64b \
--hash=sha256:13e6afb7fe71fe7485a4550a8844cc9ffbe263c0f1a1eea569bc7091d4898555 \
--hash=sha256:1473400e5b2733e58b396a04eb7f35f541e1fb976d0c0724d0223dd607e0f74c \
--hash=sha256:18983c537e04d11cf027fbb60a1e8dfd5190e2b60cc27bc0808e653e7b218d1b \
--hash=sha256:1a7ed2d9d039bd41e889f6fb9364554052ca21ce823580f6a07c4ec245c1f5d6 \
--hash=sha256:1e1fe0e6ab7775fd842bc39e86f6dcfc4507ab0ffe206093e76d61cde37225c8 \
--hash=sha256:1fb5699e4464afe5c7e65fa51d4f99e0b2eadcc176e4aa33600a3df7801d6662 \
--hash=sha256:2696993ee1eebd20b8e4ee4356483c4cb696066ddc24bd70bcbb80fa56ff9061 \
--hash=sha256:35621ae4c00e056adb0009f8e86e28eb4a41a4bfa8f9bfa9fca7d343fe94f998 \
--hash=sha256:36ccae62f64235cf8ddb682073a60519426fdd4725524ae38874adf72b5f2aeb \
--hash=sha256:3cedbfa9c940fdad3e6e941db7138e26ce8aad38ab5fe9dcfadfed9db7a54e62 \
--hash=sha256:3d57c572081fed831ad2d26fd430d565b76aa277ed1d30ff4d40670b1c0dd984 \
--hash=sha256:3fc7cb4c1c744f8c05cd5f9438a3caa6ab94ce8344e952d7c45a8ed59dd88392 \
--hash=sha256:4011d137b9955791f9084749cba9a367c68d50ab8d11d64c50ba1688c9b457f2 \
--hash=sha256:40d615e4fe22f4ad3528448c193b218e077656ca9ccb22ce2cb20db730f8d306 \
--hash=sha256:410a92fefd2e0e10d26210e1dfb4a876ddaf8439ef60d6434f21ef8d87efc5b7 \
--hash=sha256:41388e9d4d1522446fe79d3213196bd9e3b301a336965b9e27ca2788ebd122f3 \
--hash=sha256:468090021f391fe0056ad3e807e3d9034e0fd01adcd3bdfba977b6fdf4213ea9 \
--hash=sha256:49703ce2ddc220df165bd2962f8e03b84c89fee2d65e1c24a7defff6f988f4d6 \
--hash=sha256:4a721d3c943dae44f8e243b380cb645a709ba5bd35d3ad27bc2ed947e9c68192 \
--hash=sha256:4afd5814270fdf6380616b321fd31435a462019d834f83c8611a0ce7484c7317 \
--hash=sha256:4c82b8785d98cdd9fed4cac84d765d234ed3251bd6afe34cb7ac523cb93e8b4f \
--hash=sha256:4db983e7bca53819efdbd64590ee96c9213894272c776966ca6306b73e4affda \
--hash=sha256:582530701bff1dec6779efa00c516496968edd851fba224fbd86e46cc6b73563 \
--hash=sha256:58455b79ec2661c3600e65c0a716955adc2410f7383755d537584b0de41b1d8a \
--hash=sha256:58705da316756681ad3c9c73fd15499aa4d8c69f9fd38dc8a35e06c12468582f \
--hash=sha256:5bb1d0dbf99411f3d871deb6faa9aabb9d4e744d67dcaaa05399af89d847a91d \
--hash=sha256:5c803c401ea1c1c18de70a06a6f79fcc9c5acfc79133e9869e730ad7f8ad8ef9 \
--hash=sha256:5cbabee4f083b6b4cd282f5b817a867cf0b1028c54d445b7ec7cfe6505057cf8 \
--hash=sha256:612dff5db80beef9e649c6d803a8d50c409082f1fedc9dbcdfde2983b2025b82 \
--hash=sha256:62c2caa1585c82b3f7a7ab56afef7b3602021d6da34fbc1cf234ff139fed3cd9 \
--hash=sha256:69606d7bb691b50a4240ce6b22ebb319c1cfb164e5f6569835058196e0f3a845 \
--hash=sha256:6d9187b01bebc3875bac9b087948a2bccefe464a7d8f627cf6e48b1bbae30f82 \
--hash=sha256:6ed6ffac43aecfe6d86ec5b74b06a5be33d5bb9243d055141e8cabb12aa08125 \
--hash=sha256:703919b1633412ab54bcf920ab388735832fdcb9f9a00ae49387f0fe67dad504 \
--hash=sha256:766d8bbefcb9e00c3ac3b000d9acc51f1b399513f44d77dfe0eb026ad7c9a19b \
--hash=sha256:80dd7db6a7cb57ffbc279c4394246414ec99537ae81ffd702443335a61dbf3a7 \
--hash=sha256:8112e52c5822fc4253f3901b676c55ddf288614dc7011634e2719718eaa187dc \
--hash=sha256:8c8b293cd65ad716d13d8dd3624e42e5a19cc2a2f1acc74b30c2c13f15cb61a6 \
--hash=sha256:8fdbdb757d5390f7c675e558fd3186d590973244fab0c5fe63d373ade3e99d40 \
--hash=sha256:91bd7d1773e64019f9288b7a5101f3ae50d3d8e6b1de7edee9c2ccc1d32f0c0a \
--hash=sha256:95c658736ec15602da0ed73f312d410117723914a5c91a14ee4cdd72f1d790b3 \
--hash=sha256:99039fa9e6306880572915728d7f6c24a86ec57b0a83f6b2491e1d8ab0235b9a \
--hash=sha256:9a2bce789a5ea90e51a02dfcc39e31b7f1e662bc3317979aa7e5538e3a034f72 \
--hash=sha256:9a7d15bbd2bc99e92e39f49a04653062ee6085c0e18b3b7512a4f2fe91f2d681 \
--hash=sha256:9abc77a4ce4c6f2a3168ff34b1da9b0f311a8f1cfd694ec96b0603dff1c79438 \
--hash=sha256:9e8659775f1adf02eb1e6f109751268e493c73716ca5761f8acb695e52a756ae \
--hash=sha256:9fee687dce376205d9a494e9c121e27183b2a3df18037f89d69bd7b35bcf59e2 \
--hash=sha256:a5aaeff38654462bc4b09023918b7f21790efb807f54c000a39d41d69cf552cb \
--hash=sha256:a604bf7a053f8362d27eb9fefd2097f82600b856d5abe996d623babd067b1ab5 \
--hash=sha256:abbb9e76177c35d4e8568e58650aa6926040d6a9f6f03435b7a522bf1c487f9a \
--hash=sha256:acc130bc0375999da18e3d19e5a86403667ac0c4042a094fefb7eec8ebac7cf3 \
--hash=sha256:b18f2d1533a71f069c7f82d524a52599053d4c7166e9dd374ae2136b7f40f7c8 \
--hash=sha256:b4e42a40a5e164cbfdb7b386c966a588b1047558a990981ace551ed7e12ca9c2 \
--hash=sha256:b5e251054542ae57ac7f3fba5d10bfff615b6c2fb09abeb37d2f1463f841ae22 \
--hash=sha256:b60fb58b90c6d63779cb0c0c54eeb38941bae3ecf7a73c764c52c88c2dcb9d72 \
--hash=sha256:b870b5df5b71d8c3359d21be8f0d6c485fa0ebdb6477dda51a1ea54a9b558061 \
--hash=sha256:ba0f0eb61ef00ea10e00eb53a9129501f52385c44853dbd6c4ad3f403603083f \
--hash=sha256:bb87745b2e6dc56361bfde481d5a378dc314b252a98d7dd19a651a3fa58f24a9 \
--hash=sha256:bb90fb8bda722a1b9d48ac1e6c38f923ea757b3baf8ebd0c82e09c5c1a0e7a04 \
--hash=sha256:bc570b5f14a79734437cb7b0500376b6b791153314986074486e0b0fa8d71d98 \
--hash=sha256:c86563182421896d73858e08e1db93afdd2b947a70064b813d515d66549e15f9 \
--hash=sha256:c958bcfd59bacc2d0249dcfe575e71da54f9dcf4a8bdf89c4cb9a68a1170d73f \
--hash=sha256:d18a4865f46b8579d44e4fe1e2bcbc6472ad83d98e22a26c963d46e4c125ef0b \
--hash=sha256:d5e2439eecc762cd85e7bd37161d4714aa03a33c5ba884e26c81559817ca0925 \
--hash=sha256:e3890b508a23299083e065f435a492b5435eba6e304a7114d2f919d400888cc6 \
--hash=sha256:e496a8ce2c256da1eb98bd15803a79bee00fc351f5dfb9ea82594a3f058309e0 \
--hash=sha256:e8b2816ebef96d83657b56306152a93909a83f23994f4b30ad4573b00bd11bb9 \
--hash=sha256:eaf675418ed6b3b31c7a989fd007fa7c3be66ce14e5c3b27336383604c9da85c \
--hash=sha256:ec89ed91f2fa8e3f52ae53cd3cf640d6feff92ba90d62236a81e4e563ac0e991 \
--hash=sha256:ecc840861360ba9d176d413a5489b9a0aff6d6303d7e733e2c4623cfa26904a6 \
--hash=sha256:f09b286faeff3c750a879d336fb6d8713206fc97af3adc14def0cdd349df6000 \
--hash=sha256:f393cda562f79828f38a819f4788641ac7c4085f30f1ce1a68672baa686482bb \
--hash=sha256:f917c1180fdb8623c2b75a99192f4025e412597c50b2ac870f156de8fb101119 \
--hash=sha256:fc78a84e2dfbc27afe4b2bd7c80c8db9bca75cc5b85df52bfe634596a1da846b \
--hash=sha256:ff04ef6eec3eee8a5efef2401495967a916feaa353643defcc03fc74fe213b58
# via
# ddtrace
# deprecated
xmltodict==0.14.2 \
--hash=sha256:201e7c28bb210e374999d1dde6382923ab0ed1a8a5faeece48ab525b7810a553 \
--hash=sha256:20cc7d723ed729276e808f26fb6b3599f786cbc37e06c65e192ba77c40f20aac
# via ddtrace
zipp==3.21.0 \
--hash=sha256:2c9958f6430a2040341a52eb608ed6dd93ef4392e02ffe219417c1b28b5dd1f4 \
--hash=sha256:ac1bbe05fd2991f160ebce24ffbac5f6d11d83dc90891255885223d42b3cd931
# via importlib-metadata
3 changes: 1 addition & 2 deletions requirements/lint.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#!/usr/bin/env -S pip-compile --allow-unsafe --generate-hashes --output-file=requirements/lint.txt

black
curlylint
flake8
ruff
isort
96 changes: 26 additions & 70 deletions requirements/lint.txt
Original file line number Diff line number Diff line change
@@ -1,98 +1,54 @@
#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --allow-unsafe --generate-hashes --output-file=requirements/lint.txt ./requirements/lint.in
#
attrs==21.4.0 \
--hash=sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4 \
--hash=sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd
# via curlylint
black==24.3.0 \
--hash=sha256:2818cf72dfd5d289e48f37ccfa08b460bf469e67fb7c4abb07edc2e9f16fb63f \
--hash=sha256:41622020d7120e01d377f74249e677039d20e6344ff5851de8a10f11f513bf93 \
--hash=sha256:4acf672def7eb1725f41f38bf6bf425c8237248bb0804faa3965c036f7672d11 \
--hash=sha256:4be5bb28e090456adfc1255e03967fb67ca846a03be7aadf6249096100ee32d0 \
--hash=sha256:4f1373a7808a8f135b774039f61d59e4be7eb56b2513d3d2f02a8b9365b8a8a9 \
--hash=sha256:56f52cfbd3dabe2798d76dbdd299faa046a901041faf2cf33288bc4e6dae57b5 \
--hash=sha256:65b76c275e4c1c5ce6e9870911384bff5ca31ab63d19c76811cb1fb162678213 \
--hash=sha256:65c02e4ea2ae09d16314d30912a58ada9a5c4fdfedf9512d23326128ac08ac3d \
--hash=sha256:6905238a754ceb7788a73f02b45637d820b2f5478b20fec82ea865e4f5d4d9f7 \
--hash=sha256:79dcf34b33e38ed1b17434693763301d7ccbd1c5860674a8f871bd15139e7837 \
--hash=sha256:7bb041dca0d784697af4646d3b62ba4a6b028276ae878e53f6b4f74ddd6db99f \
--hash=sha256:7d5e026f8da0322b5662fa7a8e752b3fa2dac1c1cbc213c3d7ff9bdd0ab12395 \
--hash=sha256:9f50ea1132e2189d8dff0115ab75b65590a3e97de1e143795adb4ce317934995 \
--hash=sha256:a0c9c4a0771afc6919578cec71ce82a3e31e054904e7197deacbc9382671c41f \
--hash=sha256:aadf7a02d947936ee418777e0247ea114f78aff0d0959461057cae8a04f20597 \
--hash=sha256:b5991d523eee14756f3c8d5df5231550ae8993e2286b8014e2fdea7156ed0959 \
--hash=sha256:bf21b7b230718a5f08bd32d5e4f1db7fc8788345c8aea1d155fc17852b3410f5 \
--hash=sha256:c45f8dff244b3c431b36e3224b6be4a127c6aca780853574c00faf99258041eb \
--hash=sha256:c7ed6668cbbfcd231fa0dc1b137d3e40c04c7f786e626b405c62bcd5db5857e4 \
--hash=sha256:d7de8d330763c66663661a1ffd432274a2f92f07feeddd89ffd085b5744f85e7 \
--hash=sha256:e19cb1c6365fd6dc38a6eae2dcb691d7d83935c10215aef8e6c38edee3f77abd \
--hash=sha256:e2af80566f43c85f5797365077fb64a393861a3730bd110971ab7a0c94e873e7
# via -r lint.in
click==8.1.3 \
--hash=sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e \
--hash=sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48
# via
# black
# curlylint
# via curlylint
curlylint==0.13.1 \
--hash=sha256:008b9d160f3920404ac12efb05c0a39e209cb972f9aafd956b79c5f4e2162752 \
--hash=sha256:9546ea82cdfc9292fd6fe49dca28587164bd315782a209c0a46e013d7f38d2fa
# via -r lint.in
flake8==6.0.0 \
--hash=sha256:3833794e27ff64ea4e9cf5d410082a8b97ff1a06c16aa3d2027339cd0f1195c7 \
--hash=sha256:c61007e76655af75e6785a931f452915b371dc48f56efd765247c8fe68f2b181
# via -r lint.in
# via -r ./requirements/lint.in
isort==5.12.0 \
--hash=sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504 \
--hash=sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6
# via -r lint.in
mccabe==0.7.0 \
--hash=sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325 \
--hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e
# via flake8
mypy-extensions==0.4.3 \
--hash=sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d \
--hash=sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8
# via black
packaging==23.1 \
--hash=sha256:994793af429502c4ea2ebf6bf664629d07c1a9fe974af92966e4b8d2df7edc61 \
--hash=sha256:a392980d2b6cffa644431898be54b0045151319d1e7ec34f0cfed48767dd334f
# via black
# via -r ./requirements/lint.in
parsy==1.1.0 \
--hash=sha256:25bd5cea2954950ebbfdf71f8bdaf7fd45a5df5325fd36a1064be2204d9d4c94 \
--hash=sha256:36173ba01a5372c7a1b32352cc73a279a49198f52252adf1c8c1ed41d1f94e8d
# via curlylint
pathspec==0.9.0 \
--hash=sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a \
--hash=sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1
# via
# black
# curlylint
platformdirs==2.5.2 \
--hash=sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788 \
--hash=sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19
# via black
pycodestyle==2.10.0 \
--hash=sha256:347187bdb476329d98f695c213d7295a846d1152ff4fe9bacb8a9590b8ee7053 \
--hash=sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d470186815706610
# via flake8
pyflakes==3.0.1 \
--hash=sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf \
--hash=sha256:ec8b276a6b60bd80defed25add7e439881c19e64850afd9b346283d4165fd0fd
# via flake8
# via curlylint
ruff==0.9.7 \
--hash=sha256:0372c5a90349f00212270421fe91874b866fd3626eb3b397ede06cd385f6f7e0 \
--hash=sha256:042ae32b41343888f59c0a4148f103208bf6b21c90118d51dc93a68366f4e903 \
--hash=sha256:0c439bdfc8983e1336577f00e09a4e7a78944fe01e4ea7fe616d00c3ec69a3d0 \
--hash=sha256:115d1f15e8fdd445a7b4dc9a30abae22de3f6bcabeb503964904471691ef7606 \
--hash=sha256:3770fe52b9d691a15f0b87ada29c45324b2ace8f01200fb0c14845e499eb0c2c \
--hash=sha256:643757633417907510157b206e490c3aa11cab0c087c912f60e07fbafa87a4c6 \
--hash=sha256:7c1f880ac5b2cbebd58b8ebde57069a374865c73f3bf41f05fe7a179c1c8ef22 \
--hash=sha256:87862589373b33cc484b10831004e5e5ec47dc10d2b41ba770e837d4f429d721 \
--hash=sha256:88362e3227c82f63eaebf0b2eff5b88990280fb1ecf7105523883ba8c3aaf6fb \
--hash=sha256:91ff963baed3e9a6a4eba2a02f4ca8eaa6eba1cc0521aec0987da8d62f53cbef \
--hash=sha256:99d50def47305fe6f233eb8dabfd60047578ca87c9dcb235c9723ab1175180f4 \
--hash=sha256:a17e1e01bee0926d351a1ee9bc15c445beae888f90069a6192a07a84af544b6b \
--hash=sha256:b075a700b2533feb7a01130ff656a4ec0d5f340bb540ad98759b8401c32c2037 \
--hash=sha256:d59105ae9c44152c3d40a9c40d6331a7acd1cdf5ef404fbe31178a77b174ea66 \
--hash=sha256:d76b8ab60e99e6424cd9d3d923274a1324aefce04f8ea537136b8398bbae0a62 \
--hash=sha256:e63fc20143c291cab2841dbb8260e96bafbe1ba13fd3d60d28be2c71e312da49 \
--hash=sha256:e9ece95b7de5923cbf38893f066ed2872be2f2f477ba94f826c8defdd6ec6b7d \
--hash=sha256:f313b5800483770bd540cddac7c90fc46f895f427b7820f18fe1822697f1fec9
# via -r ./requirements/lint.in
toml==0.10.2 \
--hash=sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b \
--hash=sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f
# via curlylint
tomli==2.0.1 \
--hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \
--hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f
# via black
typing-extensions==4.12.2 \
--hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \
--hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8
# via black
37 changes: 37 additions & 0 deletions shipit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
machine:
environment:
KUBECONFIG: /home/deploy/.kube/config

dependencies:
override:
- "true"

deploy:
override:
- krane render -f config/deploy/$ENVIRONMENT --bindings=environment=$ENVIRONMENT --current-sha=$REVISION | krane deploy inspector-$ENVIRONMENT rubygems --stdin -f config/deploy/$ENVIRONMENT/secrets.ejson

rollback:
override:
- krane render -f config/deploy/$ENVIRONMENT --bindings=environment=$ENVIRONMENT --current-sha=$REVISION | krane deploy inspector-$ENVIRONMENT rubygems --stdin -f config/deploy/$ENVIRONMENT/secrets.ejson

tasks:
restart:
action: Restart application
description: Trigger the restart of gunicorn
steps:
- krane restart inspector-$ENVIRONMENT rubygems
autoscale_web:
action: Autoscale web deployment
description: "Autoscale the web deployment to %{MIN}-%{MAX} replicas"
steps:
- kubectl autoscale -n inspector-$ENVIRONMENT --context rubygems deployment/web --min="$MIN" --max="$MAX" --cpu-percentage="$CPU_PERCENTAGE"
variables:
- name: MIN
title: The minimum number of web replicas
default: 1
- name: MAX
title: The maximum number of web replicas
default: 3
- name: CPU_PERCENTAGE
title: The threshold of CPU utilization used for autoscaling
default: 75
22 changes: 16 additions & 6 deletions tests/test_main.py
Original file line number Diff line number Diff line change
@@ -4,7 +4,11 @@


def test_versions(monkeypatch):
stub_json = {"releases": {"0.5.1e": None}}
stub_json = [
{"number": "1.0", "platform": "ruby"},
{"number": "1.0", "platform": "java"},
{"number": "2.0", "platform": "ruby"},
]
stub_response = pretend.stub(
status_code=200,
json=lambda: stub_json,
@@ -19,14 +23,20 @@ def test_versions(monkeypatch):

inspector.main.versions("foo")

assert get.calls == [pretend.call("https://pypi.org/pypi/foo/json")]
assert get.calls == [pretend.call("https://rubygems.org/api/v1/versions/foo.json")]
assert render_template.calls == [
pretend.call(
"releases.html",
releases={"0.5.1e": None},
releases={
"1.0": [
{"number": "1.0", "platform": "ruby"},
{"number": "1.0", "platform": "java"},
],
"2.0": [{"number": "2.0", "platform": "ruby"}],
},
h2="foo",
h2_link="/project/foo",
h2_paren="View this project on PyPI",
h2_paren_link="https://pypi.org/project/foo",
h2_link="/gems/foo",
h2_paren="View this project on RubyGems.org",
h2_paren_link="https://rubygems.org/gems/foo",
)
]