Skip to content

Develop#44

Merged
0xmanhnv merged 25 commits into
mainfrom
develop
Apr 15, 2026
Merged

Develop#44
0xmanhnv merged 25 commits into
mainfrom
develop

Conversation

@0xmanhnv
Copy link
Copy Markdown
Collaborator

No description provided.

0xmanhnv and others added 23 commits April 14, 2026 15:49
- Add KEVEscalator interface + postgres implementation
- Single SQL UPDATE joins findings with kev_catalog
- Runs after daily EPSS/KEV sync in ThreatIntelRefreshController
- Only escalates non-critical, non-terminal findings
- Logs count of escalated findings
Jira Integration:
- Add Jira REST API client (internal/infra/jira/client.go)
- Add CreateTicketFromFinding method to JiraSyncService
- Add POST /findings/{id}/create-ticket endpoint
- Auto-link created ticket to finding via work_item_uris
- Map finding severity → Jira priority (critical→Highest, etc.)

BAS Execution Engine:
- Add RunSimulation method to SimulationService
- Add POST /simulations/{id}/run endpoint
- Add GET /simulations/{id}/runs endpoint
- Inline execution for atomic/campaign/control_test types
- Detection/prevention rate tracking per run
- Update simulation stats (total_runs, detection_rate) after each run
- POST /api/v1/assets/import/nessus — parse Nessus XML, create host assets
  with OS detection, IP/FQDN/MAC properties, open port count
- POST /api/v1/assets/import/kubernetes — import cluster + workloads
  with namespace, kind, replicas, images, labels as properties
- POST /api/v1/assets/import/csv — bulk import from CSV
  with name, type, sub_type, description, tags, properties columns
- All endpoints: tenant-scoped, permission-gated (AssetsWrite)
- Body size limits: Nessus 100MB, CSV 50MB
- Duplicate handling: skip existing assets (idempotent)
- Jira client: validate baseURL (block internal/metadata IPs), require HTTPS
- Jira client: limit response body to 10MB (prevent OOM)
- CSV import: cap at 100k rows and 100 errors (prevent memory exhaustion)
- KEV escalation: add clarifying comment (cross-tenant is intentional for global KEV)
- Fix filter: change from @> jsonb_build_object (type-strict) to ->> text
  comparison. Facets extract via ->> (text), filter must match same way.
  Fixes: cdn_enabled=true showing 12 in facet but 0 in filter.
- Add property alias normalization: nameserver→nameservers,
  technology→technologies, san→sans, resolved_ip→resolved_ips.
  Prevents duplicate facet entries from singular/plural variants.
- PropertiesFilter changed from map[string]string to map[string][]string
- Single key can now have multiple values (OR logic within key)
- Array JSONB values (e.g. nameservers) unwrapped in facets via
  jsonb_array_elements_text — shows individual values, not raw JSON
- Filter SQL uses ->> for scalar match + @> for array containment
- Parser supports repeated key:value pairs (up to 20 values, 10 keys)
- Stats CTE was missing 'properties' column causing 500 error
  when count_by metadata fields were requested (e.g. is_virtual)
- Update ParsePropertiesFilter tests for new map[string][]string type
- Add multi-value same key test case
- Change GetVulnerableComponents from limit-based to page+per_page
- Add COUNT query for total (accurate pagination)
- Update domain interface, service, handler, repository
- Handler now uses pagination.New(page, per_page) like all other endpoints
- Fix unit test mocks for new signature
- Auto-detect format from JSON (bomFormat vs spdxVersion)
- CycloneDX: parse components with purl, licenses, scope
- SPDX: parse packages with externalRefs (purl), licenseDeclared
- Upsert global components + link to asset via asset_id param
- Extract ecosystem from PURL (pkg:npm/ → npm, pkg:pypi/ → pypi)
- Link licenses to component_licenses table
- 50MB body limit, max 50 error messages
- Idempotent: duplicate links silently skipped
- GET /repositories/{id}/branches/compare?base=main&compare=feature/x
- Returns new_findings, resolved_findings, common_findings counts
- new_by_severity breakdown (critical, high, medium, low)
- new_items: up to 50 new finding details (title, severity, file_path)
- Comparison by fingerprint (same finding = same fingerprint)
- SQL uses CTE for efficient set operations
3 triggers that keep denormalized counts accurate:
1. findings INSERT/UPDATE/DELETE → update repository_branches.findings_*
2. repository_branches INSERT/DELETE → update asset_repositories.branch_count
3. asset_components INSERT/DELETE → update asset_repositories.component_count

Also backfills all existing stale counts from current data.
Adds report schedule management with list/create/get/delete/toggle
endpoints under /api/v1/reports/schedules with permission-gated access.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Validate cron expression length, report type, format in entity
- Validate recipients: max 50, email format, length
- Validate timezone via time.LoadLocation before storage
- Cap options to max 100 fields
- Cap per_page to 1-100 range in handler
- Add LIMIT 500 to ListDue query
- Escape ILIKE pattern in remediation campaign search
Multi-layer deduplication system for assets across different sources:

Layer 1: Name normalization (16 asset types, 158 test cases)
- DNS: lowercase + strip trailing dot
- IP: canonical via net.ParseIP (IPv4/v6, strip brackets/port/CIDR)
- Host: DNS normalize or IP canonical
- Repository: strip protocol/SSH/.git, preserve platform host
- URL: strip default ports, lowercase host, strip query
- Certificate: lowercase, normalize fingerprint format
- Database: strip protocol/credentials/query
- Network: canonical CIDR
- Storage: extract S3 bucket name from various URL formats
- Plus: service, container, kubernetes, cloud, identity types

Layer 2: IP correlation for host dedup
- Batch FindByIPs query (GIN + btree indexes)
- Staleness check (configurable per-tenant, default 30 days)
- Name quality scoring (FQDN > hostname > IP)
- Auto-rename when better name found (old name in aliases)

Layer 3: Extended correlation
- Repository: integration URL prefix + suffix match
- Cloud/IAM: external_id correlation
- Certificate: fingerprint correlation

Infrastructure:
- asset_merge_log table (audit trail)
- asset_dedup_review table (admin approval queue)
- Per-tenant config (AssetIdentitySettings in tenant settings)
- Admin endpoints: GET/POST dedup reviews, merge log
- Alias search in asset repository
- Integration test scripts
- Normalize input.Name before GetByName lookup in CreateAsset
  (prevents upsert miss when name case differs)
- Update test expectations to match normalized names
  (host type names are now lowercased by NormalizeName)
- Create docs/architecture/asset-identity-resolution.md (feature doc)
- Mark RFC-001 status as Completed with link to feature doc
- Update all phase statuses to COMPLETE
Use LEFT JOIN on findings table to compute count instead of
referencing non-existent finding_count column on assets table.
sync.Map value could theoretically be non-time.Time due to race.
Use comma-ok pattern to prevent panic, delete corrupt entry.
Previously 44 instances of `rows, _ := result.RowsAffected()` silently
ignored driver errors, causing "not found" returns instead of actual
error propagation.

Now all check rowErr and return it properly.
Comment thread internal/infra/postgres/component_repository.go Fixed
CodeQL HIGH: make([]VulnerableComponent, 0, page.Limit()) allocates
based on user-provided pagination limit. Cap at 100 to prevent
excessive memory allocation from crafted requests.
Comment thread internal/infra/postgres/component_repository.go Fixed
@0xmanhnv 0xmanhnv merged commit 598782a into main Apr 15, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants