Skip to content

v1.4.2

Choose a tag to compare

@github-actions github-actions released this 19 Oct 08:39
· 131 commits to main since this release

🚀 新增功能

  • 企业全景测绘与自动建组下沉后端:天眼查与工信部查询合并重构为企业全景测绘(全景双轨 / 官方备案),支持企业工商全称自动检索匹配天眼查 GID 并升级双轨穿透;自动建组意图(auto_group、auto_group_scan、探测策略与周期)由后端调度器消费,取代浏览器页面轮询,升级前在途任务经增量迁移补齐意图;在途与失联的 updating_icp_task_id 按任务终态自动晋升或清理。
  • 普通扫描任务关联资产组:新建资产侦查任务支持可选「关联资产组」,扫描完成后自动同步至该组;目标派生数量如实预告(域名逐个建任务,IP / CIDR 合并为一个任务);任务列表新增资产组列与同步状态透出,同步入口统一经资产组选择器下发。
  • 站点状态多选与 HTTP 状态码多选筛选:资产站点支持多 HTTP 状态码组合筛选(status__in / status_code__in),非法输入直接报错以收敛查询范围;新旧单值条件并存时按交集收敛,结果不依赖参数顺序。
  • 系统设置可搜索与凭据总览:系统设置顶部新增关键词搜索直达目标 Tab 并高亮,跨 Tab 凭据配置状态收敛为一行摘要(默认折叠),未保存改动常驻提示并支持跳转;系统更新页新增开源仓库与版本发布动态入口。
  • 任务聚合态筛选与统计口径对齐:任务列表指标卡与筛选共用后端判定(active 为非终态、abnormal 为 error + stop),卡片数量与点击筛选结果数量保持一致;阶段状态统一标注为「进行中 · <阶段名>」。

🛠️ 性能与重构

  • 系统设置信息架构重组与并发治理收口:设置页收敛为资产测绘与情报、字典与规则库、扫描与并发、通知与回调、系统版本与更新;GitHub 监控队列纳入并发治理并改用峰值模型推导推荐值,三项队列并发语义限定为 Worker 进程数,与单任务内发包并发度分开表述;保存时未传递字段保留库中现值,规避静默覆盖,热扩缩容按运行态进程数重试并回报实际收敛结果。
  • GeoIP 库按需下载与命名卷剥离镜像:ip2region / GeoLite2 库文件不再随镜像分发,部署时由 tools/fetch_geoip.py 按需获取并校验后装入命名卷 arl_geoip(无 MAXMIND_LICENSE_KEY 时 GeoLite2 跳过并降级运行);配套新增内存换出观察工具与队列 OOM 牺牲优先级区分。
  • 配置模板与运行时配置分离:运行生效配置与入库模板分离,测试与验证脚本中的环境标识与凭据改由环境变量注入;构建上下文排除微服务源码副本与已入库构建产物,运行时截图目录停止跟踪。
  • Nuclei 引擎能力缓存与启动探测合并:合并冗余启动探测并缓存引擎能力,降低重复探测开销。

🐛 问题修复与加固

  • Nuclei 结果归属与失败可观测:Nuclei 结果保留目标端口,规避不同服务结果被合并;扫描进程失败时上报并清理异常后临时文件;测绘任务异常路径收敛终态判定与日志可观测性。
  • 任务同步链路加固:任务同步改用统一范围校验并支持域名与 IP 任务,派发前原子认领规避重复下发,下发失败时置为异常并保留错误原因;任务详情数据集合修正并支持全局数据源回退;任务 ID 非法格式直接返回未找到,避免退化为系统异常。
  • WIH 内置规则启用后生效:补齐启用后未生效的内置信息提取规则;外部资产文本按文字写入 XLSX,规避公式注入。
  • OSINT 派发与取消语义收敛:按任务类型区分派发入参校验并补齐天眼查派发目标字段,取消不再误报为崩溃;删除任务前补齐派发确认与取消语义。
  • MCP 自签部署可用性:自签部署下默认证书校验导致客户端不可用的问题收敛,客户端仅在同机回环时自动信任本地证书并保留链校验,远程部署经 ARL_CA_BUNDLE 指定;双副本客户端保持一致。
  • 前端交互收敛:企业全景测绘高级参数改为常驻平铺;危险操作默认值、统计口径与状态术语统一;资产删除目标集合修正。

🛡️ 安全与配置

  • 推送通道 TLS 校验启用:Telegram、钉钉、飞书、企业微信推送启用 TLS 证书校验,收敛中间人风险。
  • MongoDB 文件描述符上限提升:docker-compose.prod.yml 增设 nofile 上限至 65535,规避资产集合较多时并发查询触发官方警告;新增 arl_geoip 命名卷,存量持久化卷名保持不变。
  • 数据库平滑演进:增量迁移补齐并发配置缺失字段(仅补缺失、不覆盖已保存值)、企业测绘自动建组意图(仅在途任务)与失联建组状态自愈;资产范围索引增补 updating_icp_task_id;全部步骤经单步容错隔离,单项异常不阻断启动。
  • 文档口径收敛:API 调用约定下沉至 docs/API_GUIDE.md(中英双语),README 保留全局能力与快速入口;收敛与代码实现不符的能力宣称;资产组术语统一,社区协作模板补齐;证书与凭据忽略规则补齐并验证生效。

🚀 New Features

  • Enterprise panorama recon with backend-driven auto-grouping: Tianyancha and MIIT ICP lookups merged into enterprise panorama recon (panorama dual-track / official filing). Registered company names auto-match Tianyancha GIDs and upgrade to dual-track pivoting; auto-grouping intents (auto_group, auto_group_scan, scan policy and interval) are consumed by the backend scheduler instead of browser polling, with in-flight tasks backfilled by incremental migration; stale updating_icp_task_id markers are promoted or cleaned up according to task terminal states.
  • Asset-group association for ordinary scan tasks: the new recon task form offers an optional related asset group, with results synced to it on completion; fan-out is previewed honestly (one task per domain, merged task for IP/CIDR); the task list shows the linked group and sync state, with sync dispatched through a unified group selector.
  • Multi-select HTTP status filtering: site assets support combined multi-status filtering (status__in / status_code__in); invalid input fails fast to narrow query scope; coexisting legacy single-value conditions intersect, independent of parameter order.
  • Searchable system settings with credential overview: keyword search jumps to target tabs with highlighting; cross-tab credential states collapse into one summary line (folded by default); unsaved changes stay visible with jump links; the update page adds open-source repo and release links.
  • Aggregate status filters aligned with statistics: task-list metric cards and filters share backend semantics (active = non-terminal, abnormal = error + stop), keeping card counts equal to filtered results; phase states render uniformly as "进行中 · <phase>".

🛠️ Performance & Refactoring

  • Settings IA regroup with concurrency governance: settings converge into recon intel, dictionaries, scan & concurrency, notifications, and version & update; the GitHub monitor queue joins concurrency governance with peak-model recommendations; the three queue values are defined as Worker process counts, distinct from per-task blasting concurrency; omitted fields keep stored values on save to avoid silent overwrites; pool scaling retries against runtime process counts and reports observed convergence.
  • On-demand GeoIP with image decoupling: ip2region / GeoLite2 databases no longer ship in images; tools/fetch_geoip.py fetches and verifies them into the arl_geoip named volume at deploy time (GeoLite2 skipped with degraded service without MAXMIND_LICENSE_KEY); memory-swap observation tooling and per-queue OOM priorities added.
  • Config template / runtime config split: effective runtime config separated from committed templates; environment markers and credentials in test scripts injected via environment variables; microservice sources and committed build artifacts excluded from build context; runtime screenshot dir untracked.
  • Nuclei engine capability caching: redundant startup probes merged with cached engine capabilities, reducing repeated probe overhead.

🐛 Fixes & Hardening

  • Nuclei result attribution and failure visibility: results keep target ports so findings from different services are not merged; scan process failures are reported with temp files cleaned up; recon abnormal paths converge on terminal-state and log observability.
  • Task sync chain hardening: sync uses unified scope validation for domain and IP tasks, atomic claim before dispatch to avoid duplicate delivery, error state with cause preserved on dispatch failure; task-detail data namespaces corrected with global source fallback; malformed task IDs return not-found instead of degrading to system errors.
  • WIH builtin rules take effect: missing builtin extraction rules backfilled so enabling them works; external asset text written as literal XLSX cells to block formula injection.
  • OSINT dispatch/cancel semantics: dispatch params validated per task type with Tianyancha target field backfilled; cancellation no longer misreported as crashes; dispatch confirmation and cancellation semantics added before task deletion.
  • MCP self-signed deployment availability: default certificate verification breaking clients on self-signed deployments mitigated — local certificates auto-trusted on loopback only with chain verification kept, remote deployments via ARL_CA_BUNDLE; both client copies kept in sync.
  • Frontend interaction convergence: panorama recon advanced params always visible; dangerous-operation defaults, statistics semantics, and status terms unified; asset deletion target namespaces corrected.

🛡️ Security & Configuration

  • TLS verification for push channels: Telegram, DingTalk, Feishu, and WeCom push enable TLS certificate verification, narrowing MITM exposure.
  • MongoDB fd limit raised: docker-compose.prod.yml sets nofile to 65535 to avoid official warnings under concurrent queries over many collections; new arl_geoip named volume; existing persistent volume names unchanged.
  • Smooth DB evolution: incremental migrations backfill missing concurrency fields (fill-missing-only), in-flight recon auto-group intents, and stale grouping-state healing; scope index gains updating_icp_task_id; every step isolated by single-step fault tolerance so one failure never blocks startup.
  • Doc wording convergence: API conventions moved to docs/API_GUIDE.md (bilingual), README keeps global capabilities and quick entries; capability claims not matching implementation removed; asset-group terminology unified with community templates added; cert/credential ignore rules completed and verified.

What's Changed

  • fix(security): 为 Telegram 推送启用 TLS 证书校验 by @jikdarren in #69
  • perf(nuclei): 合并启动探测并缓存引擎能力 by @WeilianC in #63
  • fix(task): 修正任务详情查询和删除的数据集合 by @WeilianC in #64
  • fix(nuclei): 保留目标端口,避免不同服务的结果被合并 by @owl234 in #73
  • fix(nuclei): 报告扫描进程失败并清理异常后的临时文件 by @owl234 in #74
  • fix(wih): 补齐启用后未生效的内置信息提取规则 by @owl234 in #75

New Contributors

Full Changelog: v1.4.1...v1.4.2