Repository navigation
Releases
v0.1.5
Fixed
- JavaScript analysis: a script whose own code contained text shaped like the scanner's internal string placeholders (
__S7__) stopped the inventory with anIndexError(found by ClusterFuzzLite). Such text is now kept apart from the placeholders
Upgrading from 0.1.4 needs no changes. Docs: https://sp2react.readthedocs.io
v0.1.4
OpenSSF Best Practices silver and baseline groundwork, and the fixes found by a threat model of sp2react. Each fix has a test in tests/test_threat_model.py or tests/test_sandbox.py that fails on 0.1.3.
Security
- The generated app's back-end tests and loader run in a sandbox (
pipeline.sandbox, defaultauto). They import custom handlers written by agents, which until now ran as ordinary processes with the user's file and network access.containerruns them in a throwaway Docker or Podman container with no network (or only the database host), a read-only file system except the folder the step writes, no capabilities, a non-root user and none of the host's environment;guard(used byautowhen no container runtime answers) runs them under an audit hook that refuses writes outside that folder, new processes, other network hosts andctypes;offkeeps plain processes. Setpipeline.sandbox_imageto use a check image from your registry - NTLM is an explicit, TLS-only choice.
sharepoint.authhas no default any more (NTLMv2 needs MD4 and HMAC-MD5 by protocol), and NTLM is never answered over plain HTTP (the site URL or any later request, so a redirect cannot downgrade it) unlesssharepoint.allow_ntlm_over_http: true. The generated app's SharePoint data source does the same (SP_AUTHrequired,SP_ALLOW_NTLM_OVER_HTTP=1). Upgrade note: a config withoutsharepoint.authnow fails with "Field required"; addauth: ntlm(orcookie,bearer,none). Configs written bysp2react initalready have it extract: a page or asset URL from the site with..segments (for example<script src="/sites/x/../../../y.js">in a page) was written relative to the workspace, so a page editor could make sp2react write files outside it. Every file the extract writes (page source, web part, wiki and rendered sidecars, assets, list folders) is now checked (.., absolute and drive paths, symlinks) and a refused one is listed as an extraction error, before it is downloadedmigrate: attachment names and item ids from the server are checked the same way before the attachment is downloaded and written to the export folder- The generated app's builds and tests no longer receive sp2react's own credentials: the variables named in
sharepoint.*_env,llm.api_key_envanddatabase.url_envare removed from their environment - Spec screen ids must be plain names (letters, digits,
_,-,., at most 128 characters), since they become folder and file names; generated ids longer than 100 characters are shortened and keep a hash of the full name
Fixed
- Generated app:
startswith,endswithandsubstringofin$filterwith a number instead of quoted text (endswith(Title,0)) failed with a server error; they now answer 400 like any other bad filter (found by ClusterFuzzLite). WithDATA_SOURCE=database,%and_in their text are matched literally, as SharePoint and the other data sources do, instead of acting as SQL wildcards. Regenerating an existing app picks this up (odata.py,sql_repo.py,sp_repo.py; files you changed are kept)
Added
GOVERNANCE.md,CODE_OF_CONDUCT.md,ROADMAP.md, a VEX document (vex.openvex.json), and in the documentation a security assurance case (threat model), a dependency and scanning policy, testing and releasing pages- Pull requests: a DCO check (every commit signed off) and GitHub dependency review
- CI: statement coverage must stay at 80% or more (about 86% now); a job builds the package twice and checks the files are identical, and releases are built the same way (
SOURCE_DATE_EPOCH,ci/reproducible_sdist.py); the vulnerability audit also checks versions pinned only for other Python versions (ci/all_pins.py); a job runs the generated tests in a real container; Dependabot watches the fuzzing and check images
Changed
mypy --strictandruffwith the Bandit security rules (S), comprehensions,PIEandRUFare clean and required- CI tool locks are compiled for the Python versions their jobs run (build 3.11, e2e 3.12)
- Documentation reorganised into Getting started, User guide, Reference, Development and Release notes; installation covers offline installs and verifying a download's signature, and the JFrog Curation notes no longer list the dependencies removed in 0.1.3
sp2react 0.1.3
Fixes found in a review of 0.1.2, no runtime dependencies, and signed, verifiable releases. Each fix has a test (tests/test_hardening.py, tests/test_fidelity.py, tests/test_stdlib.py) that fails on 0.1.2.
Security
- Generated app: the catch-all route that serves the front end could return any file on the server (
/..%2fbackend%2f.env,/%2Fetc%2Fhostname). It now serves only files insideFRONTEND_DIST - Generated app:
AUTH_MODEnow defaults toheaderand fails closed (401 without the identity header).devmust be set explicitly, accepts requests from the same machine only, and no longer takes the user from the?as=query parameter. Upgrade note: setAUTH_MODE=devwhen you run the app locally (the generated.env.examplealready does) - Generated app:
/api/screensand/api/specs/{id}now need a signed-in user - Generated app: deleting an item and adding or removing attachments now follow the same role and lock rules as editing it (a locked item answered 204 to
DELETE); attachment names are checked (no/,\,.., leading dots) - Generated app: legacy Content Editor / Script Editor markup is rebuilt from an allow-list (
runtime/sanitize.ts) instead of three regular expressions that<svg/onload>, unquoted and entity-encodedjavascript:links got past - Generated app (
DATA_SOURCE=sharepoint): a user is matched by exact login, not by the first login that contains the text - Agents: file paths proposed by the page, handler and fix agents must be plain relative paths inside the screen's folder; a path with
..could overwritefrontend/vite.config.tsor write outside the output folder. Custom handler names must be identifiers
Fixed
- Columns whose internal name starts with
_or a digit (2nd Approveris_x0032_nd_x0020_Approver,OData__x0032_nd_x0020_Approverin REST) were migrated empty while the reconciliation said every list matched. Fields now record their REST name (rest_name), migration and the SharePoint data source read it, and specs from 0.1.2 work through the same naming rule. Reconciliation now fails when a field is in no exported item - SharePoint throttling (429, 503) and dropped connections are retried with
Retry-After(sharepoint.retries,sharepoint.max_retry_wait). If a list's fields or views, a page's source or web parts, or the SitePages listing still cannot be read,extractfails instead of passing with an empty screen - List pages: the "Title (linked to item)" column (
LinkTitle) showed an empty column headed "LinkTitle"; it is now the Title column linked to the item. The page opens the list's default view with that view's sort order, instead of sorting byID desc - Two pages with the same file name in different libraries (
SitePages/Home.aspx,Pages/Home.aspx) no longer share one screen id: both get their folder in the id - Regenerating the app overwrote files the team had changed, such as
backend/app/auth.py. A changed file is now kept, and the new version is written next to it as<file>.sp2react-new. Apps generated by 0.1.0 to 0.1.2 are recognised, so their unchanged files are still updated - Multi-value lookup columns were edited as a single choice and an edit kept one value; they are now a multi-select list and every value is saved
- Date-and-time columns showed the UTC clock as local time on the edit form, and saved values had no zone. They are now edited in the browser's time zone and stored in UTC (
...Z), as SharePoint returns them; values with an offset are converted by the API - Hyperlink columns lost their description; they are now
{url, description}everywhere (form, list, migration, reconciliation; adescription_columnin the database mapping). Managed metadata was stored as the raw REST object ([object Object]); terms are now{label, term_id}with the label read from the field's hidden note column, shown read-only - List views ignored the view's filter and showed only the first page. The
<Where>of the view is translated to the API filter (comparisons, And/Or, In, Contains, BeginsWith, IsNull,<UserID />,<Today />), a filter that cannot be translated is reported on the page, and the list pages through all items - The list API returned every row when
$topwas not given; like SharePoint it now answers 100 rows by default and at most 5,000 - Wiki pages: their text lives in the page's
WikiFieldcolumn, which was never read, so a wiki page converted to an empty screen. It is now extracted and treated like Content Editor text - A list whose URL name contains a space failed the spec gate (
endpoint path ... must contain only path characters); the API client now also encodes list names in URLs
Added
- Supply chain: hash-locked CI tool sets (
ci/requirements-*.txt), azero-depsjob that installs and runs the package with nothing else installed, weeklypip-auditof every tool set and of the generated app's back end plusnpm auditof its front end, ClusterFuzzLite fuzzing (tests/fuzz: config YAML, spec validation, ASPX/JavaScript analysis, NTLM challenges, the generated API's$filterparser), and an end-to-end workflow that runsdata_source: databaseagainst real MySQL 8.4 and SQL Server 2022 containers - Releases: published from GitHub releases only, with a check that the wheel declares no runtime dependencies, a CycloneDX SBOM, PyPI attestations, and Sigstore-signed wheel, sdist and SBOM on the GitHub release;
verify-release.ymlchecks the signatures and that PyPI serves the same files - Documentation site configuration for Read the Docs (Sphinx + MyST, warnings are errors),
CONTRIBUTING.md,THIRD_PARTY_NOTICES.md;ruffandmypyclean
Changed
- No runtime dependencies.
requests,requests-ntlm,pydanticandPyYAMLare replaced by standard-library code in the package: an HTTP client (transport.py: TLS verification or a CA bundle, proxies fromHTTPS_PROXY/NO_PROXY, redirects checked hop by hop by the network policy), NTLMv2 Windows sign-in with channel binding on HTTPS (ntlm.py, checked against the MS-NLMP test vectors), validated records (model.py) and a YAML reader for the config file (yamlish.py, which refuses anchors, tags, tabs and duplicate keys with a line number).pip install sp2reactinstalls one wheel. Thentlmextra remains, empty - Docs: workflows, event receivers, InfoPath and Power Apps are listed as not detected (they were described as reported); remaining known gaps are listed in 12. Coverage and limitations
Assets: wheel, sdist and a CycloneDX SBOM, each with a Sigstore signature (*.sigstore.json), checked after publishing by verify-release.yml.
v0.1.2
Security and supply-chain release. See CHANGELOG.md.
- Generated app front-end template: Vite 6.4.4+, react-router-dom 7.18.4+, @vitejs/plugin-react 4.7 (
npm audit: 0 vulnerabilities; generated app builds and passes the 29-step browser walkthrough) - Minimum versions past every known advisory: requests>=2.33.0, pymysql>=1.1.1, pytest>=9.0.3
- Dependabot and CodeQL code scanning
setup.pyshim
v0.1.1
Documentation release. See CHANGELOG.md.
- End-to-end documentation in docs/: installation (Windows, JFrog), configuration, Bridger, SQL Server / MySQL database mode, walkthrough, command reference, generated app and deployment, security, troubleshooting, Python API, limitations
sp2react initand the generated.env.exampleshow SQL Server connection URLs anddatabase.schema_name- Generated API is started with
uvicorn --env-file .env - All GitHub Actions pinned to commit SHAs
v0.1.0
First public release. See CHANGELOG.md.
- Ten-stage gated pipeline: read-only SharePoint extraction, jQuery/SPServices inventory, spec with coverage gate, conversion, review, test and fix
- AI agents on Claude through GitHub Copilot Bridger, each output validated before it is accepted
- Generated React + FastAPI app on your existing MySQL / SQL Server tables via a reviewed mapping (
sp2react map-db) - No business data over HTTP in database mode; every request checked against the configured hosts