Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .changelog/NEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
- **[issue-842] Turn a draft into a Creative Director project from the editor** — With live mode enabled in Writers Room, a new "CD Bridge" panel can read the prose around your cursor and propose a short film treatment — a logline, synopsis, an overall visual style, and a handful of filmable scenes (each with its own shot description and duration). Review it in the panel and send it into a brand-new Creative Director project in one click, then jump straight to it via an "Open in Creative Director" link on the work menu. It never overwrites an existing project, and it shares the same opt-in and daily suggestion budget as the inline continuation suggestions.
- **[issue-715] "Why am I seeing this?" chips across Chief of Staff, Goals, and the dashboard** — The same provenance chip now appears wherever PortOS shows you something it derived rather than something you set. The Chief of Staff's actionable-insights banner explains how its top item was derived — a live count of your records (pending approvals, blocked tasks) reads as data-backed, while a success-rate-driven suggestion reads as inferred. A goal's detail panel marks its urgency and activity-budget readings as modeled from your health timeline and linked activities — not numbers you entered — so a derived reading never reads as a fact you stated. On the dashboard, the Proactive Alerts card declares its alerts are inferred from trends in your goals, costs, and system metrics, while the Decision Log card declares its entries are read straight from the ledger of choices the Chief of Staff actually made. Tap any chip for how it was derived and what would change it.
- **[issue-952] Cinematic depth-of-field in City photo mode** — Photo mode now blurs the background for a cinematic look: whatever the current framing points at stays sharp while the rest of the city falls off softly, and each preset focuses on its own subject (the low-angle shot uses a shallower, more dramatic focus). It's on by default; toggle it with the "Depth" button in the photo bar or the D key, and your captured postcard matches what you see. Depth-of-field only runs while you're in photo mode, so the live dashboard is unaffected.
- **[issue-912] Regenerate gallery images to defeat SynthID watermarks** — Generated images gain a "Regenerate" action in the lightbox, next to "Clean", whenever a local FLUX image generator is installed. The existing Clean only strips embedded provenance metadata and can't touch SynthID — Google's invisible per-pixel watermark — which survives by design. Regenerate is the honest defeat path: it round-trips the picture through a local FLUX model at low denoise so composition holds but the watermark signal is overwritten by fresh sampling. It runs entirely on-device, is never applied automatically, and saves the result as a new variant beside the original (which is kept); the lightbox's variant switch lets you toggle between the original and the regenerated copy.
- **[issue-912] Regenerate gallery images to defeat SynthID watermarks** — Generated images gain a "Regenerate" action in the lightbox, next to "Clean", whenever a local FLUX image generator is installed. The existing Clean only strips embedded provenance metadata and can't touch SynthID — Google's invisible per-pixel watermark — which survives by design. Regenerate is the honest defeat path: it round-trips the picture through a local FLUX model so the watermark signal is overwritten by fresh sampling, while keeping the change to the image as small as possible — it uses no prompt and a low denoise, so the result is a near-faithful copy of the original rather than a reinterpretation. It runs entirely on-device, is never applied automatically, and saves the result as a new variant beside the original (which is kept); the lightbox's variant switch lets you toggle between them. Very large source images (e.g. 4K) are handled by regenerating at a model-friendly resolution and scaling the clean result back up to the original size, so a big image no longer stalls the pass.
- **[issue-877] City records its history for a future timeline scrubber** — The City now quietly snapshots its own state on a schedule (every 5 minutes by default), capturing which apps are online, how many agents are working, system health, and the other landmark readouts. These frames accumulate locally — never synced to peers — so an upcoming timeline view can let you scrub back and watch the city change over time. There's nothing to interact with yet; this is the recording groundwork. You can tune the capture interval, set how many frames to keep, or turn it off in settings.

## Changed
Expand Down
32 changes: 30 additions & 2 deletions server/services/imageGen/local.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
*/

import { spawn } from 'child_process';
import sharp from 'sharp';
import { writeFile, readFile, readdir, stat, unlink, rm, mkdtemp } from 'fs/promises';
import { existsSync, watch as fsWatch } from 'fs';
import { join, dirname, resolve as resolvePath, sep as PATH_SEP, basename } from 'path';
Expand Down Expand Up @@ -399,8 +400,12 @@ export function buildSidecarMeta({
};
}

export async function generateImage({ pythonPath, prompt, negativePrompt = '', modelId = 'dev', width = 1024, height = 1024, steps, guidance, seed, quantize = '8', loraFilenames = [], loraPaths = [], loraScales = [], initImagePath = null, initImageStrength = null, referenceImagePaths = [], referenceImageStrengths = [], jobId: providedJobId = null, cleanC2PA = false, denoise = false, regenOf = null }) {
if (!prompt?.trim()) throw new ServerError('Prompt is required', { status: 400, code: 'VALIDATION_ERROR' });
export async function generateImage({ pythonPath, prompt, negativePrompt = '', modelId = 'dev', width = 1024, height = 1024, steps, guidance, seed, quantize = '8', loraFilenames = [], loraPaths = [], loraScales = [], initImagePath = null, initImageStrength = null, referenceImagePaths = [], referenceImageStrengths = [], jobId: providedJobId = null, cleanC2PA = false, denoise = false, regenOf = null, upscaleTo = null }) {
// A regen pass (issue #912) deliberately runs with an EMPTY prompt for
// minimal-mutation, watermark-overwriting img2img — so the usual
// prompt-required guard is waived when `regenOf` is set (the init image is
// what conditions the render, not text).
if (!regenOf && !prompt?.trim()) throw new ServerError('Prompt is required', { status: 400, code: 'VALIDATION_ERROR' });
// Single-flight is enforced by the mediaJobQueue worker upstream. Direct
// callers that bypass the queue must not run two concurrent renders — the
// activeProcess handle below would be clobbered and cancel() would orphan
Expand Down Expand Up @@ -697,6 +702,29 @@ export async function generateImage({ pythonPath, prompt, negativePrompt = '', m
imageGenEvents.emit('failed', { generationId: jobId, error: userMessage || reason });
} else {
job.status = 'complete';
// Large-source regen (issue #912): the render ran at a clamped FLUX-sane
// resolution. Upscale the result back to the requested delivery size so
// the watermark-free copy matches the original's resolution. `meta.width/
// height` currently hold the render dims; record those as render* and
// promote the delivered dims so the gallery shows the real file size.
if (upscaleTo && Number(upscaleTo.width) > 0 && Number(upscaleTo.height) > 0
&& (Math.round(upscaleTo.width) !== meta.width || Math.round(upscaleTo.height) !== meta.height)) {
const targetW = Math.round(upscaleTo.width);
const targetH = Math.round(upscaleTo.height);
const resized = await sharp(outputPath)
.resize(targetW, targetH, { fit: 'fill', kernel: 'lanczos3' })
.png({ compressionLevel: 9 })
.toBuffer()
.catch((err) => { console.warn(`⚠️ Regen upscale failed for ${filename}: ${err?.message || err}`); return null; });
if (resized) {
await writeFile(outputPath, resized).catch(() => {});
meta.renderWidth = meta.width;
meta.renderHeight = meta.height;
meta.width = targetW;
meta.height = targetH;
console.log(`🔍 Upscaled regen [${jobId.slice(0, 8)}] ${meta.renderWidth}x${meta.renderHeight} → ${targetW}x${targetH}`);
}
}
// Sidecar: persist a metadata record next to the PNG so the gallery
// and Remix flow can recover prompt/seed/steps even if mflux's own
// --metadata sidecar lives at a slightly different filename shape.
Expand Down
104 changes: 80 additions & 24 deletions server/services/imageGen/regen.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,59 @@ import { IMAGE_GEN_MODE } from './modes.js';

const IS_WIN = process.platform === 'win32';

// img2img denoise strength: lower = closer to the source (less watermark
// overwrite), higher = more of the image resampled. 0.4 is the issue's
// recommended midpoint — enough fresh sampling to overwrite the per-pixel
// signal while composition holds.
export const DEFAULT_REGEN_STRENGTH = 0.4;
export const REGEN_STRENGTH_MIN = 0.2;
// img2img denoise strength: lower = closer to the source (less mutation),
// higher = more of the image resampled. With the empty-prompt minimal-mutation
// default, the VAE round-trip already overwrites most of the per-pixel SynthID
// signal, so a low strength is enough — 0.25 is the provisional default and the
// floor is 0.1 so a user can sweep down (via the API `strength` param) to the
// minimum their SynthID detector still clears. Tune the default once that floor
// is known.
export const DEFAULT_REGEN_STRENGTH = 0.25;
export const REGEN_STRENGTH_MIN = 0.1;
export const REGEN_STRENGTH_MAX = 0.6;

// FLUX runs self-attention over latent tokens (~pixels/256) and the cost is
// O(tokens²), so render resolution can't track the source for large images:
// a 12.6 MP (4096×3072) codex render is ~60× the attention compute of a 1.5 MP
// one and will stall or OOM even on a big-memory box, well before producing a
// usable result. Cap the *render* resolution to a FLUX-sane budget; the output
// is then upscaled back to the source's exact dimensions (see generateImage's
// `upscaleTo`). Env-tunable for high-memory machines that want to push it.
export const DEFAULT_MAX_REGEN_MEGAPIXELS = (() => {
const n = Number(process.env.PORTOS_REGEN_MAX_MP);
return Number.isFinite(n) && n > 0 ? n : 2.0;
})();

// FLUX latents are /8 spatial and the transformer patchifies 2×2, so render
// dimensions must be multiples of 16. Round DOWN to the nearest multiple (so
// the megapixel budget stays a hard ceiling — rounding up could nudge a
// budget-fitted image back over and reintroduce the OOM risk), floored at 16 so
// a tiny input can't collapse to 0.
const floor16 = (n) => Math.max(16, Math.floor(n / 16) * 16);

/**
* Resolve the render dimensions for a regen pass: clamp the source to a
* FLUX-sane megapixel budget (aspect-preserved) and round to multiples of 16.
* Pure. Returns `{ width, height, scaled }` where `scaled` is true whenever the
* render dims differ from the source (either because it was downscaled to fit
* the budget OR because the source wasn't already /16) — i.e. whenever the
* caller must upscale the result back to the source's exact dimensions.
*/
export function clampRegenDimensions(srcWidth, srcHeight, maxMegapixels = DEFAULT_MAX_REGEN_MEGAPIXELS) {
const w = Math.round(Number(srcWidth));
const h = Math.round(Number(srcHeight));
if (!(w > 0) || !(h > 0)) {
// Defensive fallback for a missing/garbage sidecar — render at the FLUX
// native square and don't claim a scale-back.
return { width: 1024, height: 1024, scaled: false };
}
const budgetPx = Math.max(1, maxMegapixels) * 1_000_000;
const scale = w * h > budgetPx ? Math.sqrt(budgetPx / (w * h)) : 1;
const rw = floor16(w * scale);
const rh = floor16(h * scale);
return { width: rw, height: rh, scaled: rw !== w || rh !== h };
}

// FLUX.2 + the diffusers-family runners (Z-Image / ERNIE / HiDream / Qwen) all
// share the FLUX.2 venv and implement img2img via `--image-path`.
export const modelUsesFluxVenv = (model) => isFlux2(model) || usesDiffusersRunner(model);
Expand Down Expand Up @@ -135,19 +180,26 @@ export async function readImageDimensions(absPath) {
return null;
}

// Pure: assemble the mediaJobQueue params for a regen render. Reuses the source
// image's own prompt (falling back to a generic quality prompt when the source
// has no recorded prompt — e.g. an upload or an already-cleaned copy) so the
// round-trip preserves intent. `regenOf` is what stamps the sidecar lineage in
// `generateImage`. The validated `strength` is the img2img denoise; `steps`
// (optional) lets the caller pin a low count, otherwise the model default is
// used. Actual dimensions match the source.
export function buildRegenParams({ filename, sourceAbsPath, sourceMeta = {}, sourceDims = null, model, pythonPath, strength, steps }) {
const dims = sourceDims
|| (sourceMeta.width && sourceMeta.height ? { width: sourceMeta.width, height: sourceMeta.height } : null);
const prompt = typeof sourceMeta.prompt === 'string' && sourceMeta.prompt.trim()
? sourceMeta.prompt
: 'high quality, highly detailed';
// Pure: assemble the mediaJobQueue params for a regen render. `regenOf` is what
// stamps the sidecar lineage in `generateImage`. The validated `strength` is the
// img2img denoise; `steps` (optional) pins a low count, else the model default.
//
// Minimal-mutation default: NO prompt and NO negative prompt. With img2img a
// text prompt steers the output toward described content; an empty prompt makes
// the pass a near-pure VAE round-trip + `strength` worth of resample — which is
// what overwrites the per-pixel SynthID signal with the LEAST visible change to
// the image. A caller that wants a creative re-roll instead can pass an explicit
// `promptOverride` (then the source's negative prompt is carried along too).
//
// Render dimensions are clamped to a FLUX-sane megapixel budget (large codex
// renders — up to 12.6 MP — would otherwise stall/OOM the attention pass). When
// clamping changes the dims, `upscaleTo` carries the source's exact dimensions
// so `generateImage` resizes the result back up, delivering a watermark-free
// copy at the original resolution.
export function buildRegenParams({ filename, sourceAbsPath, sourceMeta = {}, sourceDims = null, model, pythonPath, strength, steps, promptOverride }) {
const src = sourceDims
|| (sourceMeta.width && sourceMeta.height ? { width: Math.round(sourceMeta.width), height: Math.round(sourceMeta.height) } : null);
const hasPrompt = typeof promptOverride === 'string' && promptOverride.trim();
// Anchor the variant-grouping lineage at the ROOT original, not the clicked
// image. computeImageVariantGroup groups siblings under a single original
// (an item with no `cleanedFrom`); regenerating a cleaned/regenerated variant
Expand All @@ -161,15 +213,19 @@ export function buildRegenParams({ filename, sourceAbsPath, sourceMeta = {}, sou
mode: IMAGE_GEN_MODE.LOCAL,
pythonPath,
modelId: model.id,
prompt,
negativePrompt: typeof sourceMeta.negativePrompt === 'string' ? sourceMeta.negativePrompt : '',
prompt: hasPrompt ? promptOverride : '',
negativePrompt: hasPrompt && typeof sourceMeta.negativePrompt === 'string' ? sourceMeta.negativePrompt : '',
initImagePath: sourceAbsPath,
initImageStrength: strength,
regenOf: groupRoot,
};
if (dims) {
params.width = dims.width;
params.height = dims.height;
if (src) {
const render = clampRegenDimensions(src.width, src.height);
params.width = render.width;
params.height = render.height;
// Clamped or /16-rounded → deliver the cleaned copy at the source's exact
// resolution by upscaling the render back up.
if (render.scaled) params.upscaleTo = { width: src.width, height: src.height };
}
if (steps != null) params.steps = steps;
return params;
Expand Down
Loading
Loading