Wordflow 0.7.18

Wordflow 0.7.18 makes Topic Modelling give the same topics every time for the same text, settings and seed, makes the end of a large Topic Modelling run quicker and lighter on memory, and lets a shared server show its users a notice. The 0.7.17, 0.7.16 and 0.7.15 notes follow, for anyone updating from an earlier version.

New in 0.7.18

• The same topics every time: running Topic Modelling again on the same text with the same settings and seed now gives exactly the same topics, on any computer or server. Before, a computer with several processors could give a slightly different set of topics on each run of a corpus with more than about 8,000 segments. Because of this fix, a run made with 0.7.18 can differ a little from the same run made with an earlier version.
• The end of Topic Modelling is quick and light on large corpora: the last step ("finding each topic's words") could spend many seconds on one processor and several gigabytes of memory preparing the results of a large corpus. It now takes a fraction of a second and a fraction of the memory. The topics and their sizes are unchanged.
• The Search term example is lighter: "e.g. house, pre, gov" in the Topic Modelling search box is now grey, like other examples, so it doesn't look like typed text.
• A notice from the server you use (shared servers): a shared Wordflow server, such as Wordflow online, can now show a short notice at the top of the workspace, for example where your files and Projects are stored and who can access them. Choose Got it to hide it; it shows again if the text changes. Server operators: see SERVER_NOTICE_FILE in the hosted deployment runbook.

What came in 0.7.17

• The last Topic Modelling step is much faster: "finding each topic's words" could run for minutes on one processor with a large corpus. It now takes seconds and uses all the processors it may. The topics, their words and the map are exactly as before.
• Topic Modelling Display options, beside the bubble chart on wide screens: the Result settings are now called Display options, because they change how the result is shown. When the results panel is wide, they sit in a compact column beside the bubble chart, one option per row with the colour legend, so they no longer take height between the chart and the topic list. Drag the divider between them to change the chart's width (double-click to go back), or click the panel icon at the right of the heading to move the options below the chart, where they take a compact row with each label beside its control. Click Display options to fold them to one summary line on a small screen. Find topics is now Search term, Per document is Topics per doc, and Words is Words per topic. The Topics slider shows its smallest and largest values under the bar. The tab remembers these choices on this computer.
• Search several terms at once in Topic Modelling: the Search term now takes several words separated by spaces, commas or other punctuation, and finds topics with any of them. Each term's matching words are bold in their own colour in the topic list and in a bubble's word cloud, so with "chil, home" child and children share one colour and home and homeless another. The box shows an example: "e.g. house, pre, gov".
• The Topics slider starts at two topics for new runs, because a single topic is the whole corpus. Results from earlier runs keep their range.

What came in 0.7.16

• Topic Modelling is faster on a Mac with Apple silicon: Wordflow now reads the text on the Mac's graphics processor. On a 26,000-article news corpus a first run took about 4 minutes instead of 12 on an M5 Pro, and a run on text it has already read about 1 minute. The first run after installing or updating Wordflow prepares the model for the graphics processor once, which can add up to a minute. Windows, Linux and shared servers still use the processor; they are slightly faster too.
• A Multilingual model for Chinese, Japanese and Korean: Topic Modelling has a new Model setting. English stays the default. Multilingual understands Chinese, Japanese, Korean and about 50 other languages, and reads up to 128 tokens per segment. The English model could read such text but grouped it by shared characters, so topics formed around common words rather than subjects. When a Data Block's tokeniser is a Chinese, Japanese or Korean one (set in Frequency Analysis), or its text looks like a language other than English, Multilingual is chosen for you and a note says why; you can switch back to English.
• A suggestion for Max tokens: with Suggestions on, Wordflow looks at the first 200 documents and, when at least 9 in 10 of the sentences, paragraphs or texts it keeps whole are short, suggests a smaller Max tokens (for example "Your sentences are short: 64 tokens per segment is plenty") with a button to apply it. It never suggests a larger one.
• Max tokens is a list of four sizes: 32, 64, 128 or 256 tokens per segment (up to 128 with Multilingual). A run saved with another value opens at the size below it.
• Topic Modelling on a remote GPU: if you have access to a GPU server for Topic Modelling, for example through your university or a shared Wordflow server, a Remote GPU tick at the right end of the Topic Modelling settings has it read the text on its graphics processor. That takes seconds where a computer can take many minutes on a large corpus. On the desktop app, add a server under Settings → Servers with its address and API key; Wordflow checks the key and lists the server's models for you to choose. The tick appears only once a server is set up. Most people don't have one, and Topic Modelling works as before without it. With Suggestions on, a long run on a computer without a GPU suggests adding one, if you have access.
• Concordance shows where each match is: clicking a row opens the document scrolled to the match, with the match in bold and the words around it highlighted; the highlight had been invisible since 0.7.3. In the Dispersion view the window is Match Details: Previous match and Next match step through a document's matches, in the colours of the chart's legend.
• Annotation lists only models that can annotate: when a provider, such as a LiteLLM gateway, also serves embedding or OCR models, those are left out of the Model list.
• One Servers tab in Settings: the AI tab is now Servers, with Generative AI for Annotation and Topic Modelling for remote GPUs.
• A default model for each AI provider: each Annotation provider in Settings → Servers can have a Default model, which new Annotation tabs start with. It can stay empty. A new tab in a Project starts with the model last chosen or run with that provider in the Project, on this computer; that never changes the default in Settings.

Good to know (0.7.16)

• Sign-in settings renamed (hosted servers): the OIDC sign-in settings are now OIDC_CLIENT_ID, OIDC_CLIENT_SECRET, OIDC_ISSUER, OIDC_SCOPES and OIDC_REDIRECT_URI, since they work with any OIDC provider, not only CILogon. The earlier CILOGON_* names still work, so existing servers need no change; when both are set, the OIDC_* one is used.
• The first Topic Modelling run reads the text again: Wordflow keeps what it has read so that later runs are quick. After this update it reads each text once more; saved Projects and results are unchanged.
• Topics on a Mac can differ slightly from another computer's: the graphics processor computes with very slightly different rounding, so the same text and settings can give a few different topics on a Mac and on a server.
• Servers and keys offered by a shared server (hosted servers): a shared Wordflow can offer AI providers for Annotation and Topic Modelling servers to everyone who signs in, from a configuration file (SERVER_AI_PROVIDERS_FILE); the keys stay on the server and people choose only the model. A server image is built for such deployments. See the server AI providers and server image runbooks.
• Shared servers run under any account (hosted servers): the server image runs under whatever user and group the cluster gives it, keeps its prefetched models read-only and puts new downloads in the data folder. It stops at start-up, with one line naming the folder, the account and the reason, when its data folder can't be used or holds data written by another account. See "Running as another user" in the server image runbook.
• Multilingual runs need 0.7.16: a Project opened in 0.7.15 or earlier shows a Topic Modelling run that used the Multilingual model as unavailable; the rest of the Project opens as usual. Runs with the English model open in every 0.7 version.

What came in 0.7.15

• Large Projects import and export: importing a Project could fail with "Project manifest exceeds the configured limit" once it held a number of saved Analyses, and Projects over about 500 MB could not be imported or over 2 GB exported. Projects now import and export at any size; only free disk space (and, on a shared server, your storage quota) limits them. Importing also no longer needs over 10 GB of free disk for a small Project.
• Add to Project without the document column works: in Concordance and Quotation, Add to Project with the document column unticked showed a success message but added nothing. It now adds the Data Block, and if adding a Data Block ever fails, Wordflow says so instead of only showing it in Tasks.
• Files keep their file type in the desktop app: renaming a file in the Save window no longer drops its .csv, .xlsx or .zip ending, so Windows and Excel recognise it.
• Clear messages when a desktop download fails: the desktop app shows the reason, as a browser does, instead of an internal code such as backend_download_http_400.
• Excel's cell limit explained: when text is too long for an Excel cell (32,767 characters), Wordflow names the column, suggests what to do beside Format, and links to a new help section. That section explains that Excel also breaks a CSV with such text, and which programs open it correctly (LibreOffice Calc, Numbers).
• The update window stays in view on Windows: when the check at start finds a new version, the update window no longer hides behind the main window.
• Annotation runs no longer stall for 5 minutes now and then on hosted LLM gateways: each request to an AI provider now closes its connection when it ends. Before, about 1 in 80 requests could get no answer and wait for the full 5-minute timeout before it was tried again. A connection that can't be opened now fails after 15 seconds.
• Annotation no longer splits batches when a model wraps its answer in a code block: some models put their answer in a Markdown code block, or give one label too few, and Wordflow then split the batch again and again, sometimes down to one row per request (one run needed 6 times as many requests). Wordflow now reads an answer inside a code block, and asks Custom providers that support it to return exactly one label per row. Answers with the wrong number of labels or an unknown code are still refused.
• Task context in the codebook: the Edit codebook window has an optional Task context that says what the texts are and what the codes are for. It is saved with the codebook and sent to the AI with every request, after the prompt. Projects that don't use it still open in 0.7.14.
• Model descriptions in the Annotation model list: when a provider describes its models (OpenRouter, Google, or a LiteLLM gateway with descriptions set), each model in the Model list has a one-line description under its name.
• No "(recommended)" after the document column in the Add to Project and download windows. The document column still starts ticked.
• Wider columns in the Data Loader preview: drag a column header's edge to widen it and read more; long texts no longer show as blank on a Mac.
• Sign-in with other OIDC providers (hosted servers): a server can now sign in through providers such as a Keycloak realm, with the new CILOGON_SCOPES setting and an issuer that includes a path. Existing servers are unchanged.
• "Sign in with SSO" on hosted servers: the sign-in button no longer names CILogon, since the same sign-in can be CILogon (AAF) or another single sign-on provider. It reads Sign in with SSO and takes you to whichever one the server uses.