Releases: OpenRefine/OpenRefine
OpenRefine 3.9-beta1
This is the first beta release of the 3.9 series. Please backup your workspace directory before installing and report any problems that you encounter.
New features
- Import files using drag and drop. Closes #6250 by @Ahmed-Elgamel in #6438
- Ability to exclude candidates from a cluster. Closes #105 by @zyadtaha in #6448
- Support for user defined clustering distances and keyers by @zyadtaha in #6612
- The Wikibase editing operation reports editing results in the grid by @wetneb in #6555
- A dialog to warn of history entry deletion was added by @wetneb in #6659
- Add support for adding rows to a project by @steve-kasica in #6461
- Large media files can be uploaded to Wikimedia Commons and other Wikibases (#6510) by @sebastian-berlin-wmse in #6531
- A new GREL function (
levensthteinDistance
) to calculate the edit distance between two strings by @zyadtaha in #6755 - A new GREL
zip
function was introduced by @sunilnatraj in #6739 - Ability to handle dates in Jython expressions. Fixes #692 by @tfmorris in #6862
- Alphabetical/chronological sort for multi-file import by @elebitzero in #6844
- Resizeable columns (known to work imperfectly with the Edge browser, see #6932) by @wetneb in #6563
- Option to remove Blank Columns upon project creation by @wsmmxmm in #4757
- An extension mangement page by @Abbe98 in #7012
- Allow uploading new media file versions by @sebastian-berlin-wmse in #6967
- New "remove duplicate rows" operation by @sunilnatraj in #7040
- Search filter for Open Projects page by @Babi-B in #4690
Enhancements
- Ignore JSON comments by default when importing JSON files by @zyadtaha in #6429
- Add module names to About page. Closes #5546 by @tfmorris in #6421
- New look and feel for the wikibase warnings. Closes #5702 by @t8210103 in #6535
- Switch release notification to openrefine.org and display other events by @wetneb in #6511
- The OpenRefine icon is compliant with new MacOS guidelines by @teolemon in #6592
- Improve localization of row/record count bar by @wetneb in #6648
- Replace PNG logo by SVG equivalent by @xinluz6 in #6620
- Preserve pagination after actions which trigger a grid update by @wetneb in #6546
- Limit Wikibase description length to 250 characters by @sebastian-berlin-wmse in #6853
- Avoid creating identical facets in reconciliation and Wikibase editing operations by @wetneb in #6865
- wikibase: More precise editing error messages by @wetneb in #6872
- wikibase: Render property names in issue reports #6768 by @sunilnatraj in #6895
- Move error reporting in apply-operations dialog away from alert into dialog by @wetneb in #6904
- Make the client link clickable in terminals by @Abbe98 in #6976
- Update refine.ini for
REFINE_INTERFACE
by @antoine2711 in #6983 - Adds the reconciliation service logo to column header and while schema building by @ayushrai206 in #6156
- Set a delay between opening submenu and hover. Fixes #6672 by @Shubham-M-Rathod in #6965
- Mark tables used for layout as presentational by @Abbe98 in #7003
- Wikibase: proxy manifest requests by @Abbe98 in #6136
- wikibase: Only ignore file upload warnings for matched cells by @wetneb in #7031
- Add escape() function options to escape URL path and fragment by @sunilnatraj in #6715
- wikibase: Improve recon name for created entities by @sunilnatraj in #7053
Bug fixes
- The replace dialog works when the strings involved contain double quotes (Fixes #6367) by @surajbora59 in #6374
- Improve error reporting when parsing incomplete GREL expressions by @surajbora59 in #6368
- Better report importer initialization errors. Fixes #6058 by @tfmorris in #6370
- Correct the reconciliation documentation link display type by @Abbe98 in #6184
- Fix recon dialog suggested types radio button by @Redeem-Grimm-Satoshi in #6397
- Fix swapped button labels in wikibase schema editor by @Redeem-Grimm-Satoshi in #6415
- Fix inconsistent Extract/Apply button location by @Redeem-Grimm-Satoshi in #6450
- Support localized numbers in import settings. Fixes #6351 by @Xiayucheng1212 in #6358
- Fix encoding guesser for non-UTF-8 BOM-based files. Fixes #6595 by @tfmorris in #6596
- recon: Add back missing 'search for match' button by @wetneb in #6600
- grel: Fix NullPointerException in forEachIndex by @tfmorris in #6606
- Make join handling of empty strings consistent even if they are first in array by @tfmorris in #6605
- wikibase: Better handling of the 'badtags' error by @wetneb in #6552
- Fix join() of arrays containing nulls. Fixes #6683 by @tfmorris in #6684
- Solve Excel importing from URL problem. Fixes #6418 by @Kurocifer in #6537
- Fix type of error message returned by the Filter GREL control by @sunil-atheer in #6701
- grel: Integer divide by zero returning null instead of infinity by @Kurocifer in #6708
- Increase TSV/CSV column limit to 32K (from 512). Fixes #6723 by @tfmorris in #6757
- The saving of text files on MacOS Sonoma was fixed. Fixes #6549 by @tfmorris in #6749
- fix: Ensure thread safety of Wikibase date parsing by @wetneb in #6779
- Fix cursor appearance when hovering tabs. Closes #6764 by @GittyHarsha in #6811
- Fix misplaced edit button for empty cells. Closes #5989 by @GittyHarsha in #6812
- File type detection fails when multiple files are imported at once by @sunilnatraj in #6788
- Fix Uncaught RangeError when using Cell Edit button to change String to Date data type. by @GittyHarsha in #6823
- wikibase: Fix NPE when empty edits are scheduled by @wetneb in #6776
- Turn on line separator detection for CSV/TSV import. Fixes #6691 by @tfmorris in #6790
- add documentation for
toTitleCase()
second argument (delimiters) by @akashinde in #6846 - Add error reporting to JSON preview parse. Fixes #6454 by @tfmorris in #6912
- Add missing backslash escaping in duplicate facet expression by @wetneb in #6910
- The
toString
GREL function correctly returns an error when formatting template and arguments are inconsistent. Fixes #6943 by @tfmorris in #6946 - ODS file incorrectly recognized as XLS #6877 by @sunilnatraj in #6880
- Fix resultSummary div in Clustering dialog has unnecessary squishyness by @himanshubhatt0512 in #6845
- Fix an inadequate rate-limiting when editing Wikimedia Commons by @sunilnatraj in #6969
- Really fix QuickStatements export this time by @wetneb in #7016
- Fix opening of CSV files with many columns. Closes issue #7038 by @CAMB-dev in #7041
- Minor fixes to translatable files by @SannitaSSJ in #7075
- fix: Deser...
OpenRefine 3.8.7
This release fixes an issue (#6977) which prevents exporting Wikidata QuickStatements in OpenRefine 3.8.5 and 3.8.4, as well as an issue (#6941) which prevents login to Wikidata with OpenRefine 3.8.4. This release includes the same collection of important vulnerability fixes as 3.8.4. We encourage users to upgrade swiftly. It also fixes an issue (#7001) in 3.8.6 where the update banner was always displayed. It is otherwise identical to 3.8.6.
To continue using the Google Drive and Google Sheets integration, users need to obtain their own application credentials from the Google API Console.
Note: the vulnerability fixes were originally released as 3.8.3 but that version is dysfunctional due to human errors in the release process. The description of the vulnerabilities is included again here for visibility.
Vulnerabilities in OpenRefine
- PreviewExpressionCommand, which is eval, lacks protection against cross-site request forgery (CSRF).
CVE-2024-47879, GHSA-3jm4-c6qf-jrh3. Reported by @wandernauta, fix by @wetneb. - Reflected cross-site scripting vulnerability (XSS) from POST request in ExportRowsCommand. Severity: high. CVE-2024-47880, GHSA-79jv-5226-783f. Reported by @wandernauta, fix by @wetneb.
- Error page lacks escaping, leading to potential XSS on import of malicious project. Severity: moderate. CVE-2024-47882, GHSA-j8hp-f2mj-586g
- Directory slip in LoadLanguageCommand. Severity: high. GHSA-qfwq-6jh6-8xx4. Reported and fixed by @wetneb.
Vulnerabilities in bundled extensions
- gdata: Reflected cross-site scripting vulnerability (XSS) in
authorized.vt
. CVE-2024-47878, GHSA-pw3x-c5vp-mfc3. Reported by @wandernauta, fix by @wetneb. - gdata: leak of OAuth application credentials. Severity: high. GHSA-3pg4-qwc8-426r. Reported and fixed by @wetneb.
- database: SQLite integration allows filesystem access, remote code execution (RCE). Severity: high. CVE-2024-47881, GHSA-87cf-j763-vvh8. Reported by @wandernauta, fix by @wetneb.
Vulnerabilities in Butterfly (web framework used in OpenRefine)
- Path/URL confusion in resource handling leading to multiple weaknesses. Severity: critical. CVE-2024-47883. GHSA-3p8v-w8mr-m3x8. Reported by @wandernauta, fix by @wetneb.
- parseJSON, getJSON functions eval malicious input, leading to remote code execution (RCE). Severity: moderate. GHSA-mpcw-3j5p-p99x. Reported by @wandernauta, fix by @wetneb.
Special thanks to @wandernauta for the hard work that went into analyzing and reporting those vulnerabilities responsibly and to @tfmorris for reviewing mitigations.
Full Changelog: 3.8.5...3.8.6
3.8.6
This release fixes an issue (#6977) which prevents exporting Wikidata QuickStatements in OpenRefine 3.8.5 and 3.8.4, as well as an issue (#6941) which prevents login to Wikidata with OpenRefine 3.8.4. This release includes the same collection of important vulnerability fixes as 3.8.4. We encourage users to upgrade swiftly.
To continue using the Google Drive and Google Sheets integration, users need to obtain their own application credentials from the Google API Console.
Note: the vulnerability fixes were originally released as 3.8.3 but that version is dysfunctional due to human errors in the release process. The description of the vulnerabilities is included again here for visibility.
Vulnerabilities in OpenRefine
- PreviewExpressionCommand, which is eval, lacks protection against cross-site request forgery (CSRF).
CVE-2024-47879, GHSA-3jm4-c6qf-jrh3. Reported by @wandernauta, fix by @wetneb. - Reflected cross-site scripting vulnerability (XSS) from POST request in ExportRowsCommand. Severity: high. CVE-2024-47880, GHSA-79jv-5226-783f. Reported by @wandernauta, fix by @wetneb.
- Error page lacks escaping, leading to potential XSS on import of malicious project. Severity: moderate. CVE-2024-47882, GHSA-j8hp-f2mj-586g
- Directory slip in LoadLanguageCommand. Severity: high. GHSA-qfwq-6jh6-8xx4. Reported and fixed by @wetneb.
Vulnerabilities in bundled extensions
- gdata: Reflected cross-site scripting vulnerability (XSS) in
authorized.vt
. CVE-2024-47878, GHSA-pw3x-c5vp-mfc3. Reported by @wandernauta, fix by @wetneb. - gdata: leak of OAuth application credentials. Severity: high. GHSA-3pg4-qwc8-426r. Reported and fixed by @wetneb.
- database: SQLite integration allows filesystem access, remote code execution (RCE). Severity: high. CVE-2024-47881, GHSA-87cf-j763-vvh8. Reported by @wandernauta, fix by @wetneb.
Vulnerabilities in Butterfly (web framework used in OpenRefine)
- Path/URL confusion in resource handling leading to multiple weaknesses. Severity: critical. CVE-2024-47883. GHSA-3p8v-w8mr-m3x8. Reported by @wandernauta, fix by @wetneb.
- parseJSON, getJSON functions eval malicious input, leading to remote code execution (RCE). Severity: moderate. GHSA-mpcw-3j5p-p99x. Reported by @wandernauta, fix by @wetneb.
Special thanks to @wandernauta for the hard work that went into analyzing and reporting those vulnerabilities responsibly and to @tfmorris for reviewing mitigations.
Full Changelog: 3.8.5...3.8.6
OpenRefine 3.8.5
This release fixes an issue (#6941) which prevents login to Wikidata with OpenRefine 3.8.4. This release includes the same collection of important vulnerability fixes as 3.8.4. We encourage users to upgrade swiftly.
To continue using the Google Drive and Google Sheets integration, users need to obtain their own application credentials from the Google API Console.
Note: the vulnerability fixes were originally released as 3.8.3 but that version is dysfunctional due to human errors in the release process. The description of the vulnerabilities is included again here for visibility.
Vulnerabilities in OpenRefine
- PreviewExpressionCommand, which is eval, lacks protection against cross-site request forgery (CSRF).
CVE-2024-47879, GHSA-3jm4-c6qf-jrh3. Reported by @wandernauta, fix by @wetneb. - Reflected cross-site scripting vulnerability (XSS) from POST request in ExportRowsCommand. Severity: high. CVE-2024-47880, GHSA-79jv-5226-783f. Reported by @wandernauta, fix by @wetneb.
- Error page lacks escaping, leading to potential XSS on import of malicious project. Severity: moderate. CVE-2024-47882, GHSA-j8hp-f2mj-586g
- Directory slip in LoadLanguageCommand. Severity: high. GHSA-qfwq-6jh6-8xx4. Reported and fixed by @wetneb.
Vulnerabilities in bundled extensions
- gdata: Reflected cross-site scripting vulnerability (XSS) in
authorized.vt
. CVE-2024-47878, GHSA-pw3x-c5vp-mfc3. Reported by @wandernauta, fix by @wetneb. - gdata: leak of OAuth application credentials. Severity: high. GHSA-3pg4-qwc8-426r. Reported and fixed by @wetneb.
- database: SQLite integration allows filesystem access, remote code execution (RCE). Severity: high. CVE-2024-47881, GHSA-87cf-j763-vvh8. Reported by @wandernauta, fix by @wetneb.
Vulnerabilities in Butterfly (web framework used in OpenRefine)
- Path/URL confusion in resource handling leading to multiple weaknesses. Severity: critical. CVE-2024-47883. GHSA-3p8v-w8mr-m3x8. Reported by @wandernauta, fix by @wetneb.
- parseJSON, getJSON functions eval malicious input, leading to remote code execution (RCE). Severity: moderate. GHSA-mpcw-3j5p-p99x. Reported by @wandernauta, fix by @wetneb.
Special thanks to @wandernauta for the hard work that went into analyzing and reporting those vulnerabilities responsibly and to @tfmorris for reviewing mitigations.
Full Changelog: 3.8.4...3.8.5
OpenRefine 3.8.4
This release fixes a collection of important vulnerabilities in OpenRefine. We encourage users to upgrade swiftly.
To continue using the Google Drive and Google Sheets integration, users need to obtain their own application credentials from the Google API Console.
Note: the vulnerability fixes were originally released as 3.8.3 but that version is dysfunctional due to human errors in the release process. The description of the vulnerabilities is included again here for visibility.
Vulnerabilities in OpenRefine
- PreviewExpressionCommand, which is eval, lacks protection against cross-site request forgery (CSRF).
CVE-2024-47879, GHSA-3jm4-c6qf-jrh3. Reported by @wandernauta, fix by @wetneb. - Reflected cross-site scripting vulnerability (XSS) from POST request in ExportRowsCommand. Severity: high. CVE-2024-47880, GHSA-79jv-5226-783f. Reported by @wandernauta, fix by @wetneb.
- Error page lacks escaping, leading to potential XSS on import of malicious project. Severity: moderate. CVE-2024-47882, GHSA-j8hp-f2mj-586g
- Directory slip in LoadLanguageCommand. Severity: high. GHSA-qfwq-6jh6-8xx4. Reported and fixed by @wetneb.
Vulnerabilities in bundled extensions
- gdata: Reflected cross-site scripting vulnerability (XSS) in
authorized.vt
. CVE-2024-47878, GHSA-pw3x-c5vp-mfc3. Reported by @wandernauta, fix by @wetneb. - gdata: leak of OAuth application credentials. Severity: high. GHSA-3pg4-qwc8-426r. Reported and fixed by @wetneb.
- database: SQLite integration allows filesystem access, remote code execution (RCE). Severity: high. CVE-2024-47881, GHSA-87cf-j763-vvh8. Reported by @wandernauta, fix by @wetneb.
Vulnerabilities in Butterfly (web framework used in OpenRefine)
- Path/URL confusion in resource handling leading to multiple weaknesses. Severity: critical. CVE-2024-47883. GHSA-3p8v-w8mr-m3x8. Reported by @wandernauta, fix by @wetneb.
- parseJSON, getJSON functions eval malicious input, leading to remote code execution (RCE). Severity: moderate. GHSA-mpcw-3j5p-p99x. Reported by @wandernauta, fix by @wetneb.
Special thanks to @wandernauta for the hard work that went into analyzing and reporting those vulnerabilities responsibly and to @tfmorris for reviewing mitigations.
OpenRefine 3.8.3
This version is dysfunctional, due to errors in the release process. Use OpenRefine 3.8.4 instead.
OpenRefine v3.8.2
This is the third stable release of the 3.8 series. Please backup your workspace directory before installing and report any problems that you encounter.
Changes
- Fixed the checking of whether OpenRefine is running on Linux when curl is absent (#6622) by @dino2580
- Fixed the opening of the browser for platforms where the Desktop.browse API is not supported (#6600) by @wetneb
- Fixed the preservation of the position in the grid when matching a single cell to a reconciliation candidate (#6695) by @wetneb
Full Changelog: 3.8.1...3.8.2
OpenRefine v3.8.1
This is the second stable release of the 3.8 series. Please backup your workspace directory before installing and report any problems that you encounter.
Changes
For an overview of the main changes since 3.7, refer to the changelog for 3.8-beta1. This lists the changes since 3.8.0.
OpenRefine v3.8.0
This is the first stable release of the 3.8 series. Please backup your workspace directory before installing and report any problems that you encounter.
Changes
OpenRefine v3.8-beta5
This is the second beta release of the 3.8 series (version numbers 3.8-beta2
, 3.8-beta3
and 3.8-beta4
are skipped because of publishing problems). Please backup your workspace directory before installing and report any problems that you encounter.
Changes
- Improve separator guessing for CSV/TSV files (#6516) by @tfmorris
- Avoid crashing when cell renderer throws an exception (workaround for OpenRefine/CommonsExtension#99) by @wetneb
- Fix recon deserialization error for matched cells (#6464) by @wetneb
- Normalize the ordering of "OK" / "Cancel" buttons in dialogs (#6353) by @zyadtaha
- Removed grey background from (#6440) by @abhishekkujur1307
- Correct the behavior of cell editing when confirming single cell changes with "Enter" (#6406) by @WR-Smiley
- Package Java classes into jar archives (#6257) by @wetneb