Releases: redivis/meta
Releases · redivis/meta
Releases have moved!
You can now view details about all Redivis releases here
2021-12-17
Dataset citations and DOIs
Full release notes available at: https://redivis.com/updates#2021-12-17
2021-10-13
Billing and archived tables
Redivis now has an updated billing interface for organizations to reflect lower costs for inactive data.
What’s new
- Organizations' dataset tables and versions will now automatically become “inactive” when not used for 90 days, and will be billed at a lower rate. Learn more.
- Occasionally some unused tables in projects will be archived. To reference this table in a transform or notebook it will first need to be unarchived. Learn more.
- Streamlined billing interface for organizations, and added additional options for invoice customization.
2021-09-11
Notebooks
You can now use notebooks on Redivis to analyze data in a secure environment. These notebooks leverage Jupyter Labs and allow use of open-source scientific toolkits in Python, R, Stata, and SAS. These updates integrate seamlessly with our live collaboration features in Redivis projects for an end-to-end research experience.
What's new
- You can create notebook nodes in a project to analyze data. Create a new notebook, or copy/paste from existing work you’ve done.
- Notebooks can be downloaded in .ipynb, .pdf, or HTML formats.
- Anyone can add a personal Stata license to their workspace to create and run Stata notebooks.
- We’ve updated the new project interface to help you get started with the Redivis project tool.
What's fixed
- Fixed an issue where renaming of project tables and transforms incorrectly allowed duplicate names.
2021-08-02
Institutions
Institutions now have a brand new homepage, including a search across all datasets hosted by member organizations, and new institutional branding throughout the homepage and organization pages. In addition, an updated Redivis homepage showcases specific features for researchers and for organizations.
What's new
- Institutions: Interactive homepage graphics allow exploration of institution datasets across entities, tags, and temporal ranges.
- Institution administration: Administrators can now feature organizations in the admin panel, which ensures those organizations show up at the top of the "Explore organizations" section of the institution homepage
- Organizations: Organizations now have an editable description, which is displayed in tiles on the institution page.
- General: Sign-ups for Redivis office hours are now available through the
?menu to the right of the top bar.
What's changed
- Institutions: Updated homepage with new organization tiles, multiple header images, customizable name for institutional homepage, and markdown "About" section.
- Institutions: Customizable links appear in institution footer, which allow users to navigate to these highlighted resources from anywhere in the data portal.
- Organizations: Organization (and their dataset) homepages now contain a footer with links to the parent institution.
- Institution administration: All institution logos, homepage name, description, multiple background images, and about content are configurable through an updated institution admin panel.
- Institution administration: A new organization creation flow allows configuration of organization administrators.
What's fixed
- Fixed an issue where certain organizations would show an incorrect parent institution in the top bar.
- Fixed an issue where some project tables had incorrect associations to parent transforms based on previous table merge configuration.
- Fixed an issue where sampled datasets in projects would incorrectly associate non-sampled tables with downstream transforms.
2021-07-06
This release presents an updated project tool interface which allows forking of existing projects, streamlined project- and node-level operations via a centralized top-bar, and improved project configuration and navigation between nodes.
🚀 New Features
- Project creation: Projects can be forked to create a new project in your workspace with an identical configuration of datasets, transforms and tables.
- Project tool: An empty project now has helper text and highlighted entry points to build a complete project, by adding datasets and creating transforms.
- Project tool: Datasets can now be replaced in projects, with a new interface to specify how tables from the replaced dataset should be assigned to existing branches of the project.
✨ UI
- Workspace: Improved project creation flow, with ability to add a new project to a study during project creation.
- Project tool: updated minimap zoom! In addition to the standard controls, you can pinch or cmd/ctrl+scroll to zoom and navigate your project tree
- Project tool: Project level actions – including dataset addition, value list management, search, and tree view navigation – are centralized to new menus in the top left of the toolbar.
- Project tool: Project renaming, study assignment, sharing, and description are available in a new project view, accessible through the middle toolbar button.
- Project tool: Dataset sampling, version switching, and transforming of tables centralized to menus in top right of toolbar.
- Project tool: Navigation to a transform's source or output table – and to a table's upstream or downstream transform – is centralized in buttons and menus at the top right of the toolbar.
- Project tool: Stale, edited, and error status messaging for transforms is now available to the right of the transform name.
- Project tool: Options to run a transform and its downstream nodes, or to run upstream stale nodes before running a given transform, are available in a new 'Run' button in the top right of the toolbar.
- Project tool: Node renaming is now accessible by clicking on the node title.
- Project tool: Performance improvements give a smoother scrolling and panning experience in the project tree view, as well as snappier interactions throughout the project.
🐛 Bug fixes
- Fixed an issue where the tables list in datasets with lots of tables would disappear on scroll
🏠 Internal
- Released new button and menu components to support project toolbar buttons and menus, and a new context menu available on the project tree view.
- Many minor bug fixes and optimizations
2021-05-06
This release presents a new export interface through which users can visualize data via Google Data Studio, export to Cloud Storage and BigQuery, and download various file formats directly.
Additionally, this release creates a new system (Export environments) for organization administrators to control users' export destinations, limit size of users' exports, or manually approve any export of data. Organization administrators can view all data export activity through newly refactored export logs.
🚀 New Features
- Added one-click integration with Google Data Studio to allow visualization of data from any Redivis table.
- Added support for data export to user-specified Google Cloud Storage buckets and Google BigQuery projects.
- Added support for Parquet export format
⚠️ Notable changes
- Export: To restrict users' export of data, organization administrators must now configure export environments, which can be assigned to individual datasets or to an entire permission group.
- Export environments can be configured within the administrators' settings panel
- Admins may restrict exports to specific IP addresses, cloud storage buckets, BigQuery projects, or approve a custom request to a user-specified location.
- Export: Previous export restrictions (IP address allowlist, size restriction, and requirement of admin approval) have been automatically migrated to their corresponding export environments.
✨ UI
- Export: redesigned export interface!
- Export: For organization datasets with restricted exports, users can apply for approval via the table export modal, and manage their approvals in their workspace under the relevant organization page.
- Administration: Organization administrators can approve and reject member requests for export approval in the organization admin panel under the corresponding member page.
- Logs: Export logs have a dedicated tab in the organization administration 'Logs' page, and reference the relevant download location through an export environment, if exports are restricted.
- Logs: 'Audit' (previously 'Edit') logs contain information about changes to an organization's data assets, including changes to users' access.
- Logs: 'Query' (previously 'Usage') logs contain information about all queries done to an organization's data, namely data transformations made by users in the project tool.
🏠 Internal
- Released new text input components for improved UI around organization administration and export approval.
- Many minor fixes and optimizations 💯
2021-03-04
This release improves editing of table- and variable-level metadata for organizations hosting Redivis datasets.
Additionally, this release showcases past geospatial visualization work via a stand-alone open-source application, and updates our internal low-level component infrastructure, including a new public storybook page.
🚀 New Features
- Released a geospatial visualizer to highlight use of Redivis API in a stand-alone application.
- Added support for institutional billing plans, where multiple organizations are covered by a single institutional bill.
✨ UI
- Administration: A new interface for updating description, temporal range, row entity and configuring samples on dataset tables.
- Administration: A new variable-level metadata editing flow, which allows modification of label, description, and value labels for an individual variable (in a single table, or across many tables in a dataset).
- Administration: Updated bulk variable metadata editing via file upload, with csv and json specifications.
🐛 Bug fixes
- Fixed an issue where exporting dataset tables wasn't available on administration panel and organization homepage.
- Fixed an issue preventing configuration of samples during the version release process.
🏠 Internal
- Improved coverage of new Link and Button components for improved UI around dataset administration.
- Removed support of legacy Redivis visualization tools.
- Released our storybook component library for transparency into low-level component development.
2021-01-06
This release enables federated hosting of data for Redivis organizations, where any individual dataset may be hosted in an independently-controlled Google Cloud project.
Additionally, this release updates several administrative interfaces, allowing organization administrators to link Redivis to external Google Cloud projects and to see an updated view of organization billing over time, now accounting for differential rates for data stored in Redivis and in external projects.
🚀 New Features
- Administration: Organizations can configure external Google Cloud projects to host their Redivis datasets, via the 'Storage locations' tab in your organization's 'Settings' page (documentation).
- Administration: administrators may choose place any organization dataset in their custom Google Cloud project, through the 'Manage storage location' option in the dropdown on a dataset's page header.
- Billing: New graph shows organization administrators billing and data storage over time for their organization. If any organization data is stored in an external Google Cloud project, two graph series lines will show allocation of data (and storage cost) in default and custom storage locations.
- Data upload: improved automatic date(time) parsing during data ingest to recognize common formats.
- Data upload: support Stata, SPSS, and Excel files up to 25GB
✨ UI
- Administration: A new filter on the organization datasets page allows administrators to see all datasets stored in a specific custom Google Cloud project (or in the default Redivis storage space).
- Administration: Updated interface for creating and managing an organization's Export Locations.
- Administration: automatically open access management modal when a dataset has pending access requests.
- Institution administration: Updated interface for institution account management, including allowing self-assignment as an administrator of any organization within an institution.
- Administration: Updated organization and institution custom color editing interface
- Workspace: 'Usage' page is updated to show a new graphical interface to see data storage over time.
- Workspace: updated display for all organizations that you're a member of
- Login: Updated login modal allows users to search for their institutional identity provider (instead of typing in their email).
- Emails: minor updates to notification emails to make them clearer.
🐛 Bug fixes
- Fixed an issue where the editable view of permission groups incorrectly showed a 'Public' access status on the 'Sample' level.
- Fixed an issue where import progress wasn't being properly updated on data uploads
- Fixed an issue where an institutional administrator couldn't make themselves an administrator of an organization they weren't a member of.
- Fixed an issue with the billing display related to the new year.
- Fixed an issue where integer-like variables with leading zeros could be interpreted as an integer in certain edge cases — the defined behavior is for any variable containing values with leading zeros to be ingested as a string, in order to preserve these zeros.
🏠 Internal
- Refactored Stripe integration to make invoices clearer
- Refactored data ingestion queues to improve reliability
2020-12-01
This release contains updates to membership in Redivis organizations, allowing users to submit membership applications tie to an authenticated identity, while administrators can configure automatic or manual approval and confirm the identity of their members.
Additionally, this release re-designs several relevant configurations for organization administrators, including membership configuration and approval, and gives users an updated workspace page, with the ability to manage email subscriptions to announcements and notifications.
🚀 New Features
- Access: users can now apply for membership in an organization by submitting an application with an authenticated login, which allows them to access restricted data hosted by that organization
- Access: users must now re-authenticate with their provided login after a period of inactivity to access restricted data hosted by an organizations, specifically to view cells, query tables in a transform, or export data from the Redivis platform
- Access: administrators can now configure how users apply for membership, including a timeout ensuring inactive users re-authenticate to access restricted data, as well as preferred identity providers and enabling (conditional) automatic membership approval
- Access: administrators can now configure dataset and permission group access levels to All members, which ensures an approved organization membership before a user can access data at that level
- Access: requirements are now classified only as 'Member' or 'Study' requirements, with the option to limit the requirements to specific datasets configurable within the individual requirement
- Access: users can now correspond with admins via comments before submitting membership forms, requirements, and restriction approvals
- Administration: administrators can invite anyone to join their organization via an invite email, which provides new members an easy path to gaining access to their data
- Notifications: users now receive notifications for access changes to user-owned datasets, in addition to organization-owned datasets
- Notifications: users can now subscribe to occasional emails about product updates in their workspace settings, which showcase workflow strategies and new features as they are released
- Upload: the upload parser now recognizes a wide variety of date, time, and dateTime data formats, and will automatically import these variables to the canonical format.
- Upload: automatic decompression of gzipped text files is now supported. However, we highly recommended avoiding compressed files when possible, as this will substantially slow down data ingest performance.
⚠️ Notable changes
- Access: All organization-hosted restricted datasets now require membership as a required first step in the access process, which will be automatic for any users with a previous relationship with an organization (e.g., had a requirement or direct access approved)
- Access: Previous authentication requirements have been deprecated, and replaced with an appropriate membership configuration which ensures organization administrators will collect authenticated identities from their members
- If your organization had configured access restrictions with an authentication requirement, we'll contact you!
- Access: Study requirements are now only accessible to admins via the Studies page; previously, you could access both member and study requirements through the Members page
✨ UI
- Workspace: An updated settings page allows management of a user's public profile, authentications, integrations, API tokens, and more
- Workspace: Redesigned notifications will guide users to an updated workspace page where they can manage their organization membership
- Access: Administrators can now see a member's authentication information in the top right of every access submission
- Access: Export restriction approvals have an updated submission + approval modal highlighting details about the table to be exported, including source datasets and tables, the parent project, and size statistics
- Access: Emails notifying users of updated dataset access have been re-designed
- Access: Requirement forms that apply to specific datasets now auto-select the relevant dataset when launching the submission form.
- Administration: An updated organization settings page allows management of an organization's branding & public presence, administrators, notifications, export locations, and more
- Administration: A new requirement editor allows streamlined creation of requirements, and WYSIWYG creation of text inputs, dropdown/radio selectors, file upload and signature collection form elements
- Portal: Full organization name is now shown on an organization's pages
- Notifications: updated notification emails
🐛 Bug fixes
- Fixed an issue where browser sessions with an invalid cookie would not prompt user re-authentication
- Fixed an issue where approving a user for direct access could remove other users' access requests
- Fixed an issue where files uploaded from a local environment could stall upon client disconnection
- Fixed an issue where logs for table downloads beyond the first download were not created
- Fixed an issue where users could not submit restriction exception requests for non-location restrictions
- Fixed an issue where projects had an inaccurate 'Last updated' value
- Fixed an issue where value frequency table wouldn't indicate that a variable had more than 10,000 values
🏠 Internal
- Updated strategy for storing user profile images
- Implemented mailjet to handle notification and organization invitation emails
- Separated institution and authentication concerns with a database refactor
- Updated page metadata for improved SEO and Google Dataset Search presence