Skip to content

@imathews imathews released this Aug 18, 2021

Institutions

Institutions now have a brand new homepage, including a search across all datasets hosted by member organizations, and new institutional branding throughout the homepage and organization pages. In addition, an updated Redivis homepage showcases specific features for researchers and for organizations.

What's new

  • Institutions: Interactive homepage graphics allow exploration of institution datasets across entities, tags, and temporal ranges.
  • Institution administration: Administrators can now feature organizations in the admin panel, which ensures those organizations show up at the top of the "Explore organizations" section of the institution homepage
  • Organizations: Organizations now have an editable description, which is displayed in tiles on the institution page.
  • General: Sign-ups for Redivis office hours are now available through the ? menu to the right of the top bar.

What's changed

  • Institutions: Updated homepage with new organization tiles, multiple header images, customizable name for institutional homepage, and markdown "About" section.
  • Institutions: Customizable links appear in institution footer, which allow users to navigate to these highlighted resources from anywhere in the data portal.
  • Organizations: Organization (and their dataset) homepages now contain a footer with links to the parent institution.
  • Institution administration: All institution logos, homepage name, description, multiple background images, and about content are configurable through an updated institution admin panel.
  • Institution administration: A new organization creation flow allows configuration of organization administrators.

What's fixed

  • Fixed an issue where certain organizations would show an incorrect parent institution in the top bar.
  • Fixed an issue where some project tables had incorrect associations to parent transforms based on previous table merge configuration.
  • Fixed an issue where sampled datasets in projects would incorrectly associate non-sampled tables with downstream transforms.
Assets 2

@sfmcintyre sfmcintyre released this Jul 7, 2021

This release presents an updated project tool interface which allows forking of existing projects, streamlined project- and node-level operations via a centralized top-bar, and improved project configuration and navigation between nodes.

🚀 New Features

  • Project creation: Projects can be forked to create a new project in your workspace with an identical configuration of datasets, transforms and tables.
  • Project tool: An empty project now has helper text and highlighted entry points to build a complete project, by adding datasets and creating transforms.
  • Project tool: Datasets can now be replaced in projects, with a new interface to specify how tables from the replaced dataset should be assigned to existing branches of the project.

UI

  • Workspace: Improved project creation flow, with ability to add a new project to a study during project creation.
  • Project tool: updated minimap zoom! In addition to the standard controls, you can pinch or cmd/ctrl+scroll to zoom and navigate your project tree
  • Project tool: Project level actions – including dataset addition, value list management, search, and tree view navigation – are centralized to new menus in the top left of the toolbar.
  • Project tool: Project renaming, study assignment, sharing, and description are available in a new project view, accessible through the middle toolbar button.
  • Project tool: Dataset sampling, version switching, and transforming of tables centralized to menus in top right of toolbar.
  • Project tool: Navigation to a transform's source or output table – and to a table's upstream or downstream transform – is centralized in buttons and menus at the top right of the toolbar.
  • Project tool: Stale, edited, and error status messaging for transforms is now available to the right of the transform name.
  • Project tool: Options to run a transform and its downstream nodes, or to run upstream stale nodes before running a given transform, are available in a new 'Run' button in the top right of the toolbar.
  • Project tool: Node renaming is now accessible by clicking on the node title.
  • Project tool: Performance improvements give a smoother scrolling and panning experience in the project tree view, as well as snappier interactions throughout the project.

🐛 Bug fixes

  • Fixed an issue where the tables list in datasets with lots of tables would disappear on scroll

🏠 Internal

  • Released new button and menu components to support project toolbar buttons and menus, and a new context menu available on the project tree view.
  • Many minor bug fixes and optimizations
Assets 2

@sfmcintyre sfmcintyre released this May 7, 2021

This release presents a new export interface through which users can visualize data via Google Data Studio, export to Cloud Storage and BigQuery, and download various file formats directly.

Additionally, this release creates a new system (Export environments) for organization administrators to control users' export destinations, limit size of users' exports, or manually approve any export of data. Organization administrators can view all data export activity through newly refactored export logs.

🚀 New Features

  • Added one-click integration with Google Data Studio to allow visualization of data from any Redivis table.
  • Added support for data export to user-specified Google Cloud Storage buckets and Google BigQuery projects.
  • Added support for Parquet export format

⚠️ Notable changes

  • Export: To restrict users' export of data, organization administrators must now configure export environments, which can be assigned to individual datasets or to an entire permission group.
    • Export environments can be configured within the administrators' settings panel
    • Admins may restrict exports to specific IP addresses, cloud storage buckets, BigQuery projects, or approve a custom request to a user-specified location.
  • Export: Previous export restrictions (IP address allowlist, size restriction, and requirement of admin approval) have been automatically migrated to their corresponding export environments.

UI

  • Export: redesigned export interface!
    • Provide context and sample code for working with the Redivis python and R client libraries
    • New, more prominent "Export table" button on every table
  • Export: For organization datasets with restricted exports, users can apply for approval via the table export modal, and manage their approvals in their workspace under the relevant organization page.
  • Administration: Organization administrators can approve and reject member requests for export approval in the organization admin panel under the corresponding member page.
  • Logs: Export logs have a dedicated tab in the organization administration 'Logs' page, and reference the relevant download location through an export environment, if exports are restricted.
  • Logs: 'Audit' (previously 'Edit') logs contain information about changes to an organization's data assets, including changes to users' access.
  • Logs: 'Query' (previously 'Usage') logs contain information about all queries done to an organization's data, namely data transformations made by users in the project tool.

🏠 Internal

  • Released new text input components for improved UI around organization administration and export approval.
  • Many minor fixes and optimizations 💯
Assets 2

@sfmcintyre sfmcintyre released this Mar 5, 2021

This release improves editing of table- and variable-level metadata for organizations hosting Redivis datasets.

Additionally, this release showcases past geospatial visualization work via a stand-alone open-source application, and updates our internal low-level component infrastructure, including a new public storybook page.

🚀 New Features

  • Released a geospatial visualizer to highlight use of Redivis API in a stand-alone application.
  • Added support for institutional billing plans, where multiple organizations are covered by a single institutional bill.

UI

  • Administration: A new interface for updating description, temporal range, row entity and configuring samples on dataset tables.
  • Administration: A new variable-level metadata editing flow, which allows modification of label, description, and value labels for an individual variable (in a single table, or across many tables in a dataset).
  • Administration: Updated bulk variable metadata editing via file upload, with csv and json specifications.

🐛 Bug fixes

  • Fixed an issue where exporting dataset tables wasn't available on administration panel and organization homepage.
  • Fixed an issue preventing configuration of samples during the version release process.

🏠 Internal

  • Improved coverage of new Link and Button components for improved UI around dataset administration.
  • Removed support of legacy Redivis visualization tools.
  • Released our storybook component library for transparency into low-level component development.
Assets 2

@sfmcintyre sfmcintyre released this Jan 9, 2021

This release enables federated hosting of data for Redivis organizations, where any individual dataset may be hosted in an independently-controlled Google Cloud project.

Additionally, this release updates several administrative interfaces, allowing organization administrators to link Redivis to external Google Cloud projects and to see an updated view of organization billing over time, now accounting for differential rates for data stored in Redivis and in external projects.

🚀 New Features

  • Administration: Organizations can configure external Google Cloud projects to host their Redivis datasets, via the 'Storage locations' tab in your organization's 'Settings' page (documentation).
  • Administration: administrators may choose place any organization dataset in their custom Google Cloud project, through the 'Manage storage location' option in the dropdown on a dataset's page header.
  • Billing: New graph shows organization administrators billing and data storage over time for their organization. If any organization data is stored in an external Google Cloud project, two graph series lines will show allocation of data (and storage cost) in default and custom storage locations.
  • Data upload: improved automatic date(time) parsing during data ingest to recognize common formats.
  • Data upload: support Stata, SPSS, and Excel files up to 25GB

UI

  • Administration: A new filter on the organization datasets page allows administrators to see all datasets stored in a specific custom Google Cloud project (or in the default Redivis storage space).
  • Administration: Updated interface for creating and managing an organization's Export Locations.
  • Administration: automatically open access management modal when a dataset has pending access requests.
  • Institution administration: Updated interface for institution account management, including allowing self-assignment as an administrator of any organization within an institution.
  • Administration: Updated organization and institution custom color editing interface
  • Workspace: 'Usage' page is updated to show a new graphical interface to see data storage over time.
  • Workspace: updated display for all organizations that you're a member of
  • Login: Updated login modal allows users to search for their institutional identity provider (instead of typing in their email).
  • Emails: minor updates to notification emails to make them clearer.

🐛 Bug fixes

  • Fixed an issue where the editable view of permission groups incorrectly showed a 'Public' access status on the 'Sample' level.
  • Fixed an issue where import progress wasn't being properly updated on data uploads
  • Fixed an issue where an institutional administrator couldn't make themselves an administrator of an organization they weren't a member of.
  • Fixed an issue with the billing display related to the new year.
  • Fixed an issue where integer-like variables with leading zeros could be interpreted as an integer in certain edge cases — the defined behavior is for any variable containing values with leading zeros to be ingested as a string, in order to preserve these zeros.

🏠 Internal

  • Refactored Stripe integration to make invoices clearer
  • Refactored data ingestion queues to improve reliability
Assets 2

@sfmcintyre sfmcintyre released this Dec 2, 2020

This release contains updates to membership in Redivis organizations, allowing users to submit membership applications tie to an authenticated identity, while administrators can configure automatic or manual approval and confirm the identity of their members.

Additionally, this release re-designs several relevant configurations for organization administrators, including membership configuration and approval, and gives users an updated workspace page, with the ability to manage email subscriptions to announcements and notifications.

🚀 New Features

  • Access: users can now apply for membership in an organization by submitting an application with an authenticated login, which allows them to access restricted data hosted by that organization
  • Access: users must now re-authenticate with their provided login after a period of inactivity to access restricted data hosted by an organizations, specifically to view cells, query tables in a transform, or export data from the Redivis platform
  • Access: administrators can now configure how users apply for membership, including a timeout ensuring inactive users re-authenticate to access restricted data, as well as preferred identity providers and enabling (conditional) automatic membership approval
  • Access: administrators can now configure dataset and permission group access levels to All members, which ensures an approved organization membership before a user can access data at that level
  • Access: requirements are now classified only as 'Member' or 'Study' requirements, with the option to limit the requirements to specific datasets configurable within the individual requirement
  • Access: users can now correspond with admins via comments before submitting membership forms, requirements, and restriction approvals
  • Administration: administrators can invite anyone to join their organization via an invite email, which provides new members an easy path to gaining access to their data
  • Notifications: users now receive notifications for access changes to user-owned datasets, in addition to organization-owned datasets
  • Notifications: users can now subscribe to occasional emails about product updates in their workspace settings, which showcase workflow strategies and new features as they are released
  • Upload: the upload parser now recognizes a wide variety of date, time, and dateTime data formats, and will automatically import these variables to the canonical format.
  • Upload: automatic decompression of gzipped text files is now supported. However, we highly recommended avoiding compressed files when possible, as this will substantially slow down data ingest performance.

⚠️ Notable changes

  • Access: All organization-hosted restricted datasets now require membership as a required first step in the access process, which will be automatic for any users with a previous relationship with an organization (e.g., had a requirement or direct access approved)
  • Access: Previous authentication requirements have been deprecated, and replaced with an appropriate membership configuration which ensures organization administrators will collect authenticated identities from their members
    • If your organization had configured access restrictions with an authentication requirement, we'll contact you!
  • Access: Study requirements are now only accessible to admins via the Studies page; previously, you could access both member and study requirements through the Members page

UI

  • Workspace: An updated settings page allows management of a user's public profile, authentications, integrations, API tokens, and more
  • Workspace: Redesigned notifications will guide users to an updated workspace page where they can manage their organization membership
  • Access: Administrators can now see a member's authentication information in the top right of every access submission
  • Access: Export restriction approvals have an updated submission + approval modal highlighting details about the table to be exported, including source datasets and tables, the parent project, and size statistics
  • Access: Emails notifying users of updated dataset access have been re-designed
  • Access: Requirement forms that apply to specific datasets now auto-select the relevant dataset when launching the submission form.
  • Administration: An updated organization settings page allows management of an organization's branding & public presence, administrators, notifications, export locations, and more
  • Administration: A new requirement editor allows streamlined creation of requirements, and WYSIWYG creation of text inputs, dropdown/radio selectors, file upload and signature collection form elements
  • Portal: Full organization name is now shown on an organization's pages
  • Notifications: updated notification emails

🐛 Bug fixes

  • Fixed an issue where browser sessions with an invalid cookie would not prompt user re-authentication
  • Fixed an issue where approving a user for direct access could remove other users' access requests
  • Fixed an issue where files uploaded from a local environment could stall upon client disconnection
  • Fixed an issue where logs for table downloads beyond the first download were not created
  • Fixed an issue where users could not submit restriction exception requests for non-location restrictions
  • Fixed an issue where projects had an inaccurate 'Last updated' value
  • Fixed an issue where value frequency table wouldn't indicate that a variable had more than 10,000 values

🏠 Internal

  • Updated strategy for storing user profile images
  • Implemented mailjet to handle notification and organization invitation emails
  • Separated institution and authentication concerns with a database refactor
  • Updated page metadata for improved SEO and Google Dataset Search presence
Assets 2

@sfmcintyre sfmcintyre released this Oct 5, 2020

This release contains updates to the Redivis top bar and login flow, with a focus on streamlining navigation between organization pages and workspace.

Additionally, this release presents a new homepage with updated tutorials, videos and product descriptions, and allows organizations to upload header images and other settings.

🚀 New Features

  • New home page showcases new videos, product walkthrough, and demo request forms.
  • Redivis home page top bar organizes features & security, documentation & tutorials, and company resources in new menus.
  • Log in: New account creation and log in modals
  • Administration: Restructured organization profile tab allows header image configuration, and streamlines name, color, cover photo, and about editing.
  • Users can subscribe to product updates email when creating account

UI

  • Top bar: Home icon navigates to workspace organization, projects, and datasets.
  • Top bar: Organization header image launches dropdown menu to homepage + custom organization links
  • Top bar: Redivis documentation, release notes, contact and issue links moved to '?' dropdown in top right
  • Administrator: Permission group creation flow features a streamlined modal
  • Users must select at least one dataset in order to submit a per-dataset requirement
  • Administrators can now add additional export locations inline when updating restriction configurations on permission groups and datasets

🐛 Bug fixes

  • Fixed an issue where users could not submit exception requests to download tables with row or size limits
  • Fixed an issue where public datasets would appear as 'private' in dataset searches
  • Fixed an issue where new entities (describing row content) could not be added to dataset tables
  • Fixed an issue preventing updating of datasets and permission groups without a description

🏠 Internal

  • Engineering component library improved to centralize links, buttons, inputs, etc
  • Improved HTML metadata to improve schema.org compliance
Assets 2

@imathews imathews released this May 26, 2020

This release contains major UX updates for creating and uploading datasets, with a focus on streamlining common actions while providing clear insights to data as they are uploaded, modified, and versioned.

Additionally, this release offers new functionality in managing access to datasets, allowing data owners to configure direct (shared) access in tandem with requirement access

🚀 New Features

  • Upload: You can now preview variable statistics and view cells on individual uploads
  • Upload: Cell preview is now available on unreleased tables
  • Upload: All datasets now support up to 4,000 versions
  • Upload: Tables can now have up to 9,990 variables
  • Upload: Ability to add unreleased versions to a project for pre-release testing and validation
  • Upload: Unreleased versions can now be deleted
  • Administration: Organization members can now be granted "edit" access to dataset(s), allowing for data contribution and maintenance from non-admins
  • Access: New functionality to support configuring "Direct access" as well as "Requirement access" at different levels on permission groups. Supports limiting datasets to a few specific users while enforcing completion and approval for all relevant requirements.
  • API: implement tables/listRows endpoint

⚠️ Notable changes

  • Access: Requirements can no longer be placed on the "Overview" access level of a dataset. If you would like to restrict visibility of a dataset, you can configure the dataset (or permission group) to enforce "Direct access" on overview, and then share the dataset with specific individuals
    • Any requirements that were previously on the "Overview" level have been migrated to the "Metadata" access level. In this case, the overview level has been reconfigured to "Direct access only", with only direct access granted to the specific users who previously had access at this level. No users have gained or lost access as a result of these changes.
  • API: The upload type csv has been deprecated and replaced with the delimited type. This naming change reflects the fact that Redivis supports arbitrary text-delimiters in files (e.g., semicolon, tab, pipe). Uploads with type csv will still succeed, but will be returned with the type delimited.
  • API: You no longer need to poll for version release completion. A successful call to POST version/release will result in the version being released. See examples.

UI

  • Portal: New "Versions" tab on the dataset page, providing for an overview of a dataset's version history
  • Upload: Improved and simplified flow for creating a new dataset
  • Upload: Updates to the data upload interface, improving bulk management of multiple uploads and generally supporting a streamlined data upload and audit flow
  • Upload: Preview individual data uploads to view statistics, run validation queries, and validate md5 checksums
  • Upload: Updated interface for assigning temporal range on tables
  • Upload: Updated interface for configuring the sampling strategy on a version
  • Access: Display the number of users who have access at each level of a dataset, as well as the computed change in these counts caused by any updates to the access configuration
  • Access: Improved interfaces for managing incomplete access to data within a project
  • Access: Improvements to messaging when a transform cannot be run due to the project being in an incorrect study
    • In this case, the transform will be marked as limited until the project is assigned to an approved study or that project's study is approved for the relevant access
  • Projects: updated interface for previewing unused tables in a dataset

🐛 Bug fixes

  • Fixed an issue that could prevent merging certain datasets together
  • Fixed an issue where data source integrations may become invalid
  • Fixed an issue where bulk metadata management incorrectly treated variable names as case-sensitive

🏠 Internal

  • Substantial performance improvements when uploading lots of files (with lots of variables) to a particular table
  • General improvements to data upload robustness and concurrency
  • Version release now happens instantly. Previously, releases on large datasets could take minutes to hours, during which the new version would be in a pending state.
Assets 2

@imathews imathews released this Apr 24, 2020

This release contains some major updates / enhancements when navigating between organizations and your workspace. It also provides additional functionality for organization and institution administrators

🚀 New Features

  • Organizations can now configure email alerts to be notified when new access requests are submitted (either immediately or as a daily digest)
  • Organizations can add external links to help users navigate between Redivis and other resources offered by the organization
  • New organizations can now be created from the institution administrator panel
  • Organizations can now specify whether they want to be publicly listed
  • The "Go to" interface has been improved to better surface search results most relevant to your query

⚠️ Notable changes

  • Once logged in, you will no longer navigate to the the Redivis home page - instead, "home" will take you to your workspace.
    • From your workspace, you can then navigate to any organization that you're a member of
    • If you really miss the old homepage, it's still there — redivis.com/home :)

UI

  • New top bar interface! Quickly jump between organizations and your workspace. Your most recently visited organization will always be shown in the top bar.
  • New interface for discovering organizations from your workspace
  • We've removed webpage, email, and geo location from organizations' home pages — instead, these can be provided in the organization's top bar menu.
  • Improvements to the new account flow to help new users quickly find relevant organizations
  • Quick navigation to all your organizations from the workspace side bar
  • New top bar link to documentation, support, and release notes

🐛 Bug fixes

  • Fixed an issue where header navigation links were broken after a page crash
Assets 2

@imathews imathews released this Apr 24, 2020

This release focuses on usage analytics and billing for organizations, including a new flat rate billing model. It also provides an updated mechanism for managing restrictions an organizations' datasets

🚀 New Features

  • New "Billing" tab on the administrator panel to view data storage and trends within an organization
  • Billing tab includes interfaces to input payment information and view previous invoices
  • Restrictions can now be directly assigned to shared datasets or permission groups.
  • Export locations (for example, a secure server environment) are now defined in the "Settings" tab of the administrator panel

UI

  • Updated workspace "Usage" tab to reflect dataset and storage limits, show storage trends
  • Whitespace values in table cells, value lists, and frequency tables are now displayed as a character
  • Improved the bulk management interface for editing multiple datasets

🐛 Bug fixes

  • Fixed an issue when displaying certain values with whitespaces in frequency tables
  • Fixed a timezone issue in displaying summary statistics for dateTime variables
  • Several fixes to the SQL query parser to handle some errors in parsing certain queries
  • Error messages in the SQL parser will now always display the correct line number
  • Require at least 3-character specificity on global user searches
Assets 2