2026-04-21 Custom DC stable release

[rohitkumarbhagat]

Highlights

Website

• [v0/v1 migration] Add pagination to map charts by @nick-nlb in [v0/v1 migration]Add pagination to map charts #6193
• [v0/v1 migration] /bulk/info/variable-group by @nick-nlb in [v0/v1 migration] /bulk/info/variable-group #6199
• Modify the nl search go include the logic to go through v2 resolve by @shixiao-coder in Modify the nl search go include the logic to go through v2 resolve #6181
• Switching Gemini 3 to low level thinking to reduce latency by @shixiao-coder in Switching Gemini 3 to low level thinking to reduce latency #6190
• Update the code to only do single time feature flag call within 1 Gemini client by @shixiao-coder in Update the code to only do single time feature flag call within 1 Gemini client #6209
• Sanitize url and improve regex to make more secure by @nick-nlb in Sanitize url and improve regex to make more secure. #6191
• Add support for DC_INSTRUCTIONS_DIR in Docker convenience script by @kmoscoe and @clincoln8 in Add support for DC_INSTRUCTIONS_DIR in Docker convenience script #6185
• Automated Integration Test Goldens Update by @datacommons-robot-author in Automated Integration Test Goldens Update from CI #6196
• Multiple dependency and security updates (requests, vite, lodash, pygments, protobuf, picomatch, yaml) by @dependabot

Mixer

• Implement normalized Spanner schema for observations by @keyurva in Handle errors from axios request #1839
• Add fetching, wrapper, and spanner data source for BulkVariableGroupInfo by @n-h-diaz in [custom_dc] fold in IITM updates #1838, Check empty params in Flask endpoints #1844, and [Browser] Catch name fetch errors for page rendering; Use V1 API to fetch properties #1846
• Pass timeout deadline in mirrored context by @n-h-diaz in Add missing var qualifier from #1838 #1845
• Add query hint to observation queries by @n-h-diaz in Change yapf to use indent 2 #1847
• Add use_bigquery to deployment.yaml to allow per environment overrides by @n-h-diaz in [Disaster Pages] Initial Page Setup #1848
• feat: disable /place/stat-vars in Apigee by @rohitkumarbhagat in feat: add disease prevalence rates chart #1834
• Data release: release-20260410 by @ajaits in Bump loader-utils from 1.4.0 to 1.4.2 in /static #1840

Import

• [DCP] Add a flag to only do DB initialization by @gmechali in Use chosen child place type returned by mixer #496

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive set of updates across the Data Commons platform, focusing on infrastructure improvements, dependency management, and enhanced AI integration. Key changes include the implementation of a normalized Spanner schema for observations, improved handling of variable group information, and significant optimizations for Gemini-based features. The infrastructure has been bolstered with better Docker support and security hardening, while the API layer sees improved pagination and resolution logic to support more efficient data retrieval.

Highlights

• Dependency Updates: Updated multiple dependencies including requests, vite, lodash, pygments, protobuf, picomatch, and yaml to improve security and stability.
• Gemini Integration: Optimized Gemini 3 integration by switching to low-level thinking to reduce latency and added support for per-client feature flag calls.
• API and Schema Improvements: Implemented normalized Spanner schema for observations, added bulk variable group info fetching, and introduced a flag for database initialization.
• Docker and Infrastructure: Improved the Docker convenience script with support for custom instruction directories and added environment-specific overrides for BigQuery.
• Security and Stability: Enhanced security by sanitizing URLs and improving regex, and added robust error handling for oEmbed rendering.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates several dependencies, including upgrading requests to 2.33.0 and vite to 6.4.2, and introduces significant enhancements to the Natural Language (NL) processing pipeline by integrating V2 APIs and Gemini 3 support. The run_cdc_dev_docker.sh script underwent a major refactor to improve container handling and add support for hybrid instruction modes. Feedback indicates a high-severity issue in the Docker script where volume mount paths were changed to use $FLASK_ENV instead of $CUSTOM_DIR, which will likely break custom configurations. Additionally, the variable group expansion logic in variable_group.py needs to process groups in chunks to avoid ignoring data beyond the first 50 items, and the sibling fetching in variable.py should disable pagination to ensure all results are retrieved.

[gemini-code-assist]

The groups_to_open list is truncated to MAX_SVGS_IN_CALL (50) in a single API call, and the remaining groups are not processed. This means that if more than 50 groups are passed to this function, any groups beyond the first 50 (and their descendants) will be ignored. The processing should be wrapped in a loop to handle all groups in chunks.

  recurse_groups = set()
  for i in range(0, len(groups_to_open), MAX_SVGS_IN_CALL):
    chunk = groups_to_open[i:i + MAX_SVGS_IN_CALL]
    resp = dc.get_variable_group_info(chunk, [])

    for data in resp.get("data", []):
      if not (group_dcid := data.get("node")) or group_dcid in processed_groups:
        continue
      processed_groups.add(group_dcid)
      if not (info := data.get("info")):
        continue

      for child_sv in info.get("childStatVars", []):
        if not (child_sv_dcid := child_sv.get("id")):
          continue
        sv_defns[child_sv_dcid] = child_sv.get("definition", "")
        if not disable_limit and len(sv_defns) >= MAX_V2_SVS:
          return

      for child_group in info.get("childStatVarGroups", []):
        if (not (child_group_dcid := child_group.get("id")) or
            child_group_dcid in processed_groups):
          continue
        recurse_groups.add(child_group_dcid)

[gemini-code-assist]

When fetching indirect siblings (specializationOf with out=False), there can be a large number of results. The default max_pages=1 in property_values might truncate the list of sibling SVGs, leading to incomplete variable expansion. It is safer to set max_pages=None to ensure all siblings are retrieved.

Suggested change

	siblings_resp = fetch.property_values(all_parents, "specializationOf",
	False)
	siblings_resp = fetch.property_values(all_parents, "specializationOf",
	False, max_pages=None)