Embargoes and Gaps

Embargoes and Gaps

Part of FreeREG-Core-Records.

Models: app/models/embargo_rule.rb, embargo_record.rb, gap.rb, gap_reason.rb · Controllers: embargo_rules_controller.rb, gaps_controller.rb, gap_reasons_controller.rb, freereg1_csv_entries_controller.rb (edit_embargo) · Lib/rake: lib/add_embargo_record.rb, lib/remove_stale_embargoed_flag.rb, foo:process_embargo_records, foo:add_embargo_record, foo:remove_stale_embargoed_flag

Both hang off a Register. An embargo hides records that are too recent to publish, usually because a record office has asked for it. A gap tells researchers "there are no records here for these years, and here's why".

Embargoes

There are two layers:

• EmbargoRule (a document, belongs_to :register): "baptisms in this register are embargoed for 100 years", or "…until the beginning of 2030". At most one rule per register per record type (only_one_rule_per_record_type).
• EmbargoRecord (embedded in Freereg1CsvEntry, embeds_many): the decision as applied to one entry: embargoed, release_year, who ('register_rule' or a person), why, when, rule_applied (rule id), rule_date (the rule's updated_at at the time). The last embedded record is the current state. Earlier ones are history.

The search side only looks at two denormalised fields on SearchRecord: embargoed and release_year. A record is hidden while embargoed && release_year > current year. That's enforced in the MongoDB query ($nor) and again when results are displayed (see Search-Query-System).

When a search record is built, these two fields come from the entry's current state, not straight from the last embedded record (Freereg1CsvEntry#currently_under_embargo? and #effective_embargo_release_year, Sept 2026, 2d1c9b717):

• the "current" EmbargoRecord is the most recently updated one;
• for a register_rule embargo, the release year is recomputed from the register's current rule, so stale years stored on old records don't matter;
• for a manual embargo, it's the stored release_year/release_date;
• an active embargo with no release year falls back to the register's rule for that record type, or failing that to now + 100 years (SearchRecord::INDEFINITE_EMBARGO_YEARS). Previously a nil release year let an embargoed record leak into search results.

Rule fields

record_type, rule (one of EmbargoRuleOptions::ALL_OPTIONS: 'Embargoed for the period of ' or 'Embargoed until the beginning of '), period (years, or a calendar year), period_type ('period' or 'end'), authority (who imposed it), reason, member_who_created.

EmbargoRecord.process_embargo_year(rule, entry_year) computes the release year:

• "until the beginning of" rules (period_type == 'end', or rule text starting Embargoed until the beginning of) → period (a fixed year)
• "for the period of" rules (period_type == 'period') → entry_year + period (e.g. 1950 + 100 = 2050)

period_type is derived from the rule text by EmbargoRule#assign_period_type_from_rule (a before_validation) whenever a rule is created or its text changes. It isn't overwritten if only period_type itself was changed, e.g. by a console repair.

Validation: period rules take 0–125; "until the beginning of" rules take a year between now and now + 25.

How a rule reaches the entries

1. Creating, updating or destroying a rule appends register_id,timestamp to the register embargo list file (config.register_embargo_list), via after_create/after_update/before_destroy.

2. A coordinator can apply it immediately with "Process" (EmbargoRulesController#process_embargo_rule → EmbargoRule#process_embargo_records). This spawns rake foo:process_embargo_records[rule_id,email], guarded by tmp/<rule_id>_rake_lock_file.txt, which runs AddEmbargoRecord.process_embargo_records_for_a_embargo and emails the coordinator when done.

3. Otherwise rake foo:add_embargo_record[limit] (AddEmbargoRecord.process, guarded by tmp/embargo_lock_file.txt) works through the registers named in the list file on a schedule.

4. For every entry of every batch of that record type, Freereg1CsvEntry#process_embargo decides whether to append a new EmbargoRecord:

   • an entry with an individual (non-rule) embargo is left alone;
   • no embargo yet and already past the release year → nothing;
   • currently embargoed and now past the release year → append a release record (embargoed: false);
   • the same rule version is already applied (rule_date == rule.updated_at) → nothing;
   • otherwise → append an embargoed record.

   Then the search record's embargoed/release_year are updated.

5. The same process_embargo also runs every time a search record is built (SearchRecord.update_create_search_record), so newly uploaded or edited entries pick up existing rules automatically.

rake foo:remove_stale_embargoed_flag[limit] finds entries whose last EmbargoRecord is embargoed with a release year that has passed, and sets embargoed: false on their search records. It doesn't append a release EmbargoRecord to the entry. Visibility doesn't depend on this task, since the filters compare release_year with the current year, but the entry's embargo history still says "embargoed" until the rule is next reprocessed.

Individual embargoes

Members acting as system_administrator or executive_director can embargo or release a single entry from freereg1_csv_entries/:id/edit_embargo. Since Sept 2026 that page shows the entry's embargo history (newest first), a one-line current-status summary (embargo_current_status_summary), and a single "new change" form where the member must explicitly choose embargo or release (EmbargoRecord validates embargoed is set on create). Other roles are redirected with "You are not permitted to edit embargoes". Managing rules is wider: EmbargoRule::PERMITTED_ROLES (system_administrator, executive_director, county_coordinator, data_manager, country_coordinator) can create and process rules.

The form submits only embargo_records_attributes, so such an embargo-only edit doesn't back up or lock the batch (see Batch-Records-Freereg1CsvFile-and-Entry). Freereg1CsvFilesController#embargoed_entries lists the entries of a batch that are currently embargoed (since Sept 2026 it excludes entries whose release year has passed).

Gaps

A Gap documents a span of years with no transcribed records for a register: start_date, end_date (integer years), record_type (ba/ma/bu or 'All'), reason (text chosen from the GapReason list), note, and optionally the freereg1_csv_file (batch id, stored as a String, not a relation) the gap was noticed in.

Gaps are shown on the public register contents page (freereg_contents/:id/gaps_and_embargoes, together with embargo rules) so researchers don't assume records exist. GapReason is an admin-maintained list (gap_reasons_controller).

Permissions (can_be_edited? / can_be_deleted?)

Allowed if any of these hold:

• the gap is record_type 'All' and the user is a county/country coordinator, data manager, system administrator or executive director;
• the gap's batch belongs to the user;
• the gap's batch is in one of the user's county_groups;
• belongs_to_my_syndicate? (based on the user's syndicate membership);
• the user is a system_administrator or executive_director (and data_manger, see below).