Dealing with Community Governance Reviews

[chris48s]

Local authorities may sometimes make minor 'technical' boundary changes in a Community Governance Review. These are problematic because they result in boundary changes defined in an Electoral Changes Order, but because the reviews are conducted at local level (without LGBCE's involvement), there is no provisional boundary set available before the first election using the new boundaries and none of our existing tooling for dealing with boundary reviews applies. An ECO resulting from a CGR will never change the number of wards or the number of councillors representing each ward, but they do change the internal boundaries.

This usually means we have to run the first election using the old (incorrect) boundaries, accepting that this means there will be some users who we're assigning to the wrong ward. Then we have to retrospectively import the new boundaries once the May BoundaryLine release is issued.

Within that structure, there are 2 distinct cases:

1. A Community Governance Review is conducted after an LGBCE electoral review has been concluded, but before the first election has been conducted using the new boundaries. e.g: The Test Valley (Electoral Changes) Order 2018 was subsequently amended by The Test Valley (Electoral Changes) Order 2019 before the first election using the new boundaries in May 2019.
2. A Community Governance Review is conducted making changes to electoral boundaries which have already been used to conduct one or more elections. e.g: The Newark & Sherwood (Electoral Changes) Order 2014 defined a set of boundaries which were used to conduct local elections in 2015. These were subsequently amended by The Newark and Sherwood (Electoral Changes) Order 2016. This defined changes applicable to the next scheduled election in May 2019.

The first of those two cases is easiest to deal with - it doesn't really need any special handling. Because no real election has ever been held using the provisional boundaries (LGBCE's final recommendations prior to the amendment order), there's no need to retain a record of them. We can run our usual process for back-porting codes/boundaries from BoundaryLine over provisional boundaries from LGBCE. The provisional boundaries will be overwritten. The elections we already ran using the provisional boundaries will now be attached to the correct ones and we'll use the right boundaries moving forwards.

The second case is more complicated. Because we've already run some elections using the old boundaries, we need to create a new DivisionSet for the revised boundaries and retrospectively edit the most recent elections to attach them to the boundaries in the latest DivisionSet.

The process I'm currently following to achieve that is:

• Set an end date on the old DivisionSet in /admin
• Create a new DivisionSet in /admin
• Copy all the divisions from the old DivisionSet to the new one using the copy_divisions management command
• For each ballot object which is incorrectly assigned to a division from the old set, update the division_id and division_geography_id to the corresponding ones from the new DivisionSet. Do this step while you've still got the old GSS codes attached to the new divisions because then you can match everything up on GSS.
• Pick out the divisions changed by the Electoral Change Order and switch the GSS codes in /admin on the Divisions in the new DivisionSet
• Use the boundaryline_import_boundaries management command to overwrite the old boundary with the new one for each new GSS code.

So, to work the Newark & Sherwood example through:

• Started off by setting the end date on DivSet 279 to 01/05/2019 (one day before local.newark-and-sherwood.2019-05-02 )

• Created a new DivSet 593 with a start date of 02/05/2019

• Ran python manage.py copy_divisions 279 593 to copy all the divisions from the old DivSet to the new one.

• At this point, I still had all the ballot objects under local.newark-and-sherwood.2019-05-02 attached to the divisions in DivSet 279, so while the divisions in DivSet 593 still had the same GSS codes on them, I ran

  from elections.models import Election
  from organisations.models import OrganisationDivision
  
  new_divset_id = 593
  election_group = "local.newark-and-sherwood.2019-05-02"
  
  elections = (
      Election.public_objects.all()
      .get(election_id=election_group)
      .get_children("public_objects")
      .all()
  )
  for e in elections:
      new_div = OrganisationDivision.objects.get(
          divisionset_id=new_divset_id,
          official_identifier=e.division.official_identifier,
      )
      e.division = new_div
      e.division_geography = new_div.geography
      e.save()

  as an ad-hoc task to switch the division_id and division_geography_ids on the child ballots of local.newark-and-sherwood.2019-05-02 to reference the divisions in DivSet 593.

• The 2016 Electoral Change Order defines boundary changes to the following 4 wards:

  • Balderton North & Coddington
  • Beacon
  • Castle
  • Farndon & Fernwood

  So the next step was to find the new GSS codes for those 4 wards from BoundaryLine and update the Division records to use the new codes:

  • Balderton North & Coddington (E05010063 --> E05011552)
  • Beacon (E05010065 --> E05011553)
  • Castle (E05010069 --> E05011554)
  • Farndon & Fernwood (E05010074 --> E05011555)

  All the other wards didn't have any changes, so they retained the same GSS codes.

• Finally once the Divisions in the new Divset 593 had the updated GSS codes on them, I made a JSON file

  [
      "gss:E05011552",
      "gss:E05011553",
      "gss:E05011554",
      "gss:E05011555"
  ]

  and ran manage.py boundaryline_import_boundaries --source "bdline_gb-2019-05b" --codes "codes.json" -f "/path/to/bdline_gb-2019-05b" to overwrite the old boundaries with the new ones.

• The end result of that is that:

  • We've got a new Divset with the correct boundaries for future elections
  • The ballots for the election we just ran with the incorrect boundaries (due to lack of choice) are now retrospectively attached to the correct boundaries that were actually used
  • Any older elections which were run using the old boundaries are still attached to the correct ward boundaries - we haven't overwritten anything or incorrectly changed the past

So.. what was the point of this issue?

Several reasons:

1. I wanted to document somewhere what I've just done so we can refer to it again next time it happens, but it feels a bit too half-baked to go in the wiki at this stage.
2. It might be worth thinking about whether we can produce better tooling to support/formalise this or automate some of it?
3. Now that I've done it one way, it might be worth thinking about better ways to do it (as a mental model, this process is a bit weird). e.g: instead of copying the old divisions to a new DivisionSet, would it be better to produce tooling to create a new DivisionSet using BoundaryLine as a base and then deal with editing the division_id and division_geography_ids on the old election objects using a map of old GSS codes --> new GSS codes?

[awdem]

To make this process slightly easier, there is now a management command that will try to assign the divisions of ballots in an election group to a given divisionset:

EveryElection/every_election/apps/elections/management/commands/assign_election_to_divisionset.py

Lines 1 to 83 in f773d74

	"""
	This management command assigns elections from election groups to a given divisionset's divisions.
	This is usually used when we're processing Community Governance Reviews as decscribed in this issue:
	https://github.com/DemocracyClub/EveryElection/issues/570.
	python manage.py assign_election_to_divisionset <election_ids> <divisionset_id>
	example:
	python manage.py assign_election_to_divisionset local.broxtowe.2023-05-04 local.broxtowe.2024-05-02 794
	"""
	from django.core.management.base import BaseCommand, CommandError
	from django.db import transaction
	from elections.models import Election
	from organisations.models import OrganisationDivision
	class Command(BaseCommand):
	help = "Tries to assign elections from election groups to a given divisionset's divisions"
	def add_arguments(self, parser):
	parser.add_argument(
	"election_group_ids",
	action="store",
	nargs="+",
	help="space separated list of election_ids",
	)
	parser.add_argument(
	"divset_id",
	action="store",
	type=int,
	help="The divisionset_id of the divisionset",
	)
	def handle(self, *args, **options):
	divset_id = options["divset_id"]
	election_group_ids = options["election_group_ids"]
	elections = self.get_elections(election_group_ids)
	self.assign_elections_to_divisionset(divset_id, elections)
	def assign_elections_to_divisionset(self, divset_id, elections):
	# Find election division in the new divisionset and update the election
	for e in elections:
	try:
	new_div = OrganisationDivision.objects.get(
	divisionset_id=divset_id,
	official_identifier=e.division.official_identifier,
	)
	e.division = new_div
	e.division_geography = new_div.geography
	self.stdout.write(
	f"Assigned {e.election_id} to "
	f"{new_div.name} ({new_div.official_identifier}) from divisionset {divset_id}"
	)
	except OrganisationDivision.DoesNotExist:
	raise CommandError(
	f"Division with official_identifier {e.division.official_identifier} not found in divisionset {divset_id}."
	)
	self.stdout.write("Saving...")
	# Save all elections in a single transaction
	with transaction.atomic():
	for e in elections:
	e.save()
	self.stdout.write("...Done")
	def get_elections(self, election_group_ids):
	election_groups = Election.public_objects.all().filter(
	election_id__in=election_group_ids
	)
	if len(election_group_ids) != election_groups.count():
	raise CommandError(
	f"Expected {len(election_group_ids)} election groups with ids {election_group_ids} but found {election_groups.count()}"
	)
	elections = []
	for eg in election_groups:
	elections.extend(eg.get_children("public_objects").all())
	return elections

You can use this instead of running the ad-hoc script in the code block above. It can take one or more election_ids.

example:
python manage.py assign_election_to_divisionset local.broxtowe.2023-05-04 local.broxtowe.2024-05-02 794