All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
This version -------- v2.6.7
Previous version ---- v1.121.5
Initial version ----- v1.121.5
Total commits ------- 390
- change ech-0045 address mapping fallback logic
- assert duplicated persons with same ahvn13 in person import
- update ech-0045 mapping and alliance name for loganto subsystem.
- import doi address for "no correspondence" persons
- add whitespace validation for person entity
- prevent mapping of invalid circles to a person
- enable e-voter flag in ech-0045 export for Swiss abroad
- update test data to add support for Swiss abroad e-voters
- don't write bfs statistics during cobra import
- extend entity mapper to apply lifecycle data
- update district category mapping in innosolv importer.
- refactor import
- evoting registration dashboard
- add support for foreign delivery addresses
- allow to skip forwarding of e-voter flag by municipality id configuration
- extend filter input validation
- add relegion for swiss person type in ech-0045 export
- remove deprecated person signature
- extend person identification criteria in e-voting registration
- dependency and runtime patch policy
- use latest dotnet runtime v8.0.3
- re-introduce 'Ki' doi type for acl layer.
- remove unused doi type 'Ki'
- check for dublicated SourceSystemId in the importfile for all import types.
- unique constraint for SourceSystemId, SourceSystemName and VersionCount.
- API Authorization Integration Tests
- remove expression based implementation for calculated filters
- Update voting library to register dll import resolver for pkcs11 driver
BREAKING CHANGE: Updated service to .NET 8 LTS.
- update to dotnet 8
- apply patch policy
- database query monitoring
- add filter tenant name
- add missing district mappings in innosolv import.
- update entity validation for doi type 'OG' to max. length 50 according to eCH-0011 v8.1
- update license copyright
- Random Webservice with Mockdatagenerator at loganto and cobra
- remove deprecated canton address attribute
- remove deprecated house number suplement from kewr adapter.
- csv doi and person import errors
- add eCH from voting lib
- adjust log level for abraxas authentication values
- remove required statements in model builder, update parameter constraints.
- use separate port for metrics endpoint provisioning
- extended anonymization tool for test data
- static code analysis refinment
- add tests for all filter criteria
- update solution and projects to align with hexagonal architecture requirements
- update architecture tests (ArchUnit) to align with solution design refactoring.
- exclude ACL children in query to prevent conflicts in entity framework changetracker when updating entities that may have 0..* children set in the navigation property.
- decouple port for metrics endpoint from the default application port mapping
- apply service account scopes client credential flow
- import file tests
- add cobra mock generator
- exclude tools from bbt deployment
- map innosolv contact address from residence if no contact address is available
- add allowed person import source system config
- stream persons during rename filter version
- change evoting person nationality to country name short
- ignore acl for evoting person search
- extend continuous deployment pipeline with BBT and UAT jobs.
- updated ci-cd templates to v1.8.8
- extend filter version signature and verify before renaming
- remove enum check constraints
- add setting to disabled loganto and kewr for eVoting
- replace filter reference id with enum
- deleting a person creates a new version of the person
- add residence permit dates to innosolv mock generator
- is voting allowed only for valid move in arrival dates
- remove age from person model
- add contact address post office box number
- mock data generator target file for swiss abroads
- compress queued files with gzip due to huge size of xml files
- add innosolv file verify command
- innosolv mock data generator
- pre-assign person-doi ids to calculate correct signature on first import
- add innosolv support
- verify csv export signature of persons without a filter version
- verify ech export signature of persons without a filter version
- stream csv exports
- verify signature if exporting a filter version as csv
- Verify filter version signature in eCH exports
- update ci-cd templates to v1.8.1
- use postgres host for db resolution since issue https://gitlab.com/gitlab-org/gitlab-runner/issues/2229 is fixed
- remove deprecated job extensions
- updated ci-cd template version
- removed vulnerable library
System.Linq.Dynamic.Core
- activate strict policy for dependency check
- set started date same as created date for acl import
- set modified date when updated for acl import
- set service user name as default when no user scope is available
- extend 'nicht zustellen' rule nΒ°3 with additional municipality exception behavior.
- extend e-voting exception messages with additional subsystem details.
- Update lib dependency to deterministic version
- exclude deleted persons from bfs integrity
- secure code review security refinments
- adjust cryptographic naming conventions
- stream persons when creating filter versions
- map cobra house number addition to entities dwelling number
- validate dwelling number to max length of 10 according to eCH
- validate house number to max length of 12 according to eCH
- reduce lenght of contact address zip code for foreign codes to 15 according to eCH
- extend import statistics with AES cipher metadata crypto parameters.
- adjust naming for security operations (classes, methods, etc.).
- make AES cipher metadata independent from the cryptographic implementation itself.
- store AES and MAC attributes as separate cryptographic parameters on database.
- generate cryptographic attributes IV, AES key and MAC key using save random number generator.
- the MAC key is now encrypted by the HSM with AES before being saved on the database.
- Set SendVotingCardsToDomainOfInfluenceReturnAddress flag according to updated specification
- check dateofbirth DateOnly.MinValue for voting rights
- changed RulesForDates (RuleForDateOfBirthType)
- remove contact address house number addition attribute and filter
- signature breaking change
- add fallback strategy for eCh-0045 address assignment
- map residency valid from/to only for foreigners
- map alliance name and call name in ech export
- evaluate actual number of registered e-voters
- compare entities during import instead of comparing record against entity
- remove person reactivate logic
- if a person was soft deleted but reoccurs in the import, create a new revision instead of reactivating all old revisions
- store dois on person during import
- abort request when ech file streaming fails to notify the client/browser of the failure
- add voting place to ech exports
- add missing school circle filter references to calculated fields evaluation
- update library with AES-GCM hotfix
- protect against signature payload attribute shift vulnerability
- update eCH-0045 mapping rules and entity input validators.
- Secure code review refinments
also clean up age related code
also clean up the loganto and kewr services
- add filter metadata endpoint
- count invalid persons
- apply fallback mapping for language of correspondence in eCH-0045 export
- remove voting right criteria for canton
- use cached canton enabled evoters lookup instead of municipality bfs
- changed ProtoVersion
- changed integration test
- separate authentication and access control provisioning.
- npgsql command timeout setting
- configure bbt pipeline and quality gates
- person should only be hashed in bfs integrity
- disable person signature validation on filter creation
- server timing
- eVoting filter and sync
- verify signatures when creating filter versions
- Output detailed error messages (processing errors) only for the ImportObserver role
- last used parameters api
- TransactionUtil(deprecated) replaced with DbContext.Database.BeginTransactionAsync
- PersonLogantoMappingProfile
- PersonCobraMappingProfile
- CobraUtil
- LogantoUtil
- map contact address lines for swiss abroad according to the eai tool
- ensure that residence permits with length lower than 2 could be mapped
- RuleForSexType
- RuleForReligionType
- RestrictedVotingAndElectionRightFederation
- source system name added to PersonEntity
- Cobra Person import, Loganto Person and DOI import with source system name extended
- return id when creating new filter version id
- improve filter version creation ressource consumption by loading only required data once
- additionally validate municipality id against whitelist config in case of swiss foreign source system
- add gauge to monitor mutuaded datasets count
- add get single filter version api endpoint
- speed up import person query by disabling acl check and adding an index
- use database transaction instead of ambient transaction to fix weird timing bug where transaction was not yet fully committed after complete call
- don't override import statistics municipality id with null on import failure
- ensure metrics are initialized at startup to better distinguish between not initialized and no data
- update last updated on integrity update
- read person actuality from bfs integrities
- update SendVotingCardsToDomainOfInfluenceReturnAddress correctly
- add SendVotingCardsToDomainOfInfluenceReturnAddress field on person
- fixes a bug when creating and processing imports which occasionally crashes the service due to the transaction not fully completed when fetching and processing the import job
- optimize import statistics queries
Fixed four major bugs
- Refactor the containing loop to do more than one iteration. (src/Voting.Stimmregister.Adapter.Loganto/Services/BaseDomainOfInfluenceImportService.cs)
- Refactor the containing loop to do more than one iteration. (src/Voting.Stimmregister.Adapter.Loganto/Services/BasePersonImportService.cs)
- Remove the 'using' statement; it will cause automatic disposal of 'aes'. (src/Voting.Stimmregister.Core/Services/Supporting/Signing/AesStreamEncryption.cs)
- store tenant id on filter entity
- only store first two chars of the residence permit
- new entity validation for mandatory bfs number in acl import
- assign bfs identification number from person doi for eCH-0045 export
- remove religion from eCH-0056 export
- cobra import extended with nationality and municipality name
- extended existing unit tests
- avoid raw records in log when bad data detected
- include swiss abroad person property in grpc api response
- extend eCH-0045 export with additional authorization rules
- extended existing integration tests
- cleanup unused snapshots
- allow search with formatted social security number
- Various changes due to SonarQube code smell
- Sufix "Enum" removed from all enums
- fix(VOTING-3088): support formatted social security number filters
- nationality filter
- correct age filter operations on date of birth
- map domain of influences in eCH-0045 export
- change residence permission comparison so that only first two digits are checked
- only include persons from the last revision and not deleted for the eCH-0045 export
- fix voting right evaluation for filter versions
- add domain of influence entity validation for circle names to report invalid circle references
- enrich person with additional domain of influences
- return id when saving a filter
- include whole import data set in integrity checksum
- iclude import source and type into integrity entity to distinguish persons and dois
- rename integrity table to BfsIntegrities
- include filter references in query only if reqired for further processing
- ensure transaction safety
- configured detete behavior according to data model definition
- add unique constraint for filter criteria reference id
- implement crud for filter criteria
- ensure filter version can only be saved if user has access to passed filter
- sort filter versions by deadline descending
- adjust duplicated filter name
- replace person streaming methods to use PersonEntity instead of PersonEntityModel
- remove unused model mapping statements
- OrignName(1-7), OriginCanton(1-7) and all CircleId undCircleName fields from PersonDoi
- Remove "Alter"/"Age" from CSV export (decision by customer) (VOTING-2990)
- map audit info from job to integrity audit info
- remove audit info from doi and acl table since info is available on connected import statistic
- apply null substitution for nullable audit info attributes
- restrict access to statistic data by access control list and new import observer role
- add signature for domain of influences
- ensure already soft deleted persons are not deleted again
- set delete count to the grouped register id and not to every person version
- add reactivate count to datasets created count since this would reflect the number of delivered datasets compared to the history
- use transaction scope to bundle db updates and ensure rollback
- peek bfs from created file since resetting mulipart request stream position is not possible
- update voting right evaluation based on nationality
- update api authorization configuration
- read bfs from import file before queuing
- reject import when bfs not in acl for user if not service user
- modified audit info migration to cover null value constraints
- store municipality name on the import stats
- add manual importer role for cobra import endpoint
- remove unused role check functions
- Added authentication verification for file encryption
- add audit info
- load latest filter version for filters
- cleanup filter grpc and service implementation
- Unused userId field in FilterEntity
- remove deprecated import configs
- correct saving of filter criterias
- fixed ignored filter version and filter criteria mappings
- compare against mapped domain of influence circle id if doi reference is available
- added unit tests for voting right use cases
- update evaluation for canton voting right
- extend metrics
- use static diagnostics class for metrics registrations
- get canton by bfs number filter by doi type bfs.
- import statistics list only return latest for each municipality id
- refine and fix open todo's
- use utf8 as default encoding since import files switched from latin1 to utf8
- remove old import file model with static encoding default set since it was only used within tests
- Copyright 2022 to 2023
- Moved Entities from Abstractions.Adapter.Data to Domain
- remove import service since it has been replaced by rest endpoints
- set abx auth in importers if no authentication is present
- sort person domain of influences
π Added
- SonarQube rule for exceptions added
β Removed
- Removed the Serializable attribute and the constructor (SerializationInfo, StreamingContext) for all exceptions.
- set createdDate/createdBy for integrity entities
- Return domain of influences when querying latest single person
- check municipality id against acl on import
- fix import statistics table on data view
- change enum column to be stored as a string
- regenerate InitTables migration
- create InitConstraints migration and add existing and new constraints (i.e. CantonEnum, SexTypesEnum, ...)
- regenerate all db migrations
- canton validation logic
- set manual upload flag
- integration test for deleting filter version with version persons
- fix municipality id validation in Proto and set municipality id on failed imports
- rm HasProcessingErrors from ImportStatistics
- fix Cobra import IsUpdateRequired method for origin information
- problem was when there was no person DOI information in file and DB a new person version was created
- Use paging from voting.lib
- Person Entity with field countrynameshort extended
- ech replace residence entry date with move in arrival date
- split filter save version to create / rename version
- remove signature of PersonDoiEntity and calculate PersonDoi fields to PersonEntity signature
- technical debt: strict seperation between id and register id for person
- add rest endpoint for doi import
- refactor doi import
- add "language of correspondence" to Loganto import
- Added file encryption when saving temp file for import
- input validation on "origin" and on "on canton"
- fix concurrency issues in domain of influence to person mapping
- grab domain of influences once and use a dictionary for faster lookup for mapping
- fix ResidencePermitValidTill date comparison to use last day of month
- fix for mapping and comparison of circle information for person doi
- include filter criterias for filter version to display on the filter version overview
- version of VotingRegisterProtoVersion adjusted to 1.26.4
- remove unused religions
- ensure test result ordering
- integration tests for the changes
- normalize DOI information (circle id's & circle names & origin & oncanton)
- reimplement filters, mappings for origin, oncanton, circle ids and circle names
- adapt validations, mappings, import, export, change detection
- setup hsm configuration and drivers
- change image runtime from alpine to bullseye slim
- fix findings out of business rule and database review
- adjust
PostOfficeBox
naming convention according to eCH-0010 naming convention - update cobra mapping for
PostOfficeBox
- country-iso-helper-service implemention by loganto import
- loganto unittest (PersonImportServiceUnitTest; "WhenUserWithCountryImported_ShouldCreateNewVersion")
- PersonEntity with the field country short name extended
- apply acl post-processing impersonation during person and doi import
- check for control characters in record validation for loganto for one specific field which has no validation in source system
- filter operator and types as enums
- Enum columns
- Minor changes in voting rules
- Add Rules for voting right
- new rules for Swisscitizenship filter, test ResidencePermitValid date range when SwissCitizenship = No
- integration tests for Filter rules (https://confluence.abraxas-tools.ch/confluence/display/SREC/Filter+Rules+-+SSR)
- snapshots
- add filter save version integration tests
- only save name, modified date, modified by for update
- delete filter version persons on delete filter version
- add cobra person import
- add cobra import tests
- set swiss abroad flag statically for imports from cobra
- refactor person import common parts
- replace singleton MemoryCacheEntryOptions with config object
- Flag for canton is valid for voting rights
- Calculate age by reference key date
- add Bfs country iso helper service (BFSCountryList.xml)
- The BFS number of the country name is used to search for the ISO2 code, and returned.
- add Loganto country iso helper service (LogantoCountryList.xml)
- With the Loganto country code the ISO2 code is searched and returned with the short name.
- add filter count to cache
- remove filter count from database
- REST API to upload loganto person imports
- File System queue of loganto person imports
- change fluent validations for better validation messages
- adapt tests
- Add new mapping in EK import for canton abbreviation
- add indicator of whether the date of birth is valid for the right to vote
- Added signing of entities Person, Integrity, FilterVersions
- unit tests for PersonEntityValidator
- person import ech validation
- Return person validation errors
- add record identifier to RecordValidationErrorModel
- DB index and constraint added on (RegisterId, IsLatest) combination on table Persons
- add filter reference id for "HasValidationErrors" mapped to !IsValid
- unittest for person search (GetAll)
- integrationstest for person search
- rm unused classes
- use ef fluent api configurations
- set import user context on import statistics instead of static source system identifier
- remove unused field from import statistics table
- remove created, modified, deleted by fields from person entity since the reference to the latest importstatistic is sufficient
- set filter grpc service roles and adapt Integration tests
- read csv records async instead of reading them all into a list first
- fix typo in method signature
- move state modification operations to state model
- add dicionaries for faster existing person lookups
- expose state modification methods for privatly accesable state lists on the state class
- add transaction for db modifications
- only load latest versions for processing and soft delete persons versions at the end
- fix FK problem when deleting filter version
- fix query filter for filter criteria
- eCH contact address town as residenc address town fallback
- eCH should support foreigners
- add HasUtcConversion in FilterVersionBuilder and fix snapshots
- ech 0045 export
- Get always the latest people by person repository
- aligned import type id to proto exact enum
- update proto including circle id type changes from int to string
- disable test parallelization for tests writing into database
- Integration tests for filter grpc service
- fix unittests
- Result model for person by ID. Gets only the latest person data
- add person import historization
- modified Person Entity model according to new requirements
- updated Snapper and Newtonsoft to get DateOnly support
- Fixed SonarQube issues in Core, Adapter and Tests
- add municipality id to filter save and duplicate
- add pre calculated count field for filter version and populate it on save
- combined filter service / filter repository / person service / person repository for a working version
- add filter operator seed data
- fixed unit tests for FilterProfile and PersonProfile
- Calculate person search age always with day date
- Workaround logic for case insensitive person search string comparison
- added count to Filter table / entity / model
- add icu libs to runtime image to get culture support
- use valid culture modifier
- removed dependencies to Voting.Lib.Rest and VotingExports from domain layer
- fix error that filter version were not saved and loaded properly
- person search pagination validation
- add csv export for person filter versions
- add new exporter role
- update csv helper
- CRUD for filter version
- CRUD for filter criteria
- CRUD for filter
- added additional filter for people search
- Added query filters
Add latest version id to PersonModel for filter features.
- Clean SonarQube issues
Add GetByFilterId to IPersonService interface
- person search filter for age
- person search GetByFilterId
- some generic expression logic
- make acl synchronization configurable for grpc web or grpc
- handle further data types and operators for person search
- add optional evoter limitation on municipality level
- add single person search service with prepared version support
- add minimal person search logic
- resolve issue in e-voting unregistraiton model assignment.
- integrate eService into stimmregister service
- ignore path handling
- fix default cron-tab schedule syntax.
- add domain of influence based access control list importer.
- add access control list DOI synchronization with VOTING Basis.
- add access control list DOI security layer for domain of influence and person data sets.
- add request logging
- refactor import
- update nuget packages
- add person service including mock data
- introduce auto mapper
- fixed authorization
- fixed cert pinning
- person grpc service
- person service draft
- person grpc service test
- added database entities for person import
- removed coding examples
- architecture test project
- google protobuf package ref
- architecture tests
- remove unused packages
- try build twice
- update test package references
- update Google.Protobuf version to 3.19.1
- assembly Google.Protobuf explicitly
- additional swagger gen settings
- Integration test setup
- Unit tests for core module
- Renamed integration tests project
- Solution architecture tests
- Resolved SonarQube code smells
- use connection string builder for dummy connection string
- reviewed hard-coded credential which is okay here since it is only a dummy one used at design time
- CORS configuration support
- Loganto Adapter Init
- extend person identification criteria in e-voting registration
- dependency and runtime patch policy
- use latest dotnet runtime v8.0.3
- re-introduce 'Ki' doi type for acl layer.
- remove unused doi type 'Ki'
- check for dublicated SourceSystemId in the importfile for all import types.
- unique constraint for SourceSystemId, SourceSystemName and VersionCount.
- API Authorization Integration Tests
- remove expression based implementation for calculated filters
- Update voting library to register dll import resolver for pkcs11 driver
BREAKING CHANGE: Updated service to .NET 8 LTS.
- update to dotnet 8
- apply patch policy
- database query monitoring
- add filter tenant name
- add missing district mappings in innosolv import.
- update entity validation for doi type 'OG' to max. length 50 according to eCH-0011 v8.1