v12

FEATURES:
MET-5763 Add support for stl files in media processing (#665)
MET-5764 Add support gltf files for media processing (#666)
MET-5796_MET-5803 rework register user (#664)
MET-5817 Create object for conversion record to fullbean (#660)
MET-5818 Store new 3D object mongo (#662)
MET-5756 implement new media processor subclass (#652)
featMET-5556 update xsd schemas jibx bindings tier calculations (#641)
MET-5619 update morphia record dao core lib entities (#637)
MET-5620 update morphia record redirect (#635)
MET-5619_update_morphia_record_dao (#634)
MET-5517 add new url support for euscreen embeddable media (#631)
MET-5566 update metis morphia library part 2 (#629)
MET-5757 Integrated new Media Processor (#654)
MET-5633 add new tier values for europeana enrichment (#648)
MET-5619 add address entity to morphia
MET-5619 add core lib metainfo embbeded entities (#638)

DEBT:
MET-5412 redirection case unit test (#630)
MET-5132 dates normalization cleanup part 4 (#622)
MET-5237 update morphia dependency (#625)
MET-5814 Remove socks proxy support (#659)
MET-5701 Upgrade metis-schema to 10-SNAPSHOT
MET-5632 update tier representation in mongo (#645)
MET-5474: Update to jdk21 build (#643)
MET-5579 Update redisson version 3.15.5 -> 3.24.3
MET-5473 upgrade to jdk17 (#640)
MET-5473: Upgrade to jdk 21 builder (#639)
MET-5460: metis-normalization submodule migration
MET-4463 update oai library (#626)
MET-4976 upgrade to spring boot 3 and framework 6 (#663)
MET-5469 update docker java version (#627)

BUGS:
MET-5761 fix content tier missing in solr documents (#650)
MET-5695 Tika misdiagnoses model files (#649)
MET-5648 Fix NullPointerException Tier Calculation (#642)
MET-5875: normalize URI to avoid '//' in dereference client.
MET-5875: catch correct exception for HttpGet and URL instantiations (#667)
MET-5777 closing httpClient in XsltTransformer (#656)
MET-5777 closing mimeTypeDetectHttpClient in MediaExtractorImpl (#653)
MET-5880 Fix user name not displaying when a user starts a workflow (#668)
MET-5697: redirection does not pick up predecessor if it happened bef… (#646)
MET-4985_valid_eCloud_tasks_marked_failed_in_Metis (#636)
MET-5401 spaces in dir names are replaced (#633)
MET-5553: Update cors to support array (#632)

MISCELLANEOUS:
Merge branch 'release/v12'
Set release version 12
Fixing code conflicts for release
Fix issue of non-required parameters in metis-repository
Fix hibernate query issues
Update elastic apm
Fix compile issue on metis-core-service.
Added missing line of code in common mongo RecordDao
Sync corelib dependencies in metis-enrichment
Add docker run configuration
Revert "MET-5473: Upgrade to jdk 21 builder (#639)"
Update spring run configuration properties
Set new snapshot version 12-SNAPSHOT

v11

FEATURES:
MET-5318 Accept .tar type files (#614)

DEBT:
MET-5381 All problem patterns are level warning (#624)
MET-5361 Capture tier calculation results for sandbox (#617)
MET-5392: Make WebResourceWrapper comparable and getFileByteSize (#620)

BUGS:
MET-5369 isShownBy isShownAt disappearing (#621)
MET-5110 fix entity status when no entity found for existing vocabulary II (#616)
MET-5054 fix connectivity dereference from WARNING to ERROR types (#613)
MET-5110_fix_entity_status_when_no_entity_found_for_existing_vocabulary (#611)
MET-4860 remove sonar notification for NullPointerException (#608)
MET-5392: Fix incorrect applied ECloudConfig
MET-5392: Fix mongoClient configuration conflict

MISCELLANEOUS:
Merge branch 'release/v11'
Set release version 11
MET-5439 add precedence from content-type over resource name (#623)
MET-5392 add package visibility for media processing enhancement process (#619)
MET-5054 report warning as error part II (#615)
Updated default_transformation xslt file
Make md5hex public
Reprocessing 2023 updates (#618)
Set new snapshot version 11-SNAPSHOT

v10

FEATURES:
MET-5155 Media processing is robust for missing size field (#607)
MET-4872 Solr and mongo contract implementation first review (#606)
MET-5115 Migrate Metis repository to Kubernetes (#604)
MET-5114 Migrating Metis Repository to Spring Boot (#602)

DEBT:
MET-4997 Dates normalization refactor part 3 (#601)

BUGS:
MET-5112 Fix first and last day of date (#610)
MET-5155 Fix Media Processing Robust for missing field (#609)
MET-5119 Intermittent enrichment error (#605)

MISCELLANEOUS:
Update .run configurations and fix metis-repository .gitignore
Create root module for metis-repository to be more consistent
Update metis-schema version to 8-SNAPSHOT

Dereference & enrichment services improvement features, Spring boot, Docker, Kubernetes services support, date pattern improvements

FEATURES:
MET-2018 Report error mechanism for dereference and enrichment.
MET-4823 Show throttling level in the current workflow and workflow history.
MET-4846 Prevent records tier "zero" from being published in metis.
MET-4864 Use spring boot framework in metis authentication service.
MET-4865 Support docker and kubernets for metis-authentication service.
MET-4866 Support docker and kubernetes for metis dereference service.
MET-4929 Use spring boot framework in metis dereference service.
MET-4930 Use spring boot framework in metis core service.
MET-4977 Coreferencing and Dereferencing for organisations.
MET-4984 Support docker metis core service.
MET-5032 Sketchfab 3d models as embeddable media support.

BUGS:
MET-4865 Fixed build on CustomObjectMapperTest, apm configuration loading, spring boot maven plugin version conflict.
MET-4841 Fixed normalization issue with invalid value for MonthOfYear.

MISCELLANEOUS:
MET-4657 Call enrichment APIs directly instead of via Enrichment REST service.
MET-4723 Date pattern decade date extractor cleanup.
MET-4732 Date pattern numeric date extractor with missing parts date extractor cleanup.
MET-4765 Date pattern DCMI period date extractor cleanup.
MET-4766 Date pattern century date extractor.
MET-4767 Date pattern clean operation and cleaner regular expressions.
MET-4785 Robust creation of temporary ID for zip file harvesting.
MET-4832 Date pattern numeric date range extractor with missing parts date extractor.
MET-4892 Mongo pool size configurable.
MET-5002 Decomission enrichment and validation services.

Media processing throttling, new date normalization, lookup for specific records and warn content tier zero records

FEATURES:
MET-4417 Implement warning if any records have tier 0.
MET-4564 Implement Media Processing Throttling levels.
MET-4656, MET-4711 Implement and integrate new date normalization.
MET-4669 Download records from error report.
MET-4672 Lookup/request a specific record by eCloudID.
MET-4673 Lookup/request a specific record by europeanaID.
MET-4720, MET-4761 New date normalization data is saved during indexing.

BUGS:
MET-4220 Fix image magic to clean temporary directories and prevent uses all disk space.
MET-4705 Fix Problem pattern P1 to detect titles that are identical ignoring case.

MISCELLANEOUS:
MET-4614 Unification of ZipFileReader and CompressedFileExtractor.

Problem patterns detection, Tier calculation on 3D types, Alignment of depublication status and securing dereference service.

FEATURES:
MET-4622 Support for tier recalculation based on 3D types.
MET-4472, MET-4473 Support new 3D tier calculation.
MET-4449 Implementation problem patterns p2 p6 detection.
MET-4457 Implementation problem patterns p3 p5 p7 p9 p12 detection.
MET-4458 Implementation p1 problem pattern detection.
MET-4470 Generify execution point for problem pattern service.
MET-4285 Index- new geospatial fields.
EA-2890 Semantic Enrichment entity client implmentation.

BUGS:
MET-4632 Fix Geocoordinates validation.
MET-4452 Fix metis depublication status after incremental processing.

MISCELLANEOUS:
MET-4440 Unit tests for SolrDocumentPopulator.
MET-4437 Separate metis-schema library into a separate repository.
MET-4612 Technical debt.
MET-2608 Improve dereference cache.
MET-4375 Increase code coverage in metis-repository module.
MET-4418 Changing name Macedonia for North Macedonia.
MET-4240 Improvement of pattern analysis service.
MET-4237 Align depublication status of datasets.
MET-4374 Secure regex backtracking in dereference service.
MET-4233 Secure Dereference service from url request attacks.
MET-4250 Fixing unit tests due MockMaker.
MET-4257 Remove functionality related to alternative indexing environment.
MET-4310 Reduce connection pool on mongo clients.
MET-4159 Updated Http Harvesting to process asynchronously.

Tier calculation representation and logging security vulnerability fix

FEATURES:
MET-4065 Allow non uris to be passed as keys to the xslt transformer
MET-4156, MET-4157, MET-4158, MET-4196 Implementation for tier calculation representation
MET-4175 Introduced harvesting limit number of records
MET-4094 Added parameter on validation plugins for the generation of statistics
MET-4154 Enable http incremental harvesting
MET-4007 Update to handle total number of records in indexing plugin
MET-3953 Refactoring metis harvesting library for better integration with metis sandbox

BUGS:
Fix null pointer choice list from proxy aggregation
Bug reindexing is not an incremental workflow
MET-4089 Fix tier heuristics for web resources and their license
MET-4246 Fix error deserializing task progress
MET-4037 Fix Media processing not extracting metadata
MET-3782 Retry badly applied around RemoteEntityResolver methods

MISCELLANEOUS:
Added github actions for builds, code style and contributing covenant
Removal of travis
Fix c3p0 dependency mismatch.
MET-4106 Upgrade logging framework versions to fix security vulnerability
MET-4241 Deprecate data-checker module
MET-4200 Increase unit test coverage metis common module
MET-4057 Fix of unit tests from Transformation
MET-4004 Removed any references to commons-lang library
MET-3021 Change class MetisUser name convention
MET-3965 Sonar issues fixes

Incremental processing, organization enrichment, dependencies updates, xsd merging

FEATURES:
MET-3902 Update proxy with new ecloud method with deleted records filtered
MET-2214 Add new counters to execution progress and indexing pre deletion count method
MET-3748, MET-3811 Metis repository implementation
MET-3130 Provide incremental indexing parameters
MET-3750 Create indexing library method for retrieving record IDs
MET-3707 Manage new task state for post processing
MET-3731 Extend metis indexing library to support deleting records
MET-3496 Add support for enrichment entities cleanup
MET-3747 Update indexing of organization fields
MET-3671 Add support for organization indexing
MET-3480 Maven library updates
MET-3403 Add support for organization enrichment
MET-2215 Add incremental parameter for ecloud
MET-3659 Include header in result of OAI GetRecord request
MET-3492 Enable XSL version 3 support for dereferencing
MET-3163 Merge xsds to one location to create one source of truth
MET-3478 Update zoho library com.zoho.crm.java-sdk 2.0.4 -> 3.0.1
MET-3404 Improve null check and equality for aggregation values

BUGS:
MET-3922 Fix view in collections when only deleted records exist
MET-3921 Fix workflow progress when only deleted records exists
MET-3807 Upgrade to c3p0 hibernate connection pooling
MET-3841 Counters calculation fix on execution progress
MET-3834 Fix missing incremental value on the view classes
MET-2786 Correct MP4 type if a video channel was not found
MET-3780 Fix pagination in zoho access client
MET-3626 Fix IndexerPool connection leak
MET-3675 Use closing cleaning http client for dereferencing.
MET-3667 Align XML names with the official definition
MET-3661 Null Pointer Exceptions during enrichment fixed
MET-3422 Could not Unmarshall bug fixed
MET-3762 Fix a performance issue with URL hashcodes
MET-3500 Solve issue with dereferencing multiple links
Small fix for zip upload in metis repository
When purging dereference cache, set right multi option
Solve possible NPE for certain dereferencer configurations

MISCELLANEOUS:
MET-3828 Update deprecated morphia annotations
MET-3813 Exclude spring-boot from corelib to fix statistics dashboard worker jar logging
MET-3690, MET-3544 Technical debt. Removed code smells

XX century dereferencing hotfix

This hotfix contains the following changes with respect to version 4:

1. A performance issue with URL hashcodes.
2. Support for XSL version 3 for dereferencing (part of MET-3492).
3. A bug where not all vocabularies are deleted before loading them.

XX century project support, technical debt, databases and dependencies upgrades

FEATURES:
MET-3505 Redis upgrade and addition of more settings parameters
MET-3531 Add common aggregation webresources to provider aggregation only
MET-3354 Add indexing flag for tier calculation
MET-3509 Oai harvesting simplification
MET-3504 Add rabbitmq configuration properties to support truststore properly
MET-3448 Review indexing according to API for new provider aggregation
MET-3477 Update mongo morphia versions
MET-3419, MET-2812 Use new configuration for mongo application name indicator
MET-2707 Multiple maven dependencies version upgrades
MET-2215 Incremental OAI harvesting option
MET-3240 Updated default transformation.xslt
MET-3290 Add lineage to proxy schema and dataProvider and provider for EuropeanaAggregation
MET-3405 Indexing support for ore lineage
MET-3242 Allow for multiple proxies and aggregations
MET-3239 Add generatedBy confidence

BUGS:
MET-3556 Fix dataset creation 500 error
MET-3245 Bad Request without Entity Type in enrichment service

MISCELLANEOUS:
MET-3479 Change exception from Enrichment Service to simplify test dependencies
MET-3499 Update CountryModel and LanguageModel to View
MET-3451 Hibernate configuration update
MET-3415 Fix on the fly transformation discrepancy
MET-3406 Technical debt
MET-3282 Clean web resource meta info
MET-3315 Move harvesting code to Metis
MET-3306 Unit tests metis enrichment service
MET-3303 Unit tests enrichment client
MET-3305 Unit tests enrichment rest
MET-3274 Support ffprobe4
MET-3318 Media result should stop IO
Redis db migration and upgrade
Mongo db migration and upgrade
Postgresql db migration and update