PictureComparer 0.7.0

Release Date: 09.12.2024

This is the first stable release of PictureComparer, delivering new features, performance improvements, and significant codebase modernization.

What's New?

Added:

• New Interfaces and Classes:
  • Grouper interface and its implementation CRC32Grouper.
  • Processor class, which orchestrates the entire processing pipeline.
  • AdaptiveCache class: Implements an Adaptive Cache System using the Caffeine Cache library.
• Logging:
  • Integrated the SLF4J framework with Logback as the native implementation.
• Tests:
  • Added tests for algorithms and the new Processor class.

---

Changed:

• Algorithm Interface:
  • Refactored the Algorithm interface.
  • Adjusted all Algorithm implementations to align with the updated interface version.
• Core Components:
  • Updated FileOperator and FileVisitor to use the new logging system.
• Algorithms:
  - Refactored the PixelByPixel algorithm implementation.
  - Updated JavaDoc for the PerceptualHash algorithm.
• Documentation:
  • Updated pom.xml and README.md.

---

Removed:

• Removed Deprecated Components:
  • RecordProcessor, ImageRecord, and Record classes.
  • LoggingInterface and TinyLog support.
  • FileUtils interface.

---

Release Notes

• This version focuses on scalability, modularity, and modernizing the codebase.
• Performance improvements have been achieved via the Caffeine Cache integration.
• All major logging has been streamlined using SLF4J.
• Check out our new documentation for more details.

---

How to Upgrade

1. Ensure you are running JDK 21 or higher.
2. Update your dependencies to include the latest version of PictureComparer
3. Review the new Processor class and updated Algorithm interface to adapt custom implementation if necessary.

---

Picture Comparer v.0.6.2

Description

After a significant period of development and refinement, I have applied numerous code cleanups and optimizations to enhance the library's functionality and performance.

Changelog

FileOperator Rebuild

• Rebuilded FileOperator Class: The FileOperator class has been completely rebuilt to improve efficiency and maintainability.
• New Async Handling: Introduced a new method for handling asynchronous operations using CompletableFuture, providing a more robust and modern approach to asynchronous file processing.
• FileVisitor Class: Added a new FileVisitor class to facilitate traversing and processing files within directory trees more effectively.
• FileOperation Interface: Created the FileOperation interface to define various file-related operations, enhancing modularity and flexibility.
• FileValidator Class: Developed the FileValidator class for validating files, which includes the FilePredicate interface to ensure that only valid files are processed.
• Package Reorganization: Reorganized the packages to improve code structure and readability.

Predicates Rebuild

• Rebuilded FilePredicate Interface: The FilePredicate interface has been revamped for better usability and functionality.
• Rebuilded ImageFilePredicate Class: The ImageFilePredicate class has been updated to work seamlessly with the new FilePredicate interface, enhancing its effectiveness in file validation.

DCT Algorithms Optimization and Rebuild

• Quantifier Class for DCT: Added a new Quantifier class to handle quantization in the DCT (Discrete Cosine Transform) algorithm, improving precision and performance.
• Transformer Class for DCT: Introduced the Transformer class to facilitate DCT transformations, enhancing the algorithm's efficiency.
• Rebuilded DCT Class: The DCT class has been refactored into a facade, streamlining its interface and improving ease of use.

Algorithms Rebuild and ImageRecord Rebuild

• Rebuilded ImageRecord: The ImageRecord class has been updated for better functionality and integration with the new algorithms.
• Algorithm Interface: Added a new Algorithm interface to standardize algorithm implementations and facilitate the addition of new algorithms.
• PerceptualHash Class: Introduced the PerceptualHash class for perceptual image hashing, enabling more robust image similarity comparisons.
• PixelByPixel Class: Added the PixelByPixel class for pixel-by-pixel image comparison, providing an additional method for image analysis.
• Removed Obsolete Functions: Deleted outdated and unnecessary functions from the ImageRecord class to streamline its functionality.

RecordProcessor

• Created RecordProcessor Class: Developed the RecordProcessor class to handle the processing of records, including grouping by checksum and applying algorithms for duplicate extraction.

Documentation Updates

• README.md Update: Updated the README.md file to reflect the latest changes and provide clearer instructions and information about the library.
• LICENSE.txt Created: Added a LICENSE.txt file to define the terms under which the library is distributed.

Picture Comparer v0.5.1

Fixes applied:

• Pixel-by-pixel functon fixed.

Description:

New version of the library completely changes how it works. Now user have to specify order of operations. Those changes allow user to define more flexible approach and allow some more user created content (like new comparing algorithms etc.). The library in it's current state allow you to define your own function (as in ImageRecord class), group by your own grouping algorithms (in current version I used checksum grouping, which isn't ideal), or using FileOperator load all the files that you want.
Additionally if you want, you're allowed to compare not only images but also any different files (as long as you specify your own algorithms and groupings).

Note:

What's now?
Well for now library is in good shape to leave it as it is. In future however it's quiet likely that I will enhance it, add new algorithms, change grouping algorithm for something better, maybe enhance I/O operations. Probably add some threading policy (now I use both virtual threading and traditional threading.).

Changes:

• Added new FileOperator class inside IO package, which for now handles loading files.
• Added new DCT class inside Algorithms package, which allows use of DCT algorithm (Fast DCT Lee) in Perceptual Hash creation.
• Added new FilePredicate functional interface inside Comparator package, which is mandatory in current implementation, filters allow not-valid files. This interface is similar to Java's Predicate functional interface but throws IOException.
• Added new ImageFilePredicate, which is implementation of FilePredicate, that allows to filter all non-image files.
• Added new test class (FTest) which is simple implementation of the library.
• Moved Utils package from Structures package into project directory (src/).
• Modified tinylog.properties files.
• Modified ImageRecord class, changing a little bit createChecksum method, adding two private methods: loadImage, compareImage (which are used by processing functions) and also added two fields: Function <...,...> pHashFunction, pixelByPixelFunction (which are used as processing functions).
• Modified Record class, adding two methods: gropingByChecksum (which changes given Collection of File objects into map of Record sub-type objects), process (which process given Collection by map entries with given process functions).
• Deleted Comparer class.
• Deleted PictureComparer class.
• Delete both of old test classes.
• Updated .gitignore file.
• Updated README.md file.
• Updated pom.xml file, added TwelveMonkeys ImageIO enhancement, to allow loading different types of images using ImageIO class.

---

More Information about current release you can find in README.md file.
If you're interested in supporting this project (I don't know why, you could be q:), then you're welcome to post in discussion that hopefully is attached to this release.

Picture Comparer v0.5d

Description:

New version of the library completely changes how it works. Now user have to specify order of operations. Those changes allow user to define more flexible approach and allow some more user created content (like new comparing algorithms etc.). The library in it's current state allow you to define your own function (as in ImageRecord class), group by your own grouping algorithms (in current version I used checksum grouping, which isn't ideal), or using FileOperator load all the files that you want.
Additionally if you want, you're allowed to compare not only images but also any different files (as long as you specify your own algorithms and groupings).

Note:

What's now?
Well for now library is in good shape to leave it as it is. In future however it's quiet likely that I will enhance it, add new algorithms, change grouping algorithm for something better, maybe enhance I/O operations. Probably add some threading policy (now I use both virtual threading and traditional threading.).

Changes:

• Added new FileOperator class inside IO package, which for now handles loading files.
• Added new DCT class inside Algorithms package, which allows use of DCT algorithm (Fast DCT Lee) in Perceptual Hash creation.
• Added new FilePredicate functional interface inside Comparator package, which is mandatory in current implementation, filters allow not-valid files. This interface is similar to Java's Predicate functional interface but throws IOException.
• Added new ImageFilePredicate, which is implementation of FilePredicate, that allows to filter all non-image files.
• Added new test class (FTest) which is simple implementation of the library.
• Moved Utils package from Structures package into project directory (src/).
• Modified tinylog.properties files.
• Modified ImageRecord class, changing a little bit createChecksum method, adding two private methods: loadImage, compareImage (which are used by processing functions) and also added two fields: Function <...,...> pHashFunction, pixelByPixelFunction (which are used as processing functions).
• Modified Record class, adding two methods: gropingByChecksum (which changes given Collection of File objects into map of Record sub-type objects), process (which process given Collection by map entries with given process functions).
• Deleted Comparer class.
• Deleted PictureComparer class.
• Delete both of old test classes.
• Updated .gitignore file.
• Updated README.md file.
• Updated pom.xml file, added TwelveMonkeys ImageIO enhancement, to allow loading different types of images using ImageIO class.

---

More Information about current release you can find in README.md file.
If you're interested in supporting this project (I don't know why, you could be q:), then you're welcome to post in discussion that hopefully is attached to this release.

Picture Comparer 0.4.1d

Description:

Some unnecessary features.

New features:

• Extracted all old Comparer fields to Picture Comparer
• Old Comparer was changed to interface that shares some methods.
• This change was made to make it easier to create a new and indepentent comparers. And due to change old implementation of abstract - Comparer was deleted (unfunctional anymore).
• Extended use of Java Stream API, and some visibility changes.
• Added default implementation of log methods in LoggerInterface.
• Deleted unused type for Comparer,
• Deleted processedObjectCount because it was giving unexpected values (due to parallel stream),
• Comparer's _setUp() method throws now FileNotFoundException,
• Changed behaviour when a file doesn't exist,
• Fixed _setUp() method (eariler i forget about filePredicate and forget about adding first layer of files) (due to current system recursive search is more expansive than eariler),
• Made map implementation a little bit more readable,
• Fixed compare method,
• Used try-with-resources in ImageRecord,
• Added new Test (called DataSourceTest),
• Fixed old Test class.
• Consolidated a little bit some thing in setUp method.
• Done some describing work.
• Extracted FileVisitor to new inner class.
• Added recursive mode.
• Deleted unused statements.
• Readme update
• Errors now will be throwed to separated file.
• In log.txt you will see only simple informations about exceptions, but in error.txt however you'll see all the informations about that exception.
• Refactor of code:
• Deleted unused thorws,
• Integrated Checksum interface into Record class (Checksums'll be used only in this class so there is no need for interace really).
• Hotfixes applied

Picture Comparer 0.4d

Description:

Some unnecessary features.

New features:

• Extracted all old Comparer fields to Picture Comparer
• Old Comparer was changed to interface that shares some methods.
• This change was made to make it easier to create a new and indepentent comparers. And due to change old implementation of abstract - Comparer was deleted (unfunctional anymore).
• Extended use of Java Stream API, and some visibility changes.
• Added default implementation of log methods in LoggerInterface.
• Deleted unused type for Comparer,
• Deleted processedObjectCount because it was giving unexpected values (due to parallel stream),
• Comparer's _setUp() method throws now FileNotFoundException,
• Changed behaviour when a file doesn't exist,
• Fixed _setUp() method (eariler i forget about filePredicate and forget about adding first layer of files) (due to current system recursive search is more expansive than eariler),
• Made map implementation a little bit more readable,
• Fixed compare method,
• Used try-with-resources in ImageRecord,
• Added new Test (called DataSourceTest),
• Fixed old Test class.
• Consolidated a little bit some thing in setUp method.
• Done some describing work.
• Extracted FileVisitor to new inner class.
• Added recursive mode.
• Deleted unused statements.
• Readme update
• Errors now will be throwed to separated file.
• In log.txt you will see only simple informations about exceptions, but in error.txt however you'll see all the informations about that exception.
• Refactor of code:
• Deleted unused thorws,
• Integrated Checksum interface into Record class (Checksums'll be used only in this class so there is no need for interace really).

Picture Comparer 0.3.3.1f

Description:

Recursive comparing is live now!
Recursive comparing means that you can compare any files in sub directories that are supported by Comparer.
Some code optimization was added in this update.
Some log adjustments was added also.

New features:

1. Updated readme file.
2. Deleted manual suggestion for Garbage Collector.
3. Specific errors informations are now throwed to separate file (logs/error.txt)
4. In log.txt file now you will see when error is throwed and what kind of error you're encountering.
5. Code optimization: Delete of unused throws.
6. Integrated Checksum interface into Record class. Reason for that is that whole checksum part of lib will be used only in Record class, so there is no need for interface.
7. Added recursive mode.
8. Fixed _reset() method.

Information:

• This update was tested for dataset of around 13500 image files.

Picture Comparer 0.3.3f

Description:

Recursive comparing is live now!
Recursive comparing means that you can compare any files in sub directories that are supported by Comparer.
Some code optimization was added in this update.
Some log adjustments was added also.

New features:

1. Updated readme file.
2. Deleted manual suggestion for Garbage Collector.
3. Specific errors informations are now throwed to separate file (logs/error.txt)
4. In log.txt file now you will see when error is throwed and what kind of error you're encountering.
5. Code optimization: Delete of unused throws.
6. Integrated Checksum interface into Record class. Reason for that is that whole checksum part of lib will be used only in Record class, so there is no need for interface.
7. Added recursive mode.

Information:

• This update was tested for dataset of around 13500 image files.

Picture Comparer 0.3.3d

Version 0.3.3d

Description:

Files are now checked before mapping. Now files are eliminated if header of file is in bad format. When processing files now we use paraller streams instead of plain streams (Thanks to that now library is much faster). Now when using Comparer you can get two collections, one that holds all files and one that holds only duplicates. Usage of tinylog.

New features:

1. New file checking system. Using magic numbers instead of extension comparing.
2. Use of parallel streams, include change to thread-safe collections.
3. User can get collections that hold data.
4. Use of Tinylog. Now Tinylog will create log.txt file.

Picture Comparer 0.3.2a

Version: 0.3.2a

Features:

• New system of usage.
• Code optimatization,
• Option to reset Comparer class, so you can re-use object rather than creating new one..