File Modification And Merging

Introduction

Managing files often requires modifying and merging content dynamically. Whether you're updating logs, editing configuration files, or combining multiple directories, the file-worker library provides utilities to modify and merge files and directories efficiently.

Imagine you need to clean up log files by deleting specific lines, merge all Markdown files in a documentation folder, or chain file operations fluently. This library helps automate such operations with minimal, readable code.

Features Overview

• Modify file content line by line.
• Delete specific lines from a file by index.
• Fluent API support: Methods returning File or Path objects for easy method chaining.
• Extension-based concatenation: Merge all files with a specific extension in a directory (optionally recursive).
• Merge directories while handling conflicts intelligently.
• Perform bulk modifications on files.

Getting Started

For installation and dependencies, refer to the Getting Started guide.

---

Modifying and Concatenating Files with ModifyFileExtensions

The ModifyFileExtensions class provides methods for updating file content efficiently and merging multiple files.

Example: Modifying a File Line by Line (Fluent API)

The library now supports a Fluent API, allowing you to chain operations by returning the modified Path or File.

Path filePath = Path.of("sample.txt");

// modifyFileAndGet returns the Path, enabling further operations
Path modifiedPath = ModifyFileExtensions.modifyFileAndGet(filePath, StandardCharsets.UTF_8, (index, line) -> line.toUpperCase());

System.out.println("File content converted to uppercase at: " + modifiedPath);

Example: Concatenating Files by Extension

A common task is merging all files of a specific type within a directory (e.g., combining all .md or .log files).

File sourceDirectory = new File("src/main/resources/docs");
File resultFile = new File("target/combined-docs.md");

// Concatenates all .md files in the directory alphabetically into the result file
ModifyFileExtensions.concatenateFilesWithExtension(sourceDirectory, ".md", resultFile);

System.out.println("All markdown files merged successfully into: " + resultFile.getName());

Note: There is also a concatenateFilesWithExtensionRecursive method available if you need to search through subdirectories.

Example: Concatenating a Specific List of Files

If you already have a specific list of files, you can concatenate them directly:

List<File> filesToConcatenate = List.of(new File("file1.txt"), new File("file2.txt"));
File resultFile = new File("merged.txt");

ModifyFileExtensions.concatenateAll(filesToConcatenate, resultFile);

Unit Test Coverage

The ModifyFileExtensionsTest class verifies this functionality, including the new extension-based concatenation:

@Test
public void testConcatenateFilesWithExtension() throws IOException {
    // Arrange
    File testDir = new File("test-dir");
    testDir.mkdirs();
    LineAppender.appendLines(new File(testDir, "a.txt"), new String[]{"Line 1"});
    LineAppender.appendLines(new File(testDir, "b.txt"), new String[]{"Line 2"});
    File resultFile = new File("result.txt");

    // Act
    File returnedFile = ModifyFileExtensions.concatenateFilesWithExtension(testDir, ".txt", resultFile);

    // Assert
    assertEquals(resultFile, returnedFile);
    String actual = ReadFileExtensions.fromFile(resultFile);
    assertEquals("Line 1\nLine 2\n", actual); // Files are processed in alphabetical order
    
    // Cleanup
    DeleteFileExtensions.delete(testDir);
    DeleteFileExtensions.deleteFile(resultFile);
}

---

Deleting Specific Lines with DeleteLinesByIndexInFile

The DeleteLinesByIndexInFile class allows you to remove specific lines from a file by leveraging the modification engine.

Example: Deleting Lines by Index

List<Integer> linesToDelete = List.of(1, 3); // 0-based or 1-based depending on implementation specifics
FileChangeable deleter = new DeleteLinesByIndexInFile(linesToDelete);

ModifyFileExtensions.modifyFile(Path.of("data.csv"), deleter);

System.out.println("Selected lines deleted from the file.");

Unit Test Coverage

The DeleteLinesByIndexInFileTest class ensures correct behavior:

@Test
public void testApply() throws IOException {
    Path filePath = Path.of("test-data.csv");
    List<Integer> linesToDelete = Arrays.asList(1, 4);

    ModifyFileExtensions.modifyFile(filePath, new DeleteLinesByIndexInFile(linesToDelete));

    List<String> lines = ReadFileExtensions.readLinesInList(filePath);
    assertFalse(lines.contains("Line 2")); // Assuming 1-based index deletion removed it
}

---

Merging Directories with MergeDirectoryExtensions

The MergeDirectoryExtensions class helps combine multiple directories into a single target directory. It offers both a simple timestamp-based merge and advanced, strategy-driven merging.

Example 1: Simple Merge (Newest File Wins)

By default, the library resolves conflicts by keeping the most recently modified file.

File targetDir = new File("target/mergedDir");
File dir1 = new File("source/backup_2023");
File dir2 = new File("source/backup_2024");

// Uses the default "newest file wins" logic
File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, dir1, dir2);

Example 2: Strategy-Driven Merge (Fluent API)

For more control, you can specify a MergeStrategy. This is particularly useful for synchronizing folders or cleaning up source directories after a successful merge.

import io.github.astrapi69.file.merge.strategy.MergeStrategy;

File targetDir = new File("target/master");
File sourceDir = new File("source/updates");

// TARGET_AS_MASTER: Copies new/changed files to target, then DELETES them from the source
File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, MergeStrategy.TARGET_AS_MASTER, sourceDir);

System.out.println("Synced and cleaned source directory. Target is at: " + resultDir.getAbsolutePath());

Understanding the Strategies

• TARGET_AS_MASTER:

  • If a file is only in the source: Copy to target, then delete from source.
  • If a file is only in the target: Do nothing.
  • If a file exists in both with different content: Overwrite target with source, then delete from source.
  • If content is equal: Leave target unchanged, delete from source. (Ideal for "Sync and Cleanup" operations)

• SOURCE_TO_TARGET:

  • If a file is only in the source: Copy to target.
  • If a file is only in the target: Do nothing.
  • If a file exists in both with different content: Overwrite target with source.
  • If content is equal: Leave target unchanged. (Ideal for "Backup or Update" operations without modifying the source)

Unit Test Coverage

The MergeDirectoryExtensionsTest class ensures both default and strategy-based merging work correctly:

@Test
public void testMergeAndGet_WithTargetAsMasterStrategy() throws IOException, InterruptedException {
    // Arrange
    File targetDir = Files.createTempDirectory("target").toFile();
    File sourceDir = Files.createTempDirectory("source").toFile();

    File sourceFile = new File(sourceDir, "config.txt");
    StoreFileExtensions.toFile(sourceFile, "new config data");
    
    File targetFile = new File(targetDir, "config.txt");
    StoreFileExtensions.toFile(targetFile, "old config data");

    // Act
    File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, MergeStrategy.TARGET_AS_MASTER, sourceDir);

    // Assert
    assertEquals(targetDir, resultDir);
    assertEquals("new config data", ReadFileExtensions.fromFile(targetFile)); // Target was updated
    assertFalse(sourceFile.exists()); // Source file was deleted as per strategy
    
    // Cleanup
    DeleteFileExtensions.delete(targetDir);
    DeleteFileExtensions.delete(sourceDir);
}

---

Merging File Content with MergeFileExtensions

While ModifyFileExtensions handles simple concatenation (appending files), the MergeFileExtensions class provides intelligent merge operations that go beyond simple concatenation. It supports deduplication, sorting, and various merge strategies.

Merge Strategies

The MergeFileStrategy enum defines how files are combined:

• APPEND: Simple concatenation, no processing.
• UNIQUE_LINES: Removes duplicate lines, preserves order of first occurrence.
• SORTED: Sorts all lines alphabetically.
• SORTED_UNIQUE: Removes duplicates AND sorts alphabetically.

Example: Merging Two Files with Deduplication

File file1 = new File("users_2023.txt");
File file2 = new File("users_2024.txt");
File result = new File("all_users_unique.txt");

// Merges and removes duplicate lines
MergeFileExtensions.mergeAndGet(file1, file2, result, MergeFileStrategy.UNIQUE_LINES);

System.out.println("Unique users merged into: " + result.getName());

Example: Merging Multiple Files (Fluent API)

List<File> logFiles = List.of(
    new File("logs/jan.log"),
    new File("logs/feb.log"),
    new File("logs/mar.log")
);
File merged = new File("logs/combined_sorted.log");

// Merges, deduplicates, and sorts all log entries
Path resultPath = MergeFileExtensions.mergeAndGet(
    logFiles.stream().map(File::toPath).collect(Collectors.toList()),
    merged.toPath(),
    MergeFileStrategy.SORTED_UNIQUE
);

System.out.println("Merged log available at: " + resultPath);

Unit Test Coverage

@Test
public void testMergeAndGet_UniqueLines() throws IOException {
    // Arrange
    File file1 = Files.createTempFile("f1", ".txt").toFile();
    File file2 = Files.createTempFile("f2", ".txt").toFile();
    File result = Files.createTempFile("result", ".txt").toFile();

    StoreFileExtensions.toFile(file1, "apple\nbanana\napple");
    StoreFileExtensions.toFile(file2, "banana\ncherry");

    // Act
    File returned = MergeFileExtensions.mergeAndGet(file1, file2, result, MergeFileStrategy.UNIQUE_LINES);

    // Assert
    assertEquals(result, returned);
    String content = ReadFileExtensions.fromFile(result);
    // Duplicates removed, order preserved: apple, banana, cherry
    assertTrue(content.contains("apple"));
    assertTrue(content.contains("banana"));
    assertTrue(content.contains("cherry"));
    assertEquals(1, content.split("apple").length - 1); // "apple" appears only once

    // Cleanup
    DeleteFileExtensions.deleteFile(file1);
    DeleteFileExtensions.deleteFile(file2);
    DeleteFileExtensions.deleteFile(result);
}

---

Conclusion

The file-worker library provides robust, fluent, and highly customizable tools for modifying files, deleting specific content, concatenating files by extension, and merging directories efficiently. Whether you're processing logs, editing data, or managing large file structures, this library simplifies file handling while maintaining academic and professional code quality.

Start using it today to automate your file operations and enhance productivity!

---