File Modification And Merging

Introduction

Managing files often requires modifying and merging content dynamically. Whether you're updating logs, editing configuration files, or combining multiple directories, the file-worker library provides utilities to modify and merge files and directories efficiently.

Imagine you need to clean up log files by deleting specific lines, merge all Markdown files in a documentation folder, or chain file operations fluently. This library helps automate such operations with minimal, readable code.

Features Overview

• Modify file content line by line.
• Delete specific lines from a file by index.
• Fluent API support: Methods returning File or Path objects for easy method chaining.
• Extension-based concatenation: Merge all files with a specific extension in a directory (optionally recursive).
• Merge directories while handling conflicts intelligently.
• Perform bulk modifications on files.

Getting Started

For installation and dependencies, refer to the Getting Started guide.

---

Modifying and Concatenating Files with ModifyFileExtensions

The ModifyFileExtensions class provides methods for updating file content efficiently and merging multiple files.

Example: Modifying a File Line by Line (Fluent API)

The library now supports a Fluent API, allowing you to chain operations by returning the modified Path or File.

Path filePath = Path.of("sample.txt");

// modifyFileAndGet returns the Path, enabling further operations
Path modifiedPath = ModifyFileExtensions.modifyFileAndGet(filePath, StandardCharsets.UTF_8, (index, line) -> line.toUpperCase());

System.out.println("File content converted to uppercase at: " + modifiedPath);

Example: Concatenating Files by Extension

A common task is merging all files of a specific type within a directory (e.g., combining all .md or .log files).

File sourceDirectory = new File("src/main/resources/docs");
File resultFile = new File("target/combined-docs.md");

// Concatenates all .md files in the directory alphabetically into the result file
ModifyFileExtensions.concatenateFilesWithExtension(sourceDirectory, ".md", resultFile);

System.out.println("All markdown files merged successfully into: " + resultFile.getName());

Note: There is also a concatenateFilesWithExtensionRecursive method available if you need to search through subdirectories.

Example: Concatenating a Specific List of Files

If you already have a specific list of files, you can concatenate them directly:

List<File> filesToConcatenate = List.of(new File("file1.txt"), new File("file2.txt"));
File resultFile = new File("merged.txt");

ModifyFileExtensions.concatenateAll(filesToConcatenate, resultFile);

Unit Test Coverage

The ModifyFileExtensionsTest class verifies this functionality, including the new extension-based concatenation:

@Test
public void testConcatenateFilesWithExtension() throws IOException {
    // Arrange
    File testDir = new File("test-dir");
    testDir.mkdirs();
    LineAppender.appendLines(new File(testDir, "a.txt"), new String[]{"Line 1"});
    LineAppender.appendLines(new File(testDir, "b.txt"), new String[]{"Line 2"});
    File resultFile = new File("result.txt");

    // Act
    File returnedFile = ModifyFileExtensions.concatenateFilesWithExtension(testDir, ".txt", resultFile);

    // Assert
    assertEquals(resultFile, returnedFile);
    String actual = ReadFileExtensions.fromFile(resultFile);
    assertEquals("Line 1\nLine 2\n", actual); // Files are processed in alphabetical order
    
    // Cleanup
    DeleteFileExtensions.delete(testDir);
    DeleteFileExtensions.deleteFile(resultFile);
}

---

Deleting Specific Lines with DeleteLinesByIndexInFile

The DeleteLinesByIndexInFile class allows you to remove specific lines from a file by leveraging the modification engine.

Example: Deleting Lines by Index

List<Integer> linesToDelete = List.of(1, 3); // 0-based or 1-based depending on implementation specifics
FileChangeable deleter = new DeleteLinesByIndexInFile(linesToDelete);

ModifyFileExtensions.modifyFile(Path.of("data.csv"), deleter);

System.out.println("Selected lines deleted from the file.");

Unit Test Coverage

The DeleteLinesByIndexInFileTest class ensures correct behavior:

@Test
public void testApply() throws IOException {
    Path filePath = Path.of("test-data.csv");
    List<Integer> linesToDelete = Arrays.asList(1, 4);

    ModifyFileExtensions.modifyFile(filePath, new DeleteLinesByIndexInFile(linesToDelete));

    List<String> lines = ReadFileExtensions.readLinesInList(filePath);
    assertFalse(lines.contains("Line 2")); // Assuming 1-based index deletion removed it
}

---

Merging Directories with MergeDirectoryExtensions

The MergeDirectoryExtensions class helps combine multiple directories into a single target directory. It offers both a simple timestamp-based merge and advanced, strategy-driven merging.

Example 1: Simple Merge (Newest File Wins)

By default, the library resolves conflicts by keeping the most recently modified file.

File targetDir = new File("target/mergedDir");
File dir1 = new File("source/backup_2023");
File dir2 = new File("source/backup_2024");

// Uses the default "newest file wins" logic
File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, dir1, dir2);

Example 2: Strategy-Driven Merge (Fluent API)

For more control, you can specify a MergeStrategy. This is particularly useful for synchronizing folders or cleaning up source directories after a successful merge.

import io.github.astrapi69.file.merge.strategy.MergeStrategy;

File targetDir = new File("target/master");
File sourceDir = new File("source/updates");

// TARGET_AS_MASTER: Copies new/changed files to target, then DELETES them from the source
File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, MergeStrategy.TARGET_AS_MASTER, sourceDir);

System.out.println("Synced and cleaned source directory. Target is at: " + resultDir.getAbsolutePath());

Understanding the Strategies

• TARGET_AS_MASTER:

  • If a file is only in the source: Copy to target, then delete from source.
  • If a file is only in the target: Do nothing.
  • If a file exists in both with different content: Overwrite target with source, then delete from source.
  • If content is equal: Leave target unchanged, delete from source. (Ideal for "Sync and Cleanup" operations)

• SOURCE_TO_TARGET:

  • If a file is only in the source: Copy to target.
  • If a file is only in the target: Do nothing.
  • If a file exists in both with different content: Overwrite target with source.
  • If content is equal: Leave target unchanged. (Ideal for "Backup or Update" operations without modifying the source)

Unit Test Coverage

The MergeDirectoryExtensionsTest class ensures both default and strategy-based merging work correctly:

@Test
public void testMergeAndGet_WithTargetAsMasterStrategy() throws IOException, InterruptedException {
    // Arrange
    File targetDir = Files.createTempDirectory("target").toFile();
    File sourceDir = Files.createTempDirectory("source").toFile();

    File sourceFile = new File(sourceDir, "config.txt");
    StoreFileExtensions.toFile(sourceFile, "new config data");
    
    File targetFile = new File(targetDir, "config.txt");
    StoreFileExtensions.toFile(targetFile, "old config data");

    // Act
    File resultDir = MergeDirectoryExtensions.mergeAndGet(targetDir, MergeStrategy.TARGET_AS_MASTER, sourceDir);

    // Assert
    assertEquals(targetDir, resultDir);
    assertEquals("new config data", ReadFileExtensions.fromFile(targetFile)); // Target was updated
    assertFalse(sourceFile.exists()); // Source file was deleted as per strategy
    
    // Cleanup
    DeleteFileExtensions.delete(targetDir);
    DeleteFileExtensions.delete(sourceDir);
}

---

Conclusion

The file-worker library provides robust, fluent, and highly customizable tools for modifying files, deleting specific content, concatenating files by extension, and merging directories efficiently. Whether you're processing logs, editing data, or managing large file structures, this library simplifies file handling while maintaining academic and professional code quality.

Start using it today to automate your file operations and enhance productivity!

---