AS400 Collector

[mjabascal10]

Describe the feature

Implement a fully local AS400 collector capable of reading server configuration from a single JSON file (local_storage/server.json), monitoring changes in real time, and executing ingestion tasks for each configured AS400 server.
The collector must support a clean lifecycle (install, run, uninstall), manage its own internal state, and react dynamically to configuration updates without requiring external services.

Use Case

The AS400 collector must:
• Load the configuration on startup
• Start ingestion tasks for each server
• Detect changes to the configuration file
• Restart ingestion tasks when the configuration changes
• Shut down cleanly when requested
• Install and uninstall without external dependencies

Proposed Solution

1. Local Configuration Handling
   ◦ Create local_storage/ during installation.
   ◦ Create an empty server.json if it does not exist.
   ◦ Load configuration using a dedicated FileOperations.loadConfig() method.
   ◦ Store server definitions in memory for ingestion.
2. Configuration Watcher
   ◦ Implement a hybrid watcher using:
   ▪︎ Java WatchService for real-time file events.
   ▪︎ A scheduled backup check to detect missed events.
   ◦ Provide a stop() method to terminate the watcher safely.
   ◦ Notify the system when configuration changes occur.
3. Ingestion Execution
   ◦ Use a scheduled executor pool to run ingestion tasks in parallel.
   ◦ Restart the executor pool whenever the configuration changes.
   ◦ Ensure tasks can be stopped cleanly during shutdown or uninstall.
4. Install Process
   ◦ Create required directories and files.
   ◦ No lock files, no encryption, no remote registration.
5. Uninstall Process
   ◦ Stop watcher and ingestion executors.
   ◦ Remove server.json and all files inside local_storage/.
   ◦ Ensure no background threads remain running.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change