Skip to content

bug: EtcdServiceRegistry lease renewal task doesn't handle exceptions #246

@sfloess

Description

@sfloess

Bug Description

The scheduled lease renewal task doesn't handle exceptions properly. If renewLeases() throws an exception, the ScheduledExecutorService stops scheduling future executions, causing all leases to expire.

Location

jplatform-registry-etcd/src/main/java/org/flossware/jplatform/registry/etcd/EtcdServiceRegistry.java:84-85

Problematic Code

leaseRenewalExecutor.scheduleAtFixedRate(this::renewLeases,
    config.getLeaseTtl() / 2, config.getLeaseTtl() / 2, TimeUnit.SECONDS);

From ScheduledExecutorService documentation:

"If any execution of the task encounters an exception, subsequent executions are suppressed."

Impact

  • Single exception in renewLeases() stops all future lease renewals
  • All service registrations expire from etcd
  • Services become undiscoverable
  • Silent failure - no indication that renewal stopped

Example

registry.start();
registry.registerService(MyService.class, impl);

// Later, renewLeases() throws an exception (etcd connection issue)
// Task is cancelled, no more renewals happen
// After leaseTtl seconds, service disappears from etcd
// No error logged, no notification

Proposed Fix

leaseRenewalExecutor.scheduleAtFixedRate(() -> {
    try {
        renewLeases();
    } catch (Exception e) {
        logger.error("Exception during lease renewal, will retry on next cycle", e);
    }
}, config.getLeaseTtl() / 2, config.getLeaseTtl() / 2, TimeUnit.SECONDS);

Alternative - use scheduleWithFixedDelay and reschedule on exception:

private void scheduleRenewal() {
    leaseRenewalExecutor.schedule(() -> {
        try {
            renewLeases();
        } catch (Exception e) {
            logger.error("Exception during lease renewal", e);
        } finally {
            scheduleRenewal();  // Always reschedule
        }
    }, config.getLeaseTtl() / 2, TimeUnit.SECONDS);
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions