Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement]: Improve worker message passing to reduce memory leak #41845

Closed
HindujaB opened this issue Dec 7, 2023 · 2 comments · Fixed by #41939
Closed

[Improvement]: Improve worker message passing to reduce memory leak #41845

HindujaB opened this issue Dec 7, 2023 · 2 comments · Fixed by #41939
Assignees
Labels
Lang/Workers Workers and Worker Interaction related issues Team/jBallerina All the issues related to BIR, JVM backend code generation and runtime Type/Improvement

Comments

@HindujaB
Copy link
Contributor

HindujaB commented Dec 7, 2023

Description

$subject

Describe your problem(s)

The following code which passes 10KB json data between 3 workers increases the memory usage over time.
The heap dump shows a large count of WorkerDataChannel objects.

import ballerina/time;

// json payload = some 10KB json value
public function main() returns error? {
    time:Utc utc = time:utcAddSeconds(time:utcNow(), 1800); // 30 mins running
    while (utc >= time:utcNow()) {
        check testWorkerMessaging();
    }
}

isolated function testWorkerMessaging() returns error? {

    worker w1 returns error? {
        json res1 = clientFunc1(payload);
        res1 -> w2;
    }

    worker w2 returns error? {
        json res1 = check <- w1;
        json res2 = clientFunc2(res1);
        res2 -> w3;
    }

    worker w3 returns error? {
        json res2 = check <- w2;
        json res3 = clientFunc3(res2);
        res3 -> function;
    }

    json res3 = check <- w3;
    check wait w3;
    return ();
}

type Rec record {|
    int i;

|};

isolated function clientFunc1(json payload1) returns json {
    return payload1;
}

isolated function clientFunc2(json payload1) returns json {
    return payload1;
}

isolated function clientFunc3(json payload1) returns json {
    return payload1;
}

Memory graph :
Screenshot 2023-12-07 at 16 49 09

Heap dump :
Screenshot 2023-12-07 at 16 48 59

Heap dump file :
https://drive.google.com/file/d/1dXK7C2tZiPxXkHsfhEYhXgRaG7y_Datw/view?usp=sharing

Describe your solution(s)

Clean up the channels & channel details.

Related area

-> Runtime

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

@HindujaB HindujaB added Type/Improvement Team/jBallerina All the issues related to BIR, JVM backend code generation and runtime Lang/Workers Workers and Worker Interaction related issues labels Dec 7, 2023
@HindujaB HindujaB linked a pull request Jan 26, 2024 that will close this issue
13 tasks
@HindujaB
Copy link
Contributor Author

After removing the redundant WorkerDataChannel and ChannelDetails instances, the memory usage is reduced to the following.
Memory graph :
Screenshot 2024-01-29 at 11 34 12

Heap dump :
Screenshot 2024-01-29 at 11 34 22

@HindujaB
Copy link
Contributor Author

HindujaB commented May 8, 2024

This is now available in 2201.9.0.

@HindujaB HindujaB closed this as completed May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Lang/Workers Workers and Worker Interaction related issues Team/jBallerina All the issues related to BIR, JVM backend code generation and runtime Type/Improvement
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant