close java after task completed

[dhanyweb]

i have many java defunct process when running task in loop, and cause cpu usage and memory issues, any methode to close java after task completed?

58802 ? Z 0:22 [java]
58892 ? Z 0:22 [java]
58999 ? Z 0:24 [java]
59127 ? Z 0:37 [java]
59234 ? Z 0:32 [java]
59303 ? Z 0:29 [java]
59441 ? Z 0:36 [java]

[jxmorris12]

Well, @dhanyweb -- the quick fix is to run the following command: killall -9 java. That should kill all of the processes named 'java' (but be warned, if you're running other Java processes, they'll get killed too).

But we should get to the bottom of this. You run Python scripts using language_tool_python, then stop the scripts, then the server remains running, right? This shouldn't happen, and is a bug.

[meet59patel]

Facing the same issue. My snippet that uses language_tool_python:

def foo(mylist):
    tool = language_tool_python.LanguageTool('en-US')
    matches = tool.check(mylist)
    return len(matches)

This snippet is used by some other functions and then a flask endpoint uses those in the end. Any way to programmatically terminate this Java process here itself when it's done evaluating? These processes remain there blocking a lot of memory until I terminate the Flask server. I want to terminate these right when this function is returned or request is served.

cc: @jxmorris12

[meet59patel]

Temporary fix:

os.system("ps aux | grep 'languagetool-server.jar' | awk '{print $2}' | xargs kill -9")

Added this call before my function returns.

[jxmorris12]

Thanks @meet59patel -- but what's the root cause? It seems super weird that your code leaves lots of instances of language_tool_python running.

[cdr-chakotay]

Hi,

I am also suffering under this.

My program calls language tool often in a loop and the Java processes are remaining in the memory after each iteration.
It consumed 32 GB of RAM yesterday.

Can't we force kill the process after returning ?

[jxmorris12]

@cdr-chakotay can you share your code with me? (just the loop part)

[cdr-chakotay]

Sure:

I start multiple processes via multiprocessing in order to speed up processing. (That is my loop):

def generateFittingVectorFile(sampleSize: int, iterations: int):
    results = []
    if __name__ == "__main__":
        print("Start time: {:.0f}".format(time.time()))
        cpus = multiprocessing.cpu_count()
        maxWorkers = 1
        if cpus > 2:
            maxWorkers = cpus - 1
        if maxWorkers > 8:
            maxWorkers = 8
        with futures.ProcessPoolExecutor(max_workers=maxWorkers) as e:
            for i in range(iterations):
                results.append(e.submit(VectorWorker, reader.p1, reader.p2, reader.p3, sampleSize, i, iterations))
            e.shutdown(wait=True)
            

The VectorWorker itself has multiple invocations of a specific function, which calls language tool. Those multiple invocations are necessary.
This specific function is the code which actually uses your module and, which is called by VectorWorker:

def countTypos( StringToCheck: str) -> tuple:

    tool_de = language_tool_python.LanguageTool('de-DE')
    # tool_en = language_tool_python.LanguageTool('en-US') not used
    tool_de.enabled_rules_only = True
    tool_de.enabled_rules = ['DOPPELUNG_MODALVERB', 'GERMAN_SPELLER_RULE', 'WEIS_ICH', 'MEIN_KLEIN_HAUS',
                             'DOPPELPERFEKT', 'EMPFOHLENE_GETRENNTSCHREIBUNG', 'DE_WIEDER_VS_WIDER',
                             'RAN_RUM_RAUF_REIN_RAUS_RUNTER', 'DE_WORD_COHERENCY']
                            
    matches = tool_de.check(StringToCheck)
    TypoCounter: int = len(matches)
    return tuple([TypoCounter, matches])

That's basically how it works.
Multiple Processes, which are calling language tool multiple times in parallel.
But Java never closes properly and fills up the RAM

[jxmorris12]

Hi @cdr-chakotay - thanks! This is super helpful. A couple questions:

1. Does the problem reproduce without opening multiple threads?
2. After the loop (the for i in range(iterations)) ends -- with the call to shutdown -- do the processes remain? Or is it just during the loop that there are extra processes created?

[jxmorris12]

@cdr-chakotay can you try changing the loop to this?

def countTypos( StringToCheck: str) -> tuple:
    with  language_tool_python.LanguageTool('de-DE') as tool_de:
        # tool_en = language_tool_python.LanguageTool('en-US') not used
         tool_de.enabled_rules_only = True
         tool_de.enabled_rules = ['DOPPELUNG_MODALVERB', 'GERMAN_SPELLER_RULE', 'WEIS_ICH', 'MEIN_KLEIN_HAUS',
                                  'DOPPELPERFEKT', 'EMPFOHLENE_GETRENNTSCHREIBUNG', 'DE_WIEDER_VS_WIDER',
                                  'RAN_RUM_RAUF_REIN_RAUS_RUNTER', 'DE_WORD_COHERENCY']
                            
         matches = tool_de.check(StringToCheck)
    TypoCounter: int = len(matches)
    return tuple([TypoCounter, matches])

[jxmorris12]

Hi @meet59patel @dhanyweb @cdr-chakotay -- with the latest release (2.6.3) this should be solved. You have two options:

1. Open language_tool_python in a context manager (with .. as) as shown above
2. Explicitly call tool.close() to kill the java process wherever you need.

Please feel free to reopen if you're still having issues!

[cdr-chakotay]

Hi @meet59patel @dhanyweb @cdr-chakotay -- with the latest release (2.6.3) this should be solved. You have two options:

1. Open language_tool_python in a context manager (with .. as) as shown above
2. Explicitly call tool.close() to kill the java process wherever you need.

Please feel free to reopen if you're still having issues!

The with .. as statement does not work for me. But tool.close() does the trick! Thank you very much. This speeds up everything!

[jxmorris12]

Great!