8gb memory used - how do reduce memory ?

[cameronbraid]

I've just installed weblate in kubenetes using the helm chart.

I initially didnt setup resource constraints (ram/cpu) to get it up and going quickly.

I then imported 2 translation files from i18next, totals of ~ 5k stings ~ 25k words.

Weblate is using 8gb of ram, which surprised me since the docs said it needs about 2gb

Have I misconfigured something ? Is this to be expected ?

I tried setting resource constraints to 2gb ram, but the OOM killer started killing processes which stopped weblate from starting up.

[github-actions]

This issue looks like a support question. We try to answer these reasonably fast, but in case you are looking for faster resolution, please consider purchasing support subscription and make Weblate stronger.

[cameronbraid]

There are a lot of celery processes running

weblate@weblate-7695794d85-jmr6f:/$ ps auxw | grep celery | wc -l
153

e.g.

weblate    1034  0.0  0.1 257924 118224 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1035  0.0  0.1 257872 118100 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1036  0.0  0.1 257880 118024 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1037  0.0  0.1 257928 118224 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1038  0.0  0.1 257876 118112 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1039  0.0  0.1 257932 118236 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1040  0.0  0.1 257884 118032 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1041  0.0  0.1 257936 118236 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1042  0.0  0.1 257880 118112 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1043  0.0  0.1 257888 118040 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1044  0.0  0.1 257940 118240 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1045  0.0  0.1 257884 118816 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1046  0.0  0.1 257944 118252 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1047  0.0  0.1 257892 118052 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1048  0.0  0.1 257948 118268 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1049  0.0  0.1 257888 118120 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1050  0.0  0.1 257896 118060 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1051  0.0  0.1 257952 118256 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname notify@%h --app weblate.utils --loglevel info --qu
weblate    1052  0.0  0.1 257892 118132 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1053  0.0  0.1 257900 118064 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1054  0.0  0.1 257896 118140 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1055  0.0  0.1 257904 118088 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1056  0.0  0.1 257900 118144 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1057  0.0  0.1 257908 118072 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1058  0.0  0.1 257904 118148 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1059  0.0  0.1 257912 118072 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1060  0.0  0.1 257908 118152 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1061  0.0  0.1 257916 118104 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1062  0.0  0.1 257912 118156 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1063  0.0  0.1 257920 118104 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1064  0.0  0.1 257916 118160 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1065  0.0  0.1 257924 118104 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1066  0.0  0.1 257920 118168 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1067  0.0  0.1 257928 118108 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1068  0.0  0.1 257924 118168 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1069  0.0  0.1 257932 118116 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1070  0.0  0.1 257928 118172 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1071  0.0  0.1 257936 118116 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1072  0.0  0.1 257932 118180 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1073  0.0  0.1 257940 118128 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1074  0.0  0.1 257936 118180 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1075  0.0  0.1 257944 118136 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1076  0.0  0.1 257940 118184 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1077  0.0  0.1 257948 118136 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1078  0.0  0.1 257944 118192 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1079  0.0  0.1 257952 118140 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname translate@%h --app weblate.utils --loglevel info -
weblate    1080  0.0  0.1 257948 118200 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu
weblate    1081  0.0  0.1 257952 118200 ?       S    14:12   0:00 /usr/bin/python3 /usr/local/bin/celery worker --hostname main@%h --app weblate.utils --loglevel info --queu

[cameronbraid]

Ok, that's the issue !

I am running on a 48 core pc, though I set cpu limits to 1, but it appears that python still sees all the cpus and must create a lot more processes with the default concurrency

Now I set helm values with :

extraConfig: 
  CELERY_MAIN_OPTIONS: --concurrency=1
  CELERY_NOTIFY_OPTIONS: --concurrency=1
  CELERY_TRANSLATE_OPTIONS: --concurrency=1

And now its using 1.2GB ram

Much better :)

Hope this info helps someone else

[github-actions]

The issue you have reported seems to be resolved now.

• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.