Site build is slooow

[tshepang]

It takes about 36s. Here is the top results of my profile run (which are not much different from Python 2.7):

$ time python -m cProfile -s tottime ~/.venv/33/bin/pelican --settings settings.py                                                                                 
Done: Processed 279 articles and 1 pages in 52.99 seconds.                                                                                                         
         55497600 function calls (53139173 primitive calls) in 53.438 seconds                                                                                      

   Ordered by: internal time                                                                                                                                       

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)                                                                                            
  1402659    4.137    0.000   11.843    0.000 {method 'sub' of '_sre.SRE_Pattern' objects}                                                                         
4468181/4468160    3.194    0.000    3.196    0.000 {built-in method hasattr}                                                                                      
   345637    3.180    0.000   28.886    0.000 utils.py:230(slugify)                                                                                                
   644624    3.070    0.000    4.801    0.000 posixpath.py:340(normpath)                                                                                           
   345637    3.039    0.000    4.145    0.000 __init__.py:21(unidecode)                                                                                            
   695778    2.062    0.000    5.159    0.000 __init__.py:169(func)                                                                                                
  2255268    1.693    0.000    5.179    0.000 __init__.py:68(__new__)                                                                                              
  2287743    1.465    0.000    1.792    0.000 {built-in method __new__ of type object at 0x819ee0}                                                                 
3465705/3465702    1.399    0.000    1.450    0.000 {built-in method isinstance}                                                                                   
   160629    1.296    0.000   40.585    0.000 contents.py:146(url_format)                                                                                          
   347889    1.267    0.000    2.536    0.000 __init__.py:107(join)                                                                                                
   347889    1.219    0.000    2.138    0.000 __init__.py:111(split)                                                                                               
   347889    1.199    0.000    2.209    0.000 __init__.py:123(unescape)                                                                                            
   347889    1.132    0.000   15.511    0.000 __init__.py:145(striptags)                                                                                           
  8962220    1.053    0.000    1.053    0.000 {method 'append' of 'list' objects}                                                                                  
   160630    1.025    0.000    1.025    0.000 {built-in method now}                                                                                                
   161223    1.013    0.000    5.883    0.000 posixpath.py:452(relpath)                                                                                            
  2293519    0.797    0.000    0.797    0.000 {method 'startswith' of 'str' objects}                                                                               
   695778    0.711    0.000    1.696    0.000 __init__.py:203(_escape_argspec)                                                                                     
  1017327    0.696    0.000    0.696    0.000 {method 'split' of 'str' objects}                                                                                    
   870753    0.667    0.000    0.934    0.000 re.py:268(_compile)                                                                                                  
1439219/124977    0.613    0.000   44.555    0.000 {method 'join' of 'str' objects}                                                                                
   160629    0.594    0.000   41.398    0.000 contents.py:169(_expand_settings)                                                                                    
694659/694435    0.591    0.000    1.231    0.000 <frozen importlib._bootstrap>:1594(_handle_fromlist)                                                             
   698740    0.577    0.000    3.062    0.000 re.py:163(sub)                                                                                                       
1291402/496767    0.547    0.000   41.783    0.000 {built-in method getattr}                                                                                       
   644621    0.547    0.000    6.539    0.000 posixpath.py:377(abspath)                                                                                            
   160921    0.534    0.000   10.359    0.000 contents.py:283(get_relative_source_path)                                                                            
   327701    0.530    0.000    1.125    0.000 posixpath.py:71(join)                                                                                                
    31548    0.524    0.000    0.524    0.000 {method 'search' of '_sre.SRE_Pattern' objects}                                                                      
   976213    0.462    0.000    0.867    0.000 posixpath.py:38(_get_sep)                                                                                            
  4621432    0.423    0.000    0.423    0.000 {built-in method ord}                                                                                                
   161223    0.422    0.000    0.609    0.000 genericpath.py:68(commonprefix)                                                                                      
   644934    0.402    0.000    1.193    0.000 posixpath.py:61(isabs)                                                                                               
    33726    0.320    0.000    0.961    0.000 statemachine.py:729(make_transitions)                                                                                
   347889    0.285    0.000    0.343    0.000 {built-in method escape}                                                                                             
   160629    0.273    0.000   41.900    0.000 contents.py:173(get_url_setting)                                                                                     
   346500    0.273    0.000    0.273    0.000 {built-in method normalize}                                                                                          
     8520    0.269    0.000   43.504    0.005 article.html:29(block_content)
...

[talha131]

What's your result with default theme?

[tshepang]

A bit shocking actually:

real     0m8.533s
user     0m7.704s
sys      0m0.172s

Elegant gives:

real     0m39.823s
user     0m38.732s
sys      0m0.452s

[talha131]

Remove iterations for next and previous article instead use plugin

[talha131]

My test before any changes

time pelican --settings settings.py
Done: Processed 280 articles and 1 pages in 40.54 seconds.

real    0m40.985s
user    0m40.571s
sys     0m0.377s

[talha131]

After the fix

time pelican --settings settings.py
Done: Processed 280 articles and 1 pages in 9.82 seconds.

real    0m10.239s
user    0m9.579s
sys     0m0.358s

[talha131]

@tshepang thanks a lot for reporting this issue. It didn't come across my mind to compare Elegant's performance against other themes. I do not have a large blog yet that is why I didn't face these issues.

I ran tests several times and it seems it takes on 9.18 seconds on average to process 280 articles and 1 pages. It is almost same as default theme. I deduce we have removed the major bottleneck in the performance.

Unfortunately, it means for the users to use neighbors plugin.

Kindly run profiler again and let me know if you find any other bottleneck or performance issue.

[tshepang]

Done: Processed 280 articles and 1 pages in 8.35 seconds.

real    0m8.847s
user    0m8.700s
sys     0m0.092s

\0/ \0/ \0/

[tshepang]

Using the plugin gives about the same, fast performance. How come? I expected it to be super-slow.

[talha131]

Plugin calculates the next and previous article during generation phase.

Elegant used to iterated through all the articles using loops to find next and previous article. This step was repeated for each and every article. That's why it took so long.

[silverhook]

Thanks a ton! 😸

I'm running my webserver on a small ARM device, so with me it's not about 8 vs. 40 seconds, but 350 vs. 1300. The improvement of this patch for me is about a quarter of an hour! 👍

e.g.

Done: Processed 260 articles and 2 pages in 382.31 seconds.

[talha131]

Gosh! That's a huge number. My own blog is very small for now, that's why I never noticed it myself.

The lesson I learned is to never put any sort of calculation in template 😄