Documentation: Running Luigi on Windows

[tfiers]

Description

• Added section "Running on Windows" to doc/running_luigi.rst
• Made "Running Luigi" a proper item in the documentation side-nav, with sub-items.
  (Before, all subsections of the doc/running_luigi.rst file appeared separately as top-level items in the side-nav. This became too long).

Motivation and Context

I test my Luigi-based software locally on Windows (for later deployment on Linux). That's how I discovered these Windows issues.
I have spent a little time figuring out these issues.
This will spare other Luigi users the same puzzlement.

A simple test to show the multiprocessing issue on Windows:

import luigi

class Burrito(luigi.WrapperTask):
    def requires(self):
        return (Madamme(prot=put) for put in range(30))


class Madamme(luigi.Task):
    prot = luigi.IntParameter()

    def output(self):
        return luigi.LocalTarget(f"{self.prot}.txt")

    def run(self):
        with self.output().open('w') as f:
            f.write('prot ' * self.prot)

luigi.build([Burrito()], local_scheduler=True, workers=2)

Works on Unix, but on Windows only when workers=1

Have you tested this? If so, how?

Tested locally with tox -e docs

[tfiers]

@dlstadther, this doc change look good?

[dlstadther]

This is well written. But as i do not use luigi on windows, i can't speak for its accuracy.

Given no one else has reviewed, i'll give my thumbs up and merge.

Appreciate the documentation improvement!

[mostaphaRoudsari]

I believe you can fix this issue by adding freeze_support in Windows. This code should work on Windows:

import luigi
from multiprocessing import freeze_support

class Burrito(luigi.WrapperTask):
    def requires(self):
        return (Madamme(prot=put) for put in range(30))


class Madamme(luigi.Task):
    prot = luigi.IntParameter()

    def output(self):
        return luigi.LocalTarget(f"{self.prot}.txt")

    def run(self):
        with self.output().open('w') as f:
            f.write('prot ' * self.prot)


if __name__ == '__main__':
    freeze_support()
    luigi.build([Burrito()], local_scheduler=True, workers=2)