VecISA.__bool__ is very expensive (nearly a second) on startup

[ezyang]

🐛 Describe the bug

VecISA currently requires compiling a cpp program to test for capabilities, similar to ./configure. We do this compilation on every new process invocation, dissimilar to ./configure. After performing the capability query, we should cache it and reuse the cache entry. This will save us about 1s cold start time on CPU inductor.

Versions

master

cc @msaroufim @bdhirsh @anijain2305 @zou3519 @chauhang @wconstab @soumith @ngimel

[Valentine233]

The PR is drafted #101679. However, there still exists an open issue in CI inductor / cuda11.8-py3.10-gcc7-sm86 / test (inductor, 1, 1, linux.g5.4xlarge.nvidia.gpu) (push). When dry-compiling for checking valid VecISA, we find that the same machine would give different results, compilation success or failure, resulting the issue when doing cache. The cache folder is set under /tmp/torchinductor_jenkins/. The next step is to download the .so file with compilation failure and run it locally to find out the failure reason.

[xuhancn]

I will optimze it after CppBuilder merged.

[tugsbayasgalan]

@xuhancn is this resolved?

[ezyang]

No

    @functools.lru_cache(None)
    def __bool__(self) -> bool:
        if config.cpp.vec_isa_ok is not None:
            return config.cpp.vec_isa_ok
        
        if config.is_fbcode():
            return True
        
        key, input_path = write(VecISA._avx_code, "cpp")
        from filelock import FileLock
        
        lock_dir = get_lock_dir()
        lock = FileLock(os.path.join(lock_dir, key + ".lock"), timeout=LOCK_TIMEOUT)
        with lock:
            output_path = input_path[:-3] + "so"
            build_cmd = shlex.split(
                cpp_compile_command(
                    input_path, output_path, warning_all=False, vec_isa=self
                )
            )
            try:
                # Check build result
                compile_file(input_path, output_path, build_cmd)
                subprocess.check_call(
                    [   
                        sys.executable,
                        "-c",
                        VecISA._avx_py_load.replace("__lib_path__", output_path),
                    ],
                    stderr=subprocess.DEVNULL,
                    env={**os.environ, "PYTHONPATH": ":".join(sys.path)},
                )
            except Exception as e:
                return False
            
            return True

in 0a1b3be

[xuhancn]

Possable help on this issue. #124366

[xuhancn]

The time is cost on ISA checker always need run the jit cpp build to compile check module.
I will optimize it later.
Wait on #124045 land.

[xuhancn]

@xuhancn is this resolved?

My pr will solve it: #124602