Error when run install_requirements.bat --pybind xnnpack

[amaruki]

🐛 Describe the bug

when I run install_requrements.bat an error appears like this. I tried to remove the pip-out and rebuild, but the same error persists.

(Edited)
I tried to install buck2, but there is still an error when doing the installation

Building wheels for collected packages: executorch
  Running command Building wheel for executorch (pyproject.toml)
  running bdist_wheel
  running build
  command options for 'CustomBuild':
    build_base = pip-out
    build_purelib = pip-out\lib
    build_platlib = pip-out\lib.win32-cpython-310
    build_lib = pip-out\lib.win32-cpython-310
    build_scripts = pip-out\scripts-3.10
    build_temp = pip-out\temp.win32-cpython-310
    plat_name = win32
    compiler = None
    parallel = 7
    debug = None
    force = None
    executable = C:\ProgramData\miniconda3\envs\executorch\python.exe
  creating C:\Users\Administrator\executorch\pip-out\temp.win32-cpython-310\cmake-out
  deleting C:\Users\Administrator\executorch\pip-out\temp.win32-cpython-310\cmake-out\CMakeCache.txt
  cmake -S C:\Users\Administrator\executorch -B C:\Users\Administrator\executorch\pip-out\temp.win32-cpython-310\cmake-out -DBUCK2= -DPYTHON_EXECUTABLE=C:\ProgramData\miniconda3\envs\executorch\python.exe -DCMAKE_PREFIX_PATH=C:\ProgramData\miniconda3\envs\executorch\Lib\site-packages -DCMAKE_BUILD_TYPE=Release -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_LOG_LEVEL=Info -DCMAKE_OSX_DEPLOYMENT_TARGET=10.15 -DEXECUTORCH_SEPARATE_FLATCC_HOST_PROJECT=OFF -DEXECUTORCH_BUILD_PYBIND=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED_AOT=ON -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON -DEXECUTORCH_BUILD_KERNELS_CUSTOM_AOT=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED_AOT=ON -DEXECUTORCH_BUILD_XNNPACK=ON -T ClangCL
  -- Building for: Visual Studio 17 2022
  -- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
  -- The C compiler identification is Clang 18.1.8 with MSVC-like command-line
  -- The CXX compiler identification is Clang 18.1.8 with MSVC-like command-line
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/Llvm/x64/bin/clang-cl.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/Llvm/x64/bin/clang-cl.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CMake Deprecation Warning at backends/xnnpack/third-party/cpuinfo/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
    Compatibility with CMake < 3.10 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
    to tell CMake that the project requires at least <min> but has been updated
    to work with policies introduced by <max> or earlier.


  CMake Deprecation Warning at backends/xnnpack/third-party/pthreadpool/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
    Compatibility with CMake < 3.10 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
    to tell CMake that the project requires at least <min> but has been updated
    to work with policies introduced by <max> or earlier.


  -- Downloading FXdiv to C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/FXdiv-source (define FXDIV_SOURCE_DIR to avoid it)
  CMake Deprecation Warning at CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
    Compatibility with CMake < 3.10 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
    to tell CMake that the project requires at least <min> but has been updated
    to work with policies introduced by <max> or earlier.


  -- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
  -- Configuring done (4.1s)
  -- Generating done (0.1s)
  -- Build files have been written to: C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/FXdiv-download
  MSBuild version 17.12.12+1cce77968 for .NET Framework

    1>Checking Build System
    Creating directories for 'fxdiv'
    Building Custom Rule C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/FXdiv-download/CMakeLists.txt
    Performing download step (git clone) for 'fxdiv'
    Cloning into 'FXdiv-source'...
    Your branch is up to date with 'origin/master'.
    Already on 'master'
    Performing update step for 'fxdiv'
    -- Fetching latest from the remote origin
    No patch step for 'fxdiv'
    No configure step for 'fxdiv'
    No build step for 'fxdiv'
    No install step for 'fxdiv'
    No test step for 'fxdiv'
    Completed 'fxdiv'
    Building Custom Rule C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/FXdiv-download/CMakeLists.txt
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - not found
  -- Found Threads: TRUE
  CMake Deprecation Warning at pip-out/temp.win32-cpython-310/cmake-out/FXdiv-source/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
    Compatibility with CMake < 3.10 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
    to tell CMake that the project requires at least <min> but has been updated
    to work with policies introduced by <max> or earlier.


  -- Using python executable 'C:\ProgramData\miniconda3\envs\executorch\python.exe'
  -- Resolved buck2 as buck2.
  -- Killing buck2 daemon
  'buck2 kill'
  -- executorch: Generating source lists
  -- executorch: Generating source file list C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/executorch_srcs.cmake
  Error while generating C:/Users/Administrator/executorch/pip-out/temp.win32-cpython-310/cmake-out/executorch_srcs.cmake. Exit code: 1
  Output:

  Error:
  Traceback (most recent call last):
    File "C:\Users\Administrator\executorch\build\buck_util.py", line 26, in run
      cp: subprocess.CompletedProcess = subprocess.run(
    File "C:\ProgramData\miniconda3\envs\executorch\lib\subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['buck2', 'cquery', "inputs(deps('//runtime/executor:program'))"]' returned non-zero exit status 3.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "C:\Users\Administrator\executorch\build\extract_sources.py", line 218, in <module>
      main()
    File "C:\Users\Administrator\executorch\build\extract_sources.py", line 203, in main
      target_to_srcs[name] = sorted(target.get_sources(graph, runner))
    File "C:\Users\Administrator\executorch\build\extract_sources.py", line 116, in get_sources
      sources: set[str] = set(runner.run(["cquery", query]))
    File "C:\Users\Administrator\executorch\build\buck_util.py", line 31, in run
      raise RuntimeError(ex.stderr.decode("utf-8")) from ex
  RuntimeError: 2025-01-06T22:59:04.945981Z  WARN buck2::check_user_allowed: You're running buck2 from an admin shell. Invocations from non-admin shells will likely fail going forward. To remediate, run `buck2 clean` in this admin shell, then switch to a non-admin shell.
  [2025-01-06T14:59:04.971-08:00] Build ID: 635c1fb0-826c-44c6-ab47-23e7cd8517bd
  [2025-01-06T14:59:04.972-08:00] File changed: prelude//cxx/dist_lto
  [2025-01-06T14:59:04.972-08:00] File changed: prelude//cxx/dist_lto/tools
  [2025-01-06T14:59:04.972-08:00] File changed: prelude//cxx/dist_lto/tools/tests
  [2025-01-06T14:59:04.972-08:00] 79 additional file change events
  Command failed:
  From load at implicit location

  Caused by:
      0: From load at third-party/prelude/prelude.bzl:8
      1: From load at third-party/prelude/native.bzl:17
      2: From load at third-party/prelude/cxx/cxx_toolchain.bzl:17
      3: From load at third-party/prelude/decls/cxx_rules.bzl:16
      4: Error evaluating module: `prelude//decls/common.bzl`
      5: error: String literals are not allowed in type expressions: `"PluginKind"`
           --> third-party/prelude/decls/common.bzl:23:27
            |
         23 |     uses_plugins = field([list["PluginKind"], None], None),
            |                           ^^^^^^^^^^^^^^^^^^
            |



  CMake Error at build/Utils.cmake:204 (message):
    executorch: source list generation failed
  Call Stack (most recent call first):
    CMakeLists.txt:338 (extract_sources)


  -- Configuring incomplete, errors occurred!
  error: command 'C:\\ProgramData\\miniconda3\\envs\\executorch\\Scripts\\cmake.EXE' failed with exit code 1
  error: subprocess-exited-with-error

  × Building wheel for executorch (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\ProgramData\miniconda3\envs\executorch\python.exe' 'C:\ProgramData\miniconda3\envs\executorch\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' build_wheel 'C:\Users\ADMINI~1\AppData\Local\Temp\tmpkrz98kv9'
  cwd: C:\Users\Administrator\executorch
  Building wheel for executorch (pyproject.toml) ... error
  ERROR: Failed building wheel for executorch
Failed to build executorch
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (executorch)
Traceback (most recent call last):
  File "c:\Users\Administrator\executorch\install_requirements.py", line 172, in <module>
    subprocess.run(
  File "C:\ProgramData\miniconda3\envs\executorch\lib\subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\ProgramData\\miniconda3\\envs\\executorch\\python.exe', '-m', 'pip', 'install', '.', '--no-build-isolation', '-v', '--extra-index-url', 'https://download.pytorch.org/whl/test/cpu']' returned non-zero exit status 1.

Versions

PyTorch version: 2.5.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro (10.0.19045 64-bit)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.31.2
Libc version: N/A

Python version: 3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Name: Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
Manufacturer: GenuineIntel
Family: 2
Architecture: 9
ProcessorType: 3
DeviceID: CPU0
CurrentClockSpeed: 2600
MaxClockSpeed: 2600
L2CacheSize: 256
L2CacheSpeed: None
Revision: 20225

[mergennachin]

@Ngapa

Can you try the following steps:

• Checkout the latest main branch
• ./install_requirement.bat --clean
• Remove buck2-bin folder (if it exists) in executorch root directory.
• ./install_requirement.bat

[mergennachin]

Can you also see which buck2 version it uses?

You shouldn't install buck2 manually. It should download the correct version automatically underneath the hood.

executorch/build/Utils.cmake

Lines 221 to 228 in a29dc49

	# Sets the value of the BUCK2 variable by searching for a buck2 binary with the
	# correct version.
	#
	# The resolve_buck.py script uses the following logic to find buck2: 1) If BUCK2
	# argument is set explicitly, use it. Warn if the version is incorrect. 2) Look
	# for a binary named buck2 on the system path. Take it if it is the correct
	# version. 3) Check for a previously downloaded buck2 binary (from step 4). 4)
	# Download and cache correct version of buck2.

[mergennachin]

Oh wait, for windows, we don't install the correct version automatically.

@python3kgae actually have a PR #4856 for windows support for resolving buck2 version.

@Ngapa - perhaps you can do patch @python3kgae's PR for the time being.

cc @tarun292, @GregoryComer

[mergennachin]

@Ngapa alternatively, supporting ExecuTorch natively on Windows is still work-in-progress (#4661)

I'd suggest perhaps using WSL

[mergennachin]

@Ngapa

Closing this issue in lieu of this issue #4661 and recently closed PR #4856