Build libnd4jcuda from source on windows reports Access denied

ND4J
1

Issue Description

Please describe our issue, along with:

  • Sucessfully build libnd4jcuda.dll
  • Followed instrcutions step by step but reported some naive error

Version Information

Please indicate relevant versions, including, if relevant:

  • Deeplearning4j version - current newest version
  • Platform information (OS, etc) - Windows 10
  • CUDA version, - CUDA Driver Version / Runtime Version - 11.1 / 10.2

Additional Information

Have followed the instructions step by step in https://github.com/eclipse/deeplearning4j/blob/master/libnd4j/windows.md, but till run the following command:

./buildnativeoperations.sh -c cuda clean true

Error output:

eval cmake
Running windows
Could not find OpenBLAS, please make sure to run the build with Maven or set the OPENBLAS_PATH variable
!!!
!! !!
!! !!
!! !!
!! !!
!! WARNING! !!
!! No helper packages configured! !!
!! You can specify helper by using -h key. I.e. <-h mkldnn> !!
!! !!
!! !!
!! !!
!! !!
!!!
PACKAGING = none
BUILD = release
CHIP = cuda
ARCH = x86-64
CHIP_EXTENSION =
CHIP_VERSION =
GPU_COMPUTE_CAPABILITY = all
EXPERIMENTAL = no
LIBRARY TYPE = dynamic
OPERATIONS = -DSD_ALL_OPS=true
MINIFIER = -DSD_BUILD_MINIFIER=false
TESTS = -DSD_BUILD_TESTS=OFF
NAME = -DSD_LIBRARY_NAME=nd4jcuda
OPENBLAS_PATH =
CHECK_VECTORIZATION = OFF
HELPERS =
Removing blasbuild
/f/deeplearning4j/libnd4j/blasbuild/cuda
– The C compiler identification is MSVC 19.28.29335.0
– The CXX compiler identification is MSVC 19.28.29335.0
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: D:/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
– Detecting C compile features
– Detecting C compile features - done
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: D:/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– The CUDA compiler identification is NVIDIA 10.2.89
– Detecting CUDA compiler ABI info
– Detecting CUDA compiler ABI info - failed
– Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/bin/nvcc.exe
– Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/bin/nvcc.exe - works
– Detecting CUDA compile features
– Detecting CUDA compile features - done
– Configuring done
– Generating done
– Build files have been written to: F:/deeplearning4j/libnd4j/blasbuild/cuda/flatbuffers-download
[1/9] Creating directories for ‘flatbuffers’
[2/9] Performing download step (git clone) for ‘flatbuffers’
Cloning into ‘flatbuffers-src’…
Note: switching to ‘v1.10.0’.

You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c0698cc3 Bumped version to 1.10.1 for all languages.
/mingw64/libexec/git-core/git-sh-setup: line 333: /usr/bin/uname: Permission denied
[3/9] Performing update step for ‘flatbuffers’
[4/9] No patch step for ‘flatbuffers’
[5/9] No configure step for ‘flatbuffers’
[6/9] No build step for ‘flatbuffers’
[7/9] No install step for ‘flatbuffers’
[8/9] No test step for ‘flatbuffers’
[9/9] Completed ‘flatbuffers’
CMake Warning at blasbuild/cuda/flatbuffers-src/CMakeLists.txt:21 (message):
Cannot build tests without building the compiler. Tests will be disabled.

– dir=‘F:/deeplearning4j/libnd4j/blasbuild/cuda/flatbuffers-src/include’
– dir=‘F:/deeplearning4j/libnd4j/blasbuild/cuda/include’
– dir=‘F:/deeplearning4j/libnd4j/include’
Building x86_64 binary…
Build cublas
– Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2 (found version “10.2”)
CUDA include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/include
CUDA found!
MSVC runtime for library: MultiThreadedDLL
CUDA on Windows: enabling /EHsc
– Configuring done
CMake Warning (dev) in blas/CMakeLists.txt:
Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
empty CUDA_ARCHITECTURES not allowed. Run “cmake --help-policy CMP0104”
for policy details. Use the cmake_policy command to set the policy and
suppress this warning.

CUDA_ARCHITECTURES is empty for target “samediff_obj”.
This warning is for project developers. Use -Wno-dev to suppress it.

– Generating done
CMake Warning:
Manually-specified variables were not used by the project:

BLAS
CMAKE_NEED_RESPONSE
DEV
MKL_MULTI_THREADED
PACKAGING
SD_BUILD_MINIFIER

– Build files have been written to: F:/deeplearning4j/libnd4j/blasbuild/cuda
[1/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\convertHalfs.cu.obj
convertHalfs.cu
[2/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\accumulateKernel.cu.obj
accumulateKernel.cu
[3/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\averagingKernel.cu.obj
averagingKernel.cu
[4/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\scalar.cu.obj
scalar.cu
[5/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj
FAILED: blas/CMakeFiles/samediff_obj.dir/__/include/loops/cuda/specials/bitonicSortStep.cu.obj
C:\PROGRA~1\NVIDIA~2\CUDA\v10.2\bin\nvcc.exe -forward-unknown-to-host-compiler --options-file blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj.rsp -MD -MT blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj -MF blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj.d -x cu -c …..\include\loops\cuda\specials\bitonicSortStep.cu -o blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj -Xcompiler=-Fdblas\CMakeFiles\samediff_obj.dir,-FS
Access is denied.
bitonicSortStep.cu
[6/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\scalar_int.cu.obj
scalar_int.cu
[7/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicArbitraryStep.cu.obj
bitonicArbitraryStep.cu
[8/816] Building CUDA object blas\CMakeFiles\samediff_obj.dir_\include\loops\cuda\scalar_bool.cu.obj
scalar_bool.cu
ninja: build stopped: subcommand failed.

2

This looks to just be a permissions issue during the build. On windows that can happen because a process is already using the folder among other things. Could you just manually delete the associated folders?

3

@agibsonccc Could you please tell me by which fold do you mean? Do you mean this folder: F:/deeplearning4j/libnd4j/blasbuild/cuda ? It should have been deleted and recreated every time I try to build, because the command I use is:
./buildnativeoperations.sh -c cuda clean true
with the clean attribute set.

4

@TempKonduitUser1 it’s in the error message righ there:

las\CMakeFiles\samediff_obj.dir_\include\loops\cuda\specials\bitonicSortStep.cu.obj -Xcompiler=-Fdblas\CMakeFiles\samediff_obj.dir,-FS
Access is denied.
bitonicSortStep.cu
[6/8

Also for the future, why not just use your github user name? This is the proper place to go for these kinds of questions anyways. Bugs are the only things that should go on the issue tracker.

5

@agibsonccc Thank you so much! It seems that folder is locked by another process. You are absolutely right! You solved my problem again! You are my life saver!