Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot compile sundials with oneMKL support #173

Closed
ciropom opened this issue Jul 6, 2022 · 7 comments
Closed

Cannot compile sundials with oneMKL support #173

ciropom opened this issue Jul 6, 2022 · 7 comments
Labels
question

Comments

@ciropom
Copy link

ciropom commented Jul 6, 2022

Hello all,
On an ubuntu 18.04 I installed onemkl using the intel script ( l_onemkl_p_2022.0.2.136_offline.sh )
then I'm trying to compile sundials with onemkl support.

The config phase is ok, but the generation phase fails, see the full output below

$ cmake ..     -DCMAKE_INSTALL_PREFIX=/usr/local/sundials-6.2.0     -DENABLE_OPENMP=ON -DENABLE_PTHREAD=ON -DBUILD_STATIC_LIBS=ON     -DBUILD_SHARED_LIBS=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON     -DUSE_GENERIC_MATH=OFF -DCMAKE_C_FLAGS="-O2 -march=native -fopenmp"     -DSUNDIALS_PRECISION=double      -DEXAMPLES_ENABLE_C=OFF -DEXAMPLES_INSTALL=OFF     -DSUNDIALS_INDEX_SIZE=64 -DENABLE_LAPACK=ON  -DLAPACK_WORKS=FALSE -DENABLE_ONEMKL=ON -DONEMKL_DIR="/opt/intel/oneapi/mkl/2022.0.2/lib/cmake/mkl"
-- SUNDIALS_GIT_VERSION: 
-- Using int64_t for indices
-- C standard set to 99
-- C extensions set to ON
-- Looking for POSIX timers... found
-- Determining Fortran name-mangling scheme... OK
-- CXX standard set to 11
-- C++ extensions set to ON
-- Looking for LAPACK libraries... OK
-- LAPACK_LIBRARIES:  /usr/lib/x86_64-linux-gnu/libopenblas.so;/usr/lib/x86_64-linux-gnu/libopenblas.so
-- Checking if LAPACK works with SUNDIALS... OK
-- MKL_ARCH: intel64
-- MKL_ROOT /opt/intel/oneapi/mkl/2022.0.2
-- MKL_LINK: dynamic
-- MKL_INTERFACE_FULL: gf_ilp64
-- MKL_THREADING: intel_thread
-- MKL_MPI: intelmpi
-- Found MKL: intelmpi  
-- Found MKL: /opt/intel/oneapi/mkl/2022.0.2  
-- Found MKL: /opt/intel/oneapi/mkl/2022.0.2/lib/intel64/libmkl_gf_ilp64.so  
-- Found MKL: /opt/intel/oneapi/mkl/2022.0.2/lib/intel64/libmkl_core.so  
-- Found MKL: /opt/intel/oneapi/mkl/2022.0.2/lib/intel64/libmkl_intel_thread.so  
-- Found MKL: /opt/intel/oneapi/compiler/latest/linux/compiler/lib/intel64_lin/libiomp5.so  
-- Skipped oneMKL tests, assuming oneMKL works with SUNDIALS.
-- Checking if Pthreads is available -- OK
-- Added NVECTOR_SERIAL module
-- Added NVECTOR_MANYVECTOR module
-- Added NVECTOR_OPENMP module
-- Added NVECTOR_PTHREADS module
-- Added SUNMATRIX_BAND module
-- Added SUNMATRIX_DENSE module
-- Added SUNMATRIX_SPARSE module
-- Added SUNMATRIX_ONEMKLDENSE module
-- Added SUNLINSOL_BAND module
-- Added SUNLINSOL_DENSE module
-- Added SUNLINSOL_PCG module
-- Added SUNLINSOL_SPBCGS module
-- Added SUNLINSOL_SPFGMR module
-- Added SUNLINSOL_SPGMR module
-- Added SUNLINSOL_SPTFQMR module
-- Added SUNLINSOL_LAPACKBAND module
-- Added SUNLINSOL_LAPACKDENSE module
-- Added SUNLINSOL_ONEMKLDENSE module
-- Added SUNNONLINSOL_NEWTON module
-- Added SUNNONLINSOL_FIXEDPOINT module
-- Added CVODE module
-- Configuring done
CMake Error at cmake/macros/SundialsAddLibrary.cmake:240 (target_sources):
  Error evaluating generator expression:

    $<TARGET_OBJECTS:sundials_sunmemsycl_obj>

  Objects of target "sundials_sunmemsycl_obj" referenced but no such target
  exists.
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:240 (target_sources):
  Error evaluating generator expression:

    $<TARGET_OBJECTS:sundials_sunmemsycl_obj>

  Objects of target "sundials_sunmemsycl_obj" referenced but no such target
  exists.
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:240 (target_sources):
  Error evaluating generator expression:

    $<TARGET_OBJECTS:sundials_sunmemsycl_obj>

  Objects of target "sundials_sunmemsycl_obj" referenced but no such target
  exists.
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:240 (target_sources):
  Error evaluating generator expression:

    $<TARGET_OBJECTS:sundials_sunmemsycl_obj>

  Objects of target "sundials_sunmemsycl_obj" referenced but no such target
  exists.
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunmatrixonemkldense_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunmatrixonemkldense_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunlinsolonemkldense_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunlinsolonemkldense_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunlinsolonemkldense_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:227 (add_library):
  Target "sundials_sunlinsolonemkldense_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at examples/cvode/CXX_onemkl/CMakeLists.txt:36 (add_executable):
  Target "cvRoberts_blockdiag_onemkl" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/cvode/CXX_onemkl/CMakeLists.txt:36 (add_executable):
  Target "cvRoberts_blockdiag_onemkl" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/cvode/CXX_onemkl/CMakeLists.txt:36 (add_executable):
  Target "cvRoberts_blockdiag_onemkl" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/sunmatrix/onemkldense/CMakeLists.txt:45 (add_executable):
  Target "test_sunmatrix_onemkldense" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/sunmatrix/onemkldense/CMakeLists.txt:45 (add_executable):
  Target "test_sunmatrix_onemkldense" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/sunlinsol/onemkldense/CMakeLists.txt:43 (add_executable):
  Target "test_sunlinsol_onemkldense" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/sunlinsol/onemkldense/CMakeLists.txt:43 (add_executable):
  Target "test_sunlinsol_onemkldense" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at examples/sunlinsol/onemkldense/CMakeLists.txt:43 (add_executable):
  Target "test_sunlinsol_onemkldense" links to target "MKL::MKL_DPCPP" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunmatrixonemkldense_obj_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunmatrixonemkldense_obj_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunmatrix/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunlinsolonemkldense_obj_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunlinsolonemkldense_obj_static" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunlinsolonemkldense_obj_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


CMake Error at cmake/macros/SundialsAddLibrary.cmake:153 (add_library):
  Target "sundials_sunlinsolonemkldense_obj_shared" links to target
  "MKL::MKL_DPCPP" but the target was not found.  Perhaps a find_package()
  call is missing for an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  src/sunlinsol/onemkldense/CMakeLists.txt:18 (sundials_add_library)


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

What's wrong here? Can you assist me in the installation procedure?
Thank you
Danilo
@gardner48
Copy link
Member

The oneMKL interface is intended for use with the SYCL NVector and offloading computations to Intel GPUs. From your configuration options and the CMake output, it looks like you want to use oneMKL to provide CPU LAPACK functions.

In this case, you'll need to set MKL as the LAPACK provider. Currently CMake is automatically finding and selecting OpenBLAS rather than MKL. Try removing the ENABLE_ONEMKL option and following the steps here to have CMake find MKL as the LAPACK library.

If the above does not work, you can try setting LAPACK_LIBRARIES to specify the BLAS/LAPACK libraries (some additional compiler flags may be necessary as well and this page can help in determining the libraries and compiler flags necessary).

@ciropom
Copy link
Author

ciropom commented Jul 6, 2022

Hello and thank you for your useful suggestions.
How can I enable my ODE solver to offload the computation to Intel GPUs? That was the main thing I was trying to accomplish.
I installed the standalone version of oneMKL, shoud I install something else/change my cmake configuration parameters?

thank you.
Danilo

@balos1 balos1 added the question label Jul 6, 2022
@gardner48
Copy link
Member

It looks like the Data Parallel C++ (DPC++) oneMKL interfaces are not installed as CMake cannot find the MKL::MKL_DPCPP target. Did you install the oneAPI tookit or a standalone oneMKL version?

The toolkit includes the DPC++/C++ compiler which you'll need for the DPC++ MKL interfaces and to enable SYCL support in SUNDIALS (-DENABLE_SYCL=ON) for offloading to Intel GPUs. You'll also need to set the C++ compiler to the DPC++ compiler e.g., -DCMAKE_CXX_COMPILER=dpcpp.

Additionally, any user supplied functions (e.g., the ODE right-hand side function) will need to use SYCL for offloading computations to the GPU. See cvAdvDiff_kry_sycl.cpp for an example using SYCL and cvRoberts_blockdiag_onemkl.cpp for an example using SYCL and oneMKL.

@ciropom
Copy link
Author

ciropom commented Jul 7, 2022

Thank you, I got it now.

@ciropom ciropom closed this as completed Jul 7, 2022
@ciropom
Copy link
Author

ciropom commented Jul 20, 2022

I'm sorry, just another question, is it possible to use SYCL and onemkl with a C interface? Because I'm using suncontext as

	   SUNContext_Create(NULL, &s->sunctx);
           s->cvode_mem = CVodeCreate(CV_BDF, s->sunctx);

while in your example you use c++ syntax

  sundials::Context sunctx;

@ciropom
Copy link
Author

ciropom commented Jul 20, 2022

I guess the answer is no

https://sundials.readthedocs.io/en/latest/Install_link.html?highlight=SYCL#cmakeoption-ENABLE_SYCL

@gardner48
Copy link
Member

@ciropom using the C++ convenience class is not required and you can use the C interface to create the SUNContext object like in the C examples.

@stuart-nolan stuart-nolan mentioned this issue Oct 6, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ciropom @balos1 @gardner48