Setup OpenCV-DNN module with CUDA backend support (For Linux)

In this tutorial, we will be building OpenCV from source with CUDA backend support (OpenCV-DNN-CUDA module).

IMPORTANT: The OpenCV-DNN module only supports inference so although you will get much faster inference out of it, the training however will be the same as was for the OpenCV we set up without CUDA backend support.

STEPS

1. Install CUDA & cuDNN.

2. Install the necessary Ubuntu packages if not already installed.

3. Download and unzip opencv & opencv_contrib

4. Setup python environment

5. Configure OpenCV libraries with CMake command

6. Build OpenCV from source for your NVidia GPU.

My YouTube Video on this!

LET’S BEGIN!

STEP 1) Install CUDA & cuDNN

Download and install CUDA and its corresponding cuDNN archive for your system. Go to https://developer.nvidia.com/cuda-downloads to download the latest CUDA Toolkit. You can also download previous versions from Archive of Previous CUDA Releases OR under the Resources section at the cuda-downloads link given above. Next, download and copy the contents of the cuDNN folder to where we installed CUDA.

You can download the latest version of cuDNN from https://developer.nvidia.com/cudnn . You can also download previous versions from the cudnn archive at https://developer.nvidia.com/rdp/cudnn-archive .

Follow this blog to learn how to install and set up CUDA and CUDNN on Linux.

Next, you need to add the paths to these in your bashrc script as shown in the blog link above.

After that, reboot your system and verify your CUDA installation in a terminal by typing the following 2 commands:

nvidia-smi
nvcc -V

STEP 2) Install the necessary Ubuntu packages if not already installed.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install libv4l-dev libxvidcore-dev libx264-dev
sudo apt-get install libgtk-3-dev
sudo apt-get install libblas-dev liblapack-dev gfortran
sudo apt-get install python3-dev

STEP 3) Download and unzip opencv & opencv_contrib

Navigate to the home folder, download OpenCV archive, unzip it, and rename the folder to opencv.

cd ~
wget -O opencv-4.5.5.zip https://github.com/opencv/opencv/archive/4.5.5.zip
unzip -q opencv-4.5.5.zip
mv opencv-4.5.5 opencv
rm -f opencv-4.5.5.zip

Next, download the opencv_contrib archive, unzip it, and rename the folder to opencv_contrib.

Note: Make sure to download the exact same version of the opencv_contrib as the opencv archive you downloaded before that.

wget -O opencv_contrib-4.5.5.zip https://github.com/opencv/opencv_contrib/archive/4.5.5.zip
unzip -q opencv_contrib-4.5.5.zip
mv opencv_contrib-4.5.5 opencv_contrib
rm -f opencv_contrib-4.5.5.zip

STEP 4) Setup the Python environment

Install virtualenv and virtualenvwrapper if you don’t have them on your system already.

sudo pip install virtualenv virtualenvwrapper

Open the bashrc script file using nano.

nano ~/.bashrc

Enter the following export paths to set the path for the virtualenvwrapper at the bottom and save the file. (Press ctrl+x , y and Enter to save the file)

export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh

Run source to assert the changes in the system for the bashrc script.

source ~/.bashrc

Create a virtual environment in a terminal. We will be installing the OpenCV-DNN-CUDA module in a virtual environment and not our base environment.

mkvirtualenv opencv_dnn_cuda -p python3

Install NumPy package.

pip install numpy

Activate the virtual environment by entering the following command:

workon opencv_dnn_cuda

STEP 5) Configure OpenCV using the cmake command to install some libraries

Navigate to the opencv folder we created in step 3. Create another folder inside it named build and navigate inside it.

cd ~/opencv
mkdir build
cd build

Run the CMake command to configure settings for the OpenCV you want to build from source with your custom libraries and functions selected.

Change the CUDA_ARCH_BIN parameter in the command below to your GPU’s compute capability. Find your GPU’s cc here. Also, change the virtual environment name in the PYTHON_EXECUTABLE parameter if you are using a different name else leave it as it is.

cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D CMAKE_INSTALL_PREFIX=/usr/local \
	-D INSTALL_PYTHON_EXAMPLES=ON \
	-D INSTALL_C_EXAMPLES=OFF \
	-D OPENCV_ENABLE_NONFREE=ON \
	-D WITH_CUDA=ON \
	-D WITH_CUDNN=ON \
	-D OPENCV_DNN_CUDA=ON \
	-D ENABLE_FAST_MATH=1 \
	-D CUDA_FAST_MATH=1 \
	-D CUDA_ARCH_BIN=6.1 \
	-D WITH_CUBLAS=1 \
	-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
	-D HAVE_opencv_python3=ON \
	-D PYTHON_EXECUTABLE=~/.virtualenvs/opencv_dnn_cuda/bin/python \
	-D BUILD_EXAMPLES=ON ..

In the above CMake command, we are compiling OpenCV with both CUDA and cuDNN support enabled (WITH_CUDA and WITH_CUDNN, respectively). We are also enabling the OPENCV_DNN_CUDA parameter to build the DNN module with CUDA backend support. We are also setting ENABLE_FAST_MATH, CUDA_FAST_MATH, and WITH_CUBLAS for optimization purposes.

STEP 6) Lastly, build the OpenCV-DNN module from source with CUDA backend support for your specific NVidia GPU

Run the make command to build the OpenCV with the above-configured settings. (Run nproc in terminal to print your no. of processors. Replace that with 8 below in j8.)

make -j8

This will build the OpenCV-DNN module from source with the CUDA backend. This process can take up to an hour or so.

Next, install the OpenCV on your system once it has finished building the OpenCV in the make command above. Run the following commands:

sudo make install
sudo ldconfig

Next, the final step is to create a symbolic link between the newly installed OpenCV library and our python virtual environment. For that, you need the path where the OpenCV bindings were installed. You can determine that path via the install path configuration in the CMake command step output.

In my case, the install path was lib/python3.6/site-packages/cv2/python-3.6 which means my OpenCV bindings are in /usr/local/lib/python3.6/site-packages/cv2/python-3.6. Check where your “.so” file is.

You can confirm the above path by listing its contents. Run the following ls command:

ls -l /usr/local/lib/python3.6/site-packages/cv2/python-3.6
total 10104
-rw-r--r-- 1 root staff 10345488 Mar 23 07:06 cv2.cpython-36m-x86_64-linux-gnu.so

You will see an output like the above confirming the cv2 binding is here. I have Python 3.6 and a 64-bit architecture which explains the file name. Yours might vary according to your Python and system architecture.

Next, navigate to the virtual environment “site-packages” folder, and finally create a link for the cv2 bindings we found above using the ln -s command. This creates a cv2.so sym-link inside the current working directory which is the site-packages folder in our virtual environment.

cd ~/.virtualenvs/opencv_dnn_cuda/lib/python3.6/site-packages/
ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-x86_64-linux-gnu.so cv2.so

NOTE: Make sure to double-check everything as this command silently fails even if the path is incorrect meaning you will get no warning or error message and the output will not work. So be mindful of this step.

That’s it! We have successfully built the OpenCV-DNN module with CUDA backend support.

Check if cv2 is installed correctly.

workon opencv_dnn_cuda
python
Python 3.6.9 (default, July 02 2019, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.5.5'
>>>

Check if the OpenCV is installed correctly with CUDA backend support using the test_DNN_CV.py script below. Download it below.

import numpy as np
import cv2 as cv
import time
npTmp = np.random.random((1024, 1024)).astype(np.float32)
npMat1 = np.stack([npTmp,npTmp],axis=2)
npMat2 = npMat1
cuMat1 = cv.cuda_GpuMat()
cuMat2 = cv.cuda_GpuMat()
cuMat1.upload(npMat1)
cuMat2.upload(npMat2)
start_time = time.time()
cv.cuda.gemm(cuMat1, cuMat2,1,None,0,None,1)
print("CUDA using GPU --- %s seconds ---" % (time.time() - start_time))
start_time = time.time()
cv.gemm(npMat1,npMat2,1,None,0,None,1)
print("CPU --- %s seconds ---" % (time.time() - start_time))

Download the script to test OpenCV-DNN below.

Download test_DNN_CV.py

Next Tutorial:

Run YOLOv4 inference using OpenCV-DNN Module on Linux

YOLOv4 inference using OpenCV-DNN-CUDA on Linux

Don’t forget to leave a 👏

Have a great day !!! ✌

♕ TECHZIZOU ♕