Today we are going to make a Real-time/ live Sketch-making script using OpenCV in Python. OpenCV makes it very easy for us to work with images and videos on the computer. We will also make use of Numpy and Matplotlib to make this live sketch app.
Live Sketch Algorithm OpenCV
- Capturing Real-time Video from the source example – computer’s camera
- Reading each frame of the video, so that we can make manipulations on the frame.
- Converting each frame from colored to grayscale, using OpenCV functions.
- Making Use of Blurring on the grayscale image to remove all the noises on the picture/image. ( Gaussian Blur )
- Detecting Edges of the blurred image, for making it like a sketch of the person or object. (Canny Edge)
- Thresholding the Edge Detected image.
Learn Image Manipulations if you have any problem with any of the steps.
Source Code at Divyanshu Shekhar Github.
Installing Packages for Live Sketch OpenCV
Install OpenCV, as its the most important package for computer vision in Python.
pip install opencv-python
If you have installed OpenCV, No need to install NumPy as it already comes pre-packed with OpenCV. Otherwise,
pip install numpy
Install Matplotlib for Visualization of data. (Optional)
pip install matplotlib
Importing Packages for Live Sketch OpenCV
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
OpenCV Python Video Capture
Here we are making a livesketch() function in python for recording / Capturing the video from the front camera of the computer/Laptop.
cap = cv.VideoCapture(0)
VideoCapture() Function is used to capture video either from the camera or already recorded video. cap variable returns a boolean value (True if able to retrieve/capture video successfully, False if not able to successfully capture the video). It takes one parameter:
- 0 – Front Camera
- 1 – Rear Camera
- Source of the video – Example (Absolute path / Relative Path) – “D:/Video/test.mp4”
If the Cap returns True, then read() function is applied to it and it returns two things:
- Boolean Value (Was it successfully able to read the frame, If yes)
- Returns the frame of the video.
Each Frame is sent to a sketch() function that takes the frame as an input parameter and manipulates it to return a sketch of the frame.
Don’t forget to release the captured video at the end of the while loop. Otherwise, it will consume all your machine’s memory.
def liveSketch(): cap = cv.VideoCapture(0) while True: ret, frame = cap.read() cv.imshow("Live Sketch", sketch(frame)) if cv.waitKey(1) == 27: break cap.release() cv.destroyAllWindows()
OpenCV Sketch Function
The First Step is to convert the received frame from RGB / Colored to a grayscale image, using the OpenCV cvtColor() function. The function takes two parameters:
- Source – The image to be converted
- Type – Type of conversion. (In this case BGR2GRAY)
The Image is then blurred using Gaussian Blur. The Kernel size used in this case is 5×5. To know about kernels. Read the Image manipulations operations.
The Grayscaled image is blurred in order to remove any kind of noise in the image so that it is easy for edge detection.
The Blurred image is then used for edge detection using the canny edge detection function in OpenCV. You Can learn more about the Canny Edge Detection method in Image Manipulation OpenCV Tutorial. Link Above.
The Edge Detected image then is thresholded and the method used is THRESH_BINARY_INV to get the inverse of the binary threshold. In Simple words, the sketch will be of black color on a white background.
After all these manipulations the thresholded mask is returned to the livesketch() function and the image is shown on the window, till the user manually quits the window by pressing ESC.
def sketch(image): # Convert image to gray scale img_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY) # Clean up image using Gaussian Blur img_gray_blur = cv.GaussianBlur(img_gray, (5, 5), 0) # Extract Edges canny_edges = cv.Canny(img_gray_blur, 30, 70) # Do an invert binarize the image ret, mask = cv.threshold(canny_edges, 120, 255, cv.THRESH_BINARY_INV) return mask