Image Blending OpenCV Python

What is Image Blending? Basically, It is Mixing up two images. In this blog we will learn how image Blending works in OpenCV Python.

Let’s learn Image Blending in OpenCV Python!

Before learning Image Blending we will have to learn some important terms that we need for Image Blending.

Pyramid in OpenCV

Pyramid, or Pyramid representation, is a type of multi-state signal representation in which a signal or an image is subject to repeated smoothing or sub-sampling.

• Lower resolution– lr
• Higher resolution– hr

lr = cv.pyrDown(image)

hr = cv.pyrUp(image)

How Laplacian Pyramid is formed in OpenCV

A level in Laplacian Pyramid is formed by the difference between that level in the Gaussian Pyramid and the expanded version of its upper level in the Gaussian Pyramid.

Prerequisite of Image Blending in OpenCV

• Both the Images should be of the Same size.

If the shape and size of both the images differ then it will not be a perfect blend.

Types of Image Blending in OpenCV

Image Blending using Stack

Images can be blended or added using a stack. There are mainly two types of stacking:

• Horizontal Stacking and,
• Vertical Stacking

Both these stacking techniques are done with the help of the NumPy package.

NumPy helps us to work with matrix and it also simplifies the work of slicing and adding the two matrices.

NumPy comes loaded with the OpenCV package. You don’t need to install NumPy separately.

These techniques do not give us a Smooth blend. It also does not soothe the human eye. We can easily differentiate between both the pictures.

Horizontal Stacking

n1_n2 = np.hstack((n1[:, :256], n2[:, 256:]))

Vertical Stacking

n1_n2 = np.vstack((n1[:256, :], n2[:256, :]))

Image Blending using Pyramid

Another way to blend two images in OpenCV is via the Pyramid technique. In the Pyramid method, an image is scaled up and down up to certain levels and then reconstructed, Hence a Smooth blend is found.

Steps for Image Blending using Pyramid:

1. Load two images that you want to blend.
2. Find Gaussian Pyramids for two images. Up to 6 levels.
3. From Gaussian Pyramids, find their Laplacian Pyramid.
4. Now Join the images in each level of the Laplacian Pyramid.
5. Finally from this joint image pyramids, reconstruct the original image.

If you have difficulty in any operation or Image manipulation techniques. Read the blogs related to that topic first as those functions will be widely used in this topic.

Image Blending Algorithm

Load Images

n1 = cv.imread(“./img/nature1.jpg”)
n2 = cv.imread(“./img/nature2.jpg”)

Generate Gaussian Pyramids

n1_copy = n1.copy()
gp_n1 = [n1_copy]

for i in range(6):
    n1_copy = cv.pyrDown(n1_copy)
    gp_n1.append(n1_copy)

Note: Copy of the Original image is used for generating Gaussian Pyramid of that image.

Do the same for the second image. Gaussian Pyramid is generated up to 6 levels.

Generate Laplacian Pyramids

n1_copy = gp_n1[5]lp_n1 = [n1_copy]

for i in range(5, 0, -1):
    gp_ex = cv.pyrUp(gp_n1[i])
    lap = cv.subtract(gp_n1[i-1], gp_ex)
    lp_n1.append(lap)

Laplacian Pyramid is generated from the Gaussian Pyramid image, so use a copy of the Gaussian pyramid image so that the original image is retained.

Join the Pyramid

Laplacian Pyramid images of the both the images are stacked horizontally. The stacked images are stored in a new list.

n1_n2_pyramid = []n = 0
for n1_lap, n2_lap in zip(lp_n1, lp_n2):
n += 1
cols, rows, ch = n1_lap.shape
laplacian = np.hstack(
(n1_lap[:, 0:int(cols/2)], n2_lap[:, int(cols/2):]))
n1_n2_pyramid.append(laplacian)

Reconstructing Image

reconstruct = n1_n2_pyramid[0]for i in range(1, 6):
reconstruct = cv.pyrUp(reconstruct)
reconstruct = cv.add(n1_n2_pyramid[i], reconstruct)

Display Blended Image

cv.imshow(“N1_N2”, reconstruct)
cv.waitKey()
cv.destroyAllWindows()

Image Blending OpenCV Source code

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt


def blend():
    '''
    Image Blending Using Pyramid
    1. Load two images
    2. Find Gaussian Pyramids for two images. Upto 6 level
    3. From Gaussian Pyramids,find their Laplacian Pyramids.
    4. Now Join the images in each levels of laplacian P.
    5. Finally from this joint image Pyramids, reconstruct the
        Original Image.
    '''
    n1 = cv.imread("./img/nature1.jpg")
    n2 = cv.imread("./img/nature2.jpg")

    print(n1.shape)
    print(n2.shape)

    # n1_n2 = np.hstack((n1[:, :256], n2[:, 256:]))

    '''
    Generate Gaussian Pyramid for Nature 1
    '''
    n1_copy = n1.copy()
    gp_n1 = [n1_copy]

    for i in range(6):
        n1_copy = cv.pyrDown(n1_copy)
        gp_n1.append(n1_copy)

    '''
    Generate Gaussian Pyramid for Nature 2
    '''
    n2_copy = n2.copy()
    gp_n2 = [n2_copy]

    for i in range(6):
        n2_copy = cv.pyrDown(n2_copy)
        gp_n2.append(n2_copy)

    '''
    Laplacian Pyramid for Nature 1
    '''
    n1_copy = gp_n1[5]
    lp_n1 = [n1_copy]

    for i in range(5, 0, -1):
        gp_ex = cv.pyrUp(gp_n1[i])
        lap = cv.subtract(gp_n1[i-1], gp_ex)
        lp_n1.append(lap)

    '''
    Laplacian Pyramid for Nature 2
    '''
    n2_copy = gp_n2[5]
    lp_n2 = [n2_copy]

    for i in range(5, 0, -1):
        gp_ex = cv.pyrUp(gp_n2[i])
        lap = cv.subtract(gp_n2[i-1], gp_ex)
        lp_n2.append(lap)

    '''
    Join Half the Pyramid
    '''
    n1_n2_pyramid = []
    n = 0
    for n1_lap, n2_lap in zip(lp_n1, lp_n2):
        n += 1
        cols, rows, ch = n1_lap.shape
        laplacian = np.hstack(
            (n1_lap[:, 0:int(cols/2)], n2_lap[:, int(cols/2):]))
        n1_n2_pyramid.append(laplacian)

    '''
    Reconstruct
    '''
    reconstruct = n1_n2_pyramid[0]
    for i in range(1, 6):
        reconstruct = cv.pyrUp(reconstruct)
        reconstruct = cv.add(n1_n2_pyramid[i], reconstruct)

    # cv.imshow("Nature 1", n1)
    # cv.imshow("Nature 2", n2)
    cv.imshow("N1_N2", reconstruct)
    cv.waitKey()
    cv.destroyAllWindows()


if __name__ == "__main__":
    blend()

Image Blending OpenCV Documentation

Learn more about image Blending in official OpenCV Documentation.