OpenCV and Image Processing - Bitwise Operation

The last section introduced image blending by implementing the algorithm introduced by the OpenCV document. As you can see from the result in Figure from previous post, the image looks a little bit faded out, and both original images become transparent to some extent. It depends on what effects you want to achieve, sometimes this kind of faded-out effect is not ideal, you might want the images to be opaquely added together without fading out.

This post will introduce another way to blend two images, a blending mask will be used in the image blending. The blending mask is a grayscale image that specifies the contribution of each pixel in the input images to the output image. It is also referred to as an alpha mask or alpha matte. The blending mask is used as a weighting function to compute the weighted average of the pixel values from the input images. The values of the blending mask typically range from 0 to 255, where 0 represents complete opacity and 255 represents complete transparency. The blending mask is usually a grayscale image, but it can also be a color image in some cases. However, in this section we use a binary mask, the values are either 0 or 255, with no values in between.

In this post we will create a mask from the image of the bird, and apply it to both images as a bitwise operation to achieve a different blending effect.

OpenCV provides bitwise operations – AND, OR, NOT and XOR. It is very useful when we want to extract something partially from an image and put it in another image. These bitwise operations can be used here to achieve the effects we want.

The algorithm is as following:

g( i , j ) = mask & f0 ( i , j ) + (1 − mask) & f1 ( i , j )

f0 ( i , j ) and f1 ( i , j ) are the original images to blend, and g( i , j ) is the result image; & is the bitwise AND operation.

The bitwise blending process is shown in Figure 1, a mask is created based on Original Image 2 which is bird image. And (1 - mask) is created based on the mask.

Figure 1 Blending Images with Bitwise

The mask is applied to Original Image 2 by bitwise AND, and produces the foreground of the output image, only the bird appears transparently in the foreground image, and other pixels are opaque and appears as black in the foreground, as shown in Figure 2.

Then (1 -mask) is applied to the Original Image 1 by bitwise AND, and produces the background image. The bird appears black here, and all other pixels appear transparently.

Finally, the foreground and background are joined together by bitwise OR to produce the result, as shown in Figure 2.

There are different ways to create a mask based on an image. An effective way to create a mask is introduced in Section 6.8.1 later, the image is converted to HSV color space, and find out the lower and upper range of HSV value to pick up the interested part of the image, in our case the bird. Then use cv2.inRange() function to get the mask. And lastly convert the mask image from grayscale to BGR color space.

Below are the code snippets to create our mask.

1 def remove_background_by_color(self,hsv_lower,hsv_upper,image=None):

2 if image is None:

3 image = self.image

4 imgHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

5 mask = cv2.inRange(imgHSV, hsv_lower, hsv_upper)

6 mask = 255 - mask

7 mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)

8 bg_removed = cv2.bitwise_and(image, mask)

9 return bg_removed, mask

Figure 2 below shows the foreground image by applying the mask on the bird image with bitwise AND operation.

Foreground Image After mask Applied

As shown in the above result, all the background from the original image is removed, only the bird is left, which is the interesting part of this image. This implements the first part of the above formula:

g( i , j ) = mask & f0 ( i , j ) + (1 − mask) & f1 ( i , j )

We'll continue in next post.

Read More

OpenCV and Image Processing - Blend Image

Blending images refers to the process of combining two images to create a single image that has the characteristics of both original images. The result is a combination of the corresponding pixel values of the two original images with weights.

The blending process involves specifying a weight for each pixel in one of the original images, computing (1 – weight) for each pixel in another image, and then merging them together to produce the output image. Blending images is a common operation in computer vision and image processing, and is used for a variety of applications, such as creating panoramas, compositing images, and generating special effects in videos and images.

Say, there are two images, Original Image 1 and Original Image2, as shown in Figures below.

 Blend Image: Original Image 1

Blend Image: Original Image 2

The OpenCV documents describe the algorithm to blend them together,

g( i , j ) = α f0 ( i , j ) + (1 − α) f1 ( i , j )

from OpenCV official document at https://docs.opencv.org/4.7.0/d5/dc4/tutorial_adding_images.html

f0 ( i , j ) and f1 ( i , j ) are the two original images, and g( i , j ) is the output image. α is the weight (value from 0 to 1) on the first image, (1 – α) is the weight on the second image.

By adjusting the value of α, we can achieve different blending effects. Same as the previous sections, cv2.addWeighted() function is used to blend the two images.

The blend() function is to be added to the ImageProcessing class:

1 def blend(self, blend, alpha, image=None):

2 if image is None:

3 image = self.image

4 blend = cv2.resize(blend, (image.shape[1], image.shape[0]))

5 result = cv2.addWeighted(image, alpha, blend, (1.0 - alpha), 0)

6 return result

Explanations:

Line 1 Define blend() function, the parameters are the image to blend, and alpha which is between 0 and 1.

Line 4 The two images must be in the same size, make them the same using cv2.resize().

Line 5 Use cv2.addWeighted() function to implement the blending algorithm.

Here are the main codes:

A trackbar is created for adjusting alpha from 0 to 100, with a default value of 50. The callback function change_alpha() divides the value by 100, then it’s from 0 to 1.0.

The result looks something like Figure shown below.

The default alpha of 50 makes the two original images blend evenly. By adjusting the alpha value, the output image will be changing. If alpha=100, the result becomes pure Original Image 1; if alpha=0, the result is pure Original Image 2. If the alpha is in between, the result is mixed, but the weight of each image is different based on the alpha value.

Read More

OpenCV and Image Processing - Adjust Hue, Saturation and Value

We have already explained a color image can be split not only to BGR channels but also HSV channels, the former represents blue, green and red channels, the latter represents hue, saturation and value. The hue represents the color of the image, different hue values have different colors; saturation represents the purity of the color, a higher saturation means a more colorful image; while value represents the brightness of the image, a higher value means a brighter image.

In this post we will explain how to adjust hue, saturation and value of an image, three trackbars will be created for the three variables, we will see how the image is changing when users adjust these variables in real-time.

The first thing is to convert the image into HSV using cv2.cvtColor() with cv2.COLOR_BGR2HSV parameter, then split it into hue, saturation and value channels using cv2.split().

Then add weights to hue, saturation and value channels respectively, and merge them back to HSV. Finally convert the HSV to BGR for displaying.

Same as the previous section, cv2.addWeighted() is used to adjust each channel with a weight. The below line of code is to add a weight h_weight to the hue channel,

1 hue = cv2.addWeighted(hue, 1.0, zeros, 0, h_weight)

And add the weights to other channels in the same way.

Here are the codes, the function hue_saturation_value() is added into our ImageProcessing class:

1 def hue_saturation_value(self, hue, saturation,

value, image=None):

2 if image is None:

3 image = self.image

4 hsvImage = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

5 h, s, v = cv2.split(hsvImage)

6 zeros = np.zeros(h.shape, h.dtype)

7 h = cv2.addWeighted(h, 1.0, zeros, 0, hue)

8 s = cv2.addWeighted(s, 1.0, zeros, 0, saturation)

9 v = cv2.addWeighted(v, 1.0, zeros, 0, value)

10 result = cv2.merge([h, s, v])

11 result = cv2.cvtColor(result, cv2.COLOR_HSV2BGR)

12 return result

Explanations:

Line 1 Define the function in ImageProcessing class, the parameters are hue, saturation and value.

Line 4 Convert the image into HSV image.

Line 5 Split the HSV image into separate h, s, v channels.

Line 7 - 9 Add the weights to the three channels. The hue, saturation and value are the weights to h, s, and v channels respectively.

Line 10 Merge the h, s, v channels into the HSV image.

Line 11 Convert the HSV image back to BGR image.

This function is called from the HueSaturation.py file, in this example we add a feature to save the image into a file on the local disk drive, in addition to pressing ESC key to exit, when press “s” key the file will be saved to a specified file using cv2.imwrite() function.

1 import cv2

2 import common.ImageProcessing as ip

3

4 def change_hue(value):

5 global hue

6 hue = (value * 2) – 255

7

8 def change_saturation(value):

9 global sat

10 sat = (value * 2) – 255

11

12 def change_value(value):

13 global val

14 val = (value * 2) – 255

15

16 if __name__ == "__main__":

17 hue, sat, val = 0, 0, 0

18 title = "Adjust Hue, Saturation and Value"

19

ip = ip.ImageProcessing(title,

"../res/flower003.jpg")

20

cv2.createTrackbar('Hue', title, 127, 255,

change_hue)

21

cv2.createTrackbar('Saturation', title,

127, 255, change_saturation)

22

cv2.createTrackbar('Value', title, 127,

255, change_value)

23 ip.show("Original")

24

print("Press s key to save image,

ESC to exit.")

25

26 while True:

27

adjusted_image =

ip.hue_saturation_value(hue, sat, val)

28 ip.show(image=adjusted_image)

29 ch = cv2.waitKey(10)

30 if (ch & 0xFF) == 27:

31 Break

32 elif ch == ord('s'):

33 # press 's' key to save image

34 filepath = "C:/temp/flower003.png"

35 cv2.imwrite(filepath, adjusted_image)

36 print("File saved to " + filepath)

37 cv2.destroyAllWindows()

Explanations:

Line 4 - 15 Define three callback functions for trackbars for hue, saturation and value. The value is changing from -255 to 255 for each. You can modify the range base on your needs.

Line 19 Instantiate the ImageProcess class with a title and an image.

Line 20 - 22 Create three trackbars for hue, saturation and value respectively.

Line 23 Show the original image.

Line 27 Call the function we just defined in ImageProcess class with parameters of the adjusted hue, saturation and value.

Line 28 Show the result image.

Line 32 - 36 Save the result image when pressing “s” key.

Execute the code, the result shows the original image and the adjusted image together with three trackbars, as Figure 5.10, for adjusting hue, saturation and value respectively.

The original image and adjusted image are omitted here, feel free to change different images in the codes and observe how they are changing when adjusting the hue, saturation and value by moving the knobs of the trackbars.

Read More

OpenCV and Image Processing - Adjust Contrast and Brightness of an Image

We already saw how to adjust the brightness and contrast for a webcam, however unfortunately OpenCV doesn’t provide functions to adjust the brightness and contrast for an image. But OpenCV official document recommends an alternative way to do it, see the link at:

https://docs.opencv.org/4.7.0/d3/dc1/tutorial_basic_linear_transform.html

We dot not intend to deep dive into the algorithms or theories, instead, we focus on the implementation of the techniques using OpenCV with Python. However, in order to make things clear, sometimes we will summarize, or quote from other sources, the related algorithms or theories.

To summarize the techniques of contrast and brightness adjustment, the below formula from OpenCV official document depicts the calculation of the contrast and brightness, 

f ( i , j ) is the original image and g( i , j ) is the result. α changes the contrast, greater than 1 for higher contrast, less than 1 for lower contrast. It should be greater than 0. β changes the brightness, values vary from -127 to +127. ( i , j ) indicates the coordinates of the image pixels.

The OpenCV official document provides an example to explain how to use the Python array and matrix operations to implement the above formula. Alternatively, OpenCV provides cv2.addWeighted() function which is designed for blending two images with weight added on each, we will use this function to implement the above formula to adjust the contrast and brightness.

cv2.addWeighted(image, contrast, zeros, 0, brightness)

The function cv2.addWeighted() implements the below formula:

g( i , j ) = α1 f1 ( i , j ) + α2 f2 ( i , j ) + β

g( i , j ) is the result, f1( i , j ) is the image and passed to cv2.addWeighted() as the first parameter, α1 is the second parameter which is contrast; because only one image is interested in this example, so f2( i , j ) is an all-zero image, and α2 is also a zero, therefore the third and fourth parameters of the function are an all-zero array and a zero value. β is the last parameter, which is brightness, this value will be added to the result.

Now add a new contrast_brightness() function to the class ImageProcessing, which we have defined in previous posts.

45

def contrast_brightness(self, contrast,

brightness, image=None):

46

# contrast:

# between 0 and 1: less contrast;

47 # > 1: more contrast;

48 # 1: unchanged

49 # brightness: -127 to 127;

50 # 0 unchanged

51 if image is None:

52 image = self.image

53 zeros = np.zeros(image.shape, image.dtype)

54

result = cv2.addWeighted(image, contrast,

zeros, 0, brightness)

55 return result

Line 45 Define contrast_brightness() function, specify contrast and brightness as parameters, and optionally an image.

Line 53 Create an all-zero array, because cv2.addWeighted() requires two images, we don’t use the second one, so pass the all-zero array to it.

Line 54 Call cv2.addWeighted() function to apply the weights to the image.

Below is the code to adjust the brightness and contrast using the function just created above. Two trackbars are added for users to adjust the brightness and contrast.

1 import cv2

2 import common.ImageProcessing as ip

3

4 def change_brightness(value):

5 global brightness

6 brightness = value - 128

7

8 def change_contrast(value):

9 global contrast

10 contrast = float(value)/100

11

12 if __name__ == "__main__":

13 brightness = 0

14 contrast = 1.0

15 title = "Adjust Brightness and Contrast"

16 ip = ip.ImageProcessing(title,

"../res/flower003.jpg")

17 cv2.createTrackbar('Brightness', title, 128,

255, change_brightness)

18 cv2.createTrackbar('Contrast', title, 100,

300, change_contrast)

19 20 while True:

21 adjusted_image = ip.contrast_brightness(

contrast, brightness)

22 ip.show(image=adjusted_image)

23 if cv2.waitKey(10) & 0xFF == 27:

24 break

25 cv2.destroyAllWindows()

Line 2 Import the ImageProcessing class.

Line 4 - 6 Define the callback function for the trackbar to adjust the brightness.

Line 8 - 10 Define the callback function for the trackbar to adjust the contrast.

Line 12 Main entrance.

Line 16 Instantiate the ImageProcessing object.

Line 17 - 18 Create two trackbars for brightness and contrast.

Line 21 – 22 Call contrast_brightness(), the function we just defined, to adjust brightness and contrast.

Execute the code, the result shows the image together with two trackbars, as Figure shown below, one is for adjusting brightness and another for contrast:

Use the mouse to drag the two trackbars to change the brightness and contrast values, the image will be changed accordingly in real-time. The range of contrast in trackbar is from 0 to 300, and the default is 100. In the callback function change_contrast(), the value is divided by 100, therefore the contrast range is from 0.0 to 3.0, default is 1.0.

Feel free to play with the codes by changing the image files and adjusting the two trackbars and observe how the image is changing accordingly.

Read More

OpenCV and Image Processing - Resize, Crop and Rotate an Image continued...

Now the class can perform the resize, crop and rotate operations. In the source codes of ResizeCropRotate.py the class is imported in Line 2, this is a typical way to include another Python file for reference.

1 import cv2

2 import common.ImageProcessing as ip

3

4 if __name__ == "__main__":

5 # Create an ImageProcessing object

6 ip = ip.ImageProcessing("Resize,Crop and Rotate", "../res/flower005.jpg")

7

8 # Show original image

9 ip.show()

10

11 # Resize the original image and show it

12 resized_image = ip.resize(50)

13 ip.show("Resized -- 50%", resized_image)

14

15 # Rotate the resized image and show it

16 rotated_image = ip.rotate(45, resized_image)

17 ip.show("Rotated -- 45 degree", rotated_image)

18

19 # Crop the original image and show it

20 cropped_image = ip.crop((300, 10), (600, 310))

21 ip.show("Cropped", cropped_image)

22

23 cv2.waitKey(0)

24 cv2.destroyAllWindows()

Below are the results, First Figure shows the original image and the second shows the 50% resized image:

Figures below shows the cropped image and the 45-degree rotated image:

Feel free to play with the source codes, for example change an image file, change the degree of rotation, change the percentage of the resizing, and observe how the image is processed.

Read More

OpenCV and Image Processing - Resize, Crop and Rotate an Image

Python is an excellent language to support object-oriented programming, this post will start to use object-oriented techniques to write the codes. We will create classes to include properties and functions, then instantiate the classes and invoke their functions.

Now create a class called ImageProcessing, and define three functions inside, resize(), rotate() and crop(). When a class is created it always has a built-in __init__() function, which is always executed when the class is instantiated. It’s used to assign initial values to the properties or other operations that are necessary when the object is created. Here we set the window_name and image_name properties inside __init__() function.

1 import cv2

2

3 class ImageProcessing(object):

4 def __init__(self, window_name, image_name):

5 self.window_name = window_name 

6 self.image_name = image_name

7 self.image = cv2.imread(self.image_name)

8

9 def show(self, title=None, image=None):

10 if image is None:

11 image = self.image

12 if title is None:

13 title = self.window_name

14 cv2.imshow(title, image)

To continue, add three functions, resize(), crop() and rotate(), to the class:

16 def resize(self, percent, image=None):

17 if image is None:

18 image = self.image

19 width = int(image.shape[1]*percent/100)

20 height = int(image.shape[0]*percent/100)

21 resized_image = cv2.resize(image, (width, height) )

22 return resized_image

23

24 def crop(self,pt_first,pt_second,image=None):

25 if image is None:

26 image = self.image

27 # top-left point

x_tl, y_tl = pt_first

28 # bottom-right point

x_br, y_br = pt_second

29 # swap x value if opposite

if x_br < x_tl:

30 x_br, x_tl = x_tl, x_br

31 # swap y value if opposite

if y_br < y_tl:

32 y_br, y_tl = y_tl, y_br

33 cropped_image=image[y_tl:y_br,x_tl:x_br]

34 return cropped_image

35

36 def rotate(self,angle,image=None,scale=1.0):

37 if image is None:

38 image = self.image

39 (h, w) = image.shape[:2]

40 center = (w / 2, h / 2)

41 rot_mat = cv2.getRotationMatrix2D(center, angle, scale)

42 rotated_image = cv2.warpAffine(image, rot_mat, (w, h))

43 return rotated_image

The class definition for ImageProcessing is done for now, more functions will be added later.

Read More

OpenCV and Image Processing - Conversion of Color Spaces (Convert BGR to HSV and vice versa)

As explained earlier, an image can be represented not only in BGR but also in HSV color spaces, OpenCV can easily convert the image between BGR and HSV. Like converting it to grayscale, the same function cv2.cvtColor() is used but the second parameter is different, cv2.COLOR_BGR2HSV in this case.

The HSV image can also be split into three channels, hue, saturation and value. Then each channel can be adjusted to achieve some different effects, later we will explain how to adjust each channel to change the image.

Below code snippets are to convert an image to HSV color space:

1 def convert_bgr2hsv(image):

2 hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

3 return hsv 

As shown in Figure below, the color image is converted to HSV color space, and then it can be further split into three channels in hue, saturation and value by using cv2.split() function.

Convert HSV to BGR

When OpenCV displays an image, the function cv2.imshow() is used and by default in BGR color space. If an image is converted to HSV and followed by some image processing operations, it can not be directly sent to cv2.imshow() for displaying, instead it must be converted back to BGR, then sent to cv2.imshow() for displaying, otherwise the image will not be displayed correctly.

In last section an image is converted to HSV, and here we will convert it back to BGR with the same function cv2.cvtColor() but different parameter cv2.COLOR_HSV2BGR.

1 def convert_hsv2bgr(image):

2 bgr = cv2.cvtColor(image, cv2.COLOR_HSV2BGR)

3 return bgr

Read More

OpenCV and Image Processing - Conversion of Color Spaces (Convert Grayscale to BGR)

A grayscale image can also be converted to BGR color space, in this case the source is a grayscale image which only has one channel and does not have any color information, therefore the result BGR image looks the same as the original grayscale one. But the difference is the BGR image has three channels although looks the same as the grayscale one, while the original gray image only has one channel.

To convert a grayscale image to BGR, use cv2.cvtColor() function with cv2.COLOR_GRAY2BGR as the second parameter.

1 def convert_gray2bgr(image):

2 bgr = cv2.cvtColor(image,cv2.COLOR_GRAY2BGR)

3 return bgr

Figure below shows the idea:

The left image is the original grayscale image, which has only one channel. The right image, however, is in BGR color space although looks same as the grayscale one, it can be split into three channels. Since the right image looks same as the left one, why we should convert it? There are several reasons to do it, sometimes in the image processing, some operations need to be performed on two or more images, it’s important that all the images are in the same color space. For example, if we want to blend, or mix, two images together, but one is color image and another is grayscale, the grayscale one must be converted into BGR before blending them.

Sometimes when you want to colorize a grayscale image, it must be converted to BGR first, then paint colors on different channels, because you are not able to paint colors on a single-channel grayscale image.

Read More

OpenCV and Image Processing - Conversion of Color Spaces

Image processing is a technique to apply some mathematical algorithms and use various functions and techniques to manipulate digital images in order to get the desired outcomes with enhanced effects, such as improving their quality or extracting useful information from them.

OpenCV provides a number of methods for image processing, we will look into some commonly and widely used methods for image processing. 

Conversion of Color Spaces

We have already seen that the color image can be represented using either BGR or HSV color spaces, and a color image can also be converted to a grayscale image.OpenCV provides many methods for the conversion of color spaces, here we introduce two of them: BGR to and from Grayscale, and BGR to and from HSV.

Convert BGR to Gray

There are two ways to obtain a grayscale image from the color one, 1) use cv2.cvtColor() function with cv2.COLOR_BGR2GRAY as its second parameter, and 2) load the image with grayscale mode by using cv2.imread() function with cv2.IMREAD_GRAYSCALE as the second parameter.

The first method loads the color image and then converts it to grayscale, then we have both the color and grayscale images available in the memory. However, if the color image is not needed, the second method loads the image in grayscale only, the color one is not available in this case and could save some memory. Below code snippets show the two methods:

1 def convert_bgr2gray(image):

2 gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

3 return gray

4

5 def load_image_gray(file_name):

6 gray = cv2.imread(file_name, cv2.IMREAD_GRAYSCALE)

7 return gray

Line 1 to 3 is the first method, the color image in image is passed as a parameter, cv2.cvtColor() function will convert it to grayscale one.

Line 5 to 7 is the second method, it loads the image in grayscale mode directly from the file.

As shown in Figure above, the color image (left) is converted to a grayscale one (right).

As discussed earlier, in OpenCV the default color space for an image is BGR, so a color image can be split into three channels of blue, green and red, each channel is in grayscale, as shown in Figure below.

The cv2.split() function can be used to split a color image into the three channels:

1 def split_image(image):

2 ch1, ch2, ch3 = cv2.split(image)

3 return ch1, ch2, ch3

And each channel is in grayscale.

Read More