OpenCV and Image Processing - Histogram for Grayscale Images

There are two ways to compute and display histograms. First, OpenCV provides cv2.calcHist() function to compute a histogram for an image, second, use matplotlib to plot the histogram diagram, matplotlib is a Python library for creating static, animated, and interactive visualizations.

Figure 7 shows the histogram of a real image. 

Figure 7

Look at a specific point, i.e the • point in the histogram plot at the right-side of Figure 7, it means there are about 1,200 pixels with color value of 50 in the leftside grayscale image.

This is how to read the histogram diagram which gives an overall idea of how the color value is distributed.

Here are the codes to produce the histogram in Figure 7:

1 # Get histogram using OpenCV

2 def show_histogram_gray(image):

3 hist = cv2.calcHist([image], [0], None, [256], [0, 256])

4 fig = plt.figure(figsize=(6, 4))

5 fig.suptitle('Histogram - using OpenCV', fontsize=18)

6 plt.plot(hist)

7 plt.show()

The second way to display a histogram is to use matplotlib, which provides plt.hist() function to generate a histogram plot, it does the exact same thing just looks a little bit differently, as Figure 8:

Here are the codes to produce the histogram in Figure 8-

9 # Alternative way for histogram using matplotlib

10 def show_histogram_gray_alt(image):

11 fig = plt.figure(figsize=(6, 4))

12 fig.suptitle('Histogram - using matplotlib',fontsize=18)

13 plt.hist(image.ravel(), 256, [0, 256])

14 plt.show()

Explanations:

Line 1 - 7 Use OpenCV cv2.calchist to generate a histogram.

Line 3 Call cv2.calcHist() function, pass the image as the parameter.

Line 4 - 6 Create a plot using matplotlib, specify the plot size, set the title, and plot the histogram created in line 3.

Line 7 Show the plot

Line 10 - 14 Alternatively, use matplotlib function to generate a histogram.

Line 11 -12 Create a plot using matplotlib, specify the plot size, set the title.

Line 13 Call plt.hist() function to create a histogram of the image.

Read More

OpenCV and Image Processing - Histogram

A histogram is a graphical representation of the distribution of pixel values in an image. In image processing, a histogram can be used to analyze the brightness, contrast, and overall intensity of an image. It is plotted in a x-y chart, the x-axis of a histogram represents the pixel values ranging from 0 to 255, while the y-axis represents the number of pixels in the image that have a given value. A histogram can show whether an image is predominantly dark or light, and whether it has high or low contrast. It can also be used to identify any outliers or unusual pixel values.

To better understand the histogram, plot an image and draw some squares and rectangles and fill them with the color value of 0, 50, 100, 150, 175, 200 and 255, as shown in the left-side of Figure 6. The coordinates of each square/rectangle are also displayed in the image, so we can easily calculate how many pixels for each color. 

In the histogram plot in the right-side, x-axis is the color value from 0 to 255, and yaxis is the number of pixels. The y value is 20,000 at x=0, meaning there are 20,000 pixels that have a color value of 0, from the left-side image we can see the black rectangle (color: 0) is from (0, 0) to (200, 100), the total number of black pixels is 200 x 100 = 20,000. The histogram plot also shows the color value 0 has 20,000 pixels. The white square (color: 255) has 40,000 pixels. In the same way, calculate the number of pixels for other colors, there are 20,000 pixels for color values of 50, 100, 150, 175 and 200.

This is how the histogram works, the number of pixels and the color values are shown in the histogram.

Below is the code to create the above image and histogram plot.

1. def show_histogram():
2. img = np.zeros((400, 400), np.uint8)
3. cv2.rectangle(img, (200,0), (400, 100), (100), -1)
4. cv2.rectangle(img, (0, 100), (200, 200), (50), -1)
5. cv2.rectangle(img, (200, 100), (400, 200), (150), -1)
6. cv2.rectangle(img, (0,200), (200, 300), (175), -1)
7. cv2.rectangle(img, (0, 300), (200, 400), (200), -1)
8. cv2.rectangle(img, (200,200), (400, 400), (255), -1)
9. fig = plt.figure(figsize=(6, 4))
10. fig.suptitle('Histogram', fontsize=20)
11. plt.xlabel('Color Value', fontsize=12)
12. plt.ylabel('# of Pixels', fontsize=12)
13. plt.hist(img.ravel(), 256, [0, 256])
14. plt.show()

Explanations:

Line 2 Create a numpy array as a blank canvas with all zeros.

Line 3 - 8 Draw squares and rectangles and fill them with specific color values.

Line 9 – 12 Define a plot using matplotlib library, set title and X, Y-axis labels.

Line 13 Create a histogram using matplotlib function.

Line 14 Show the histogram plot

Histograms can be used for various purposes in image processing, below is a list of some of the use cases, Image equalization, by modifying the distribution of pixel values in the histogram, it is possible to improve the contrast and overall appearance of an image.

Thresholding, by analyzing the histogram, it is possible to determine the optimal threshold value for separating the foreground and background of an image. Color balance, by analyzing the histograms of individual color channels, it is possible to adjust the color balance of an image.

In conclusion, histograms are an important tool in image processing for analyzing and manipulating the distribution of pixel values in an image.

Read More

OpenCV and Image Processing - Blur Image (Median Blur)

Same as Gaussian Blur, the Median Blur is also widely used in image processing, it is often used for noise reduction purposes. Similar to the Gaussian Blur filter, instead of applying Gaussian formula, the Median Blur calculates the median of all the pixels inside the kernel filter and the central pixel is replaced with this median value. OpenCV also provides a function for this purpose, cv2.medianBlur().

Here is the code in ImageProcessing class to apply the median blur function, 

1 def median_blur(self, ksize=1, image=None):

2 if image is None:

3 image = self.image

4 result = cv2.medianBlur(image, ksize)

5 return result

In BlurImage.py file the codes for Median Blur are quite similar to the Gaussian Blur, a trackbar can change the kernel size, and the effects can be observed in real time. Below shows the original image vs. the median-blurred image.

Read More

OpenCV and Image Processing - Blur Image (Gaussian Blur)

Now we add the Gaussian blur function to the ImageProcessing class.

1 def blur(self, ksize=(1,1), image=None):

2 if image is None:

3 image = self.image

4 if ksize[0] % 2 == 0:

5 ksize = (ksize[0] + 1, ksize[1])

6 if ksize[1] % 2 == 0:

7 ksize = (ksize[0], ksize[1] + 1)

8 result = cv2.GaussianBlur(image,ksize, cv2.BORDER_DEFAULT)

9 return result 

Explanations:

Line 1 Define the blur() function, pass ksize as parameter.

Line 4 – 7 ksize must be odd numbers, check ksize if not odd then change it to odd.

Line 8 Invoke cv2.GaussianBlur() function.

BlurImage.py file has the source codes to perform the Gaussian blur, a trackbar is added to change the kernel size, by changing the kernel size we can observe how the blurring effects are different. In this example a square kernel is used, meaning the width and height are equal. Feel free to modify the codes to use a rectangle kernel and observe the blurring effects.

1 import cv2

2 import common.ImageProcessing as ip

3

4 def change_ksize(value):

5 global ksize

6 if value % 2 == 0:

7 ksize = (value+1, value+1)

8 else:

9 ksize = (value, value)

10

11 if __name__ == “__main__”:

12 global ksize

13 ksize = (5,5)

14 iproc = ip.ImageProcessing(“Original”,

“../res/flower003.jpg”)

15 iproc.show()

16

17 cv2.namedWindow(“Gaussian Blur”)

18 cv2.createTrackbar(“K-Size”, “Blur”, 5, 21,

change_ksize)

19

20 while True:

21 blur = iproc.blur(ksize)

22 iproc.show(“Blur”, blur)

23

24 ch = cv2.waitKey(10)

25 # Press ‘ESC’ to exit

if (ch & 0xFF) == 27:

26 break

27 # Press ‘s’ to save

elif ch == ord(‘s’):

28 filepath = “C:/temp/blend_image.png”

29 cv2.imwrite(filepath, blur)

30 print(“File saved to “ + filepath)

31 cv2.destroyAllWindows()

Explanations:

Line 4 – 9 Trackbar callback function, get kernel size from trackbar, and change it to odd number if it’s not.

Line 14 – 15 Instantiate the ImageProcessing class with an image and show it as original image.

Line 17 – 18 Create a trackbar in another window called “Gaussian Blur”.

Line 21 Call blur() function defined above in ImageProcessing class, pass the kernel size as a parameter.

Line 22 Show the blurred image in the “Gaussian Blur” window.

Figure 4 is the result, as the K-Size trackbar is changing, the degree of blurring is also changing in real-time.

Read More

OpenCV and Image Processing - Blur Image

Blurring an image is a common image processing technique used to reduce noises, smooth out edges, and simplify the image. This post will introduce two types of image blurring techniques, Gaussian Blur and Median Blur. 

Gaussian Blur

Gaussian Blur is a widely used effect for image processing and is available in many image-editing software. Basically it reduces noises and hides details of the image, and makes the image smoothing. Figure 1 below shows what it looks like, the left one is the original image, and the right one is a Gaussian blurred image. By changing the Kernel size, the level of blurring will be changed.

Figure 1 Gaussian Blur

Let’s take a close look at how the Gaussian blur works, notice the small area highlighted by a white square pointed by a white arrow in Figure 1 above. Now zoom in this area and show the details in pixel level, as shown in Figure 2, this square is 9 by 9 pixels. Apply a Gaussian blur on this 9 × 9 area, we get the rightside image in Figure 2, the edge of the flower becomes blurred. The size of this area is called kernel size, in this example the area is a square of 9 × 9 pixels, then its kernel size is 9. It doesn’t have to be a square, it could be a rectangle, say 5 × 9 pixels. However, the kernel size must be odd numbers.

This small area is called filter, the Gaussian filter will be applied throughout the entire image starting from the top-left corner, moving from left towards right and from top towards down until the bottom-right corner, in another word the filter swept over the whole image, this is the image blurring process.

Figure 2

Now, let’s look at how the Gaussian function works, in other words how the filter is calculated and applied to the area.

Gaussian filter does not simply calculate the average value of the area, in this case the 9 × 9 area. Instead, it calculates a weighted average of the value for this area, the pixels near the center get more weights, and the pixels far away from the center get less weights. And the calculation is done on a channel-by-channel basis, which means it calculates the blue, green and red channels respectively.

Let’s dig a little deeper and see how the Gaussian function works, this is the formula in one dimension:

However, a pixel is determined by x and y coordinates in an image, so the two dimensional Gaussian formula should be used for image processing, it’s shown as Figure 3.

Figure 3

The two-dimensional Gaussian formula is:

Now imagine the Gaussian filter mentioned above, in our case an area of 9 x 9 pixels, is overlaying on the vertex of the above plot, the height is the weight for calculation. The pixels near the center are more important and then get more weights, the pixels near the edge are less important and then get less weights.

This is a basic idea of how Gaussian blur works in general, although the algorithm is not as simple as described above. Fortunately, OpenCV provides a function for this purpose, cv2.GaussianBlur(), and all the complexities are hidden behind the scenes. All we need to do is to call this function and specify the kernel size in width and height, which doesn’t have to be the same, but must be odd numbers.

In the next post we'll implement Gaussian blur function.

Read More

OpenCV and Image Processing - Warp Image contd...

In this post, as an example in Figure shown below, the left picture is taken by a tablet camera from the homepage of OpenCV.org, the picture looks distorted, the perspective warping will be used to correct it and make it aligned properly, the result will be shown in the right side of Figure shown below.

OpenCV provides two functions to perform perspective warping, cv2.getPerspectiveTransform() is to calculate the transformation matrix, it takes the four source points and four target points as input. Then cv2.warpPerspective() is to perform the perspective warping, it takes the original image, the transformation matrix and the output size.

We will take the four source points from the original image, which indicate the area to transform, and then use them to do the perspective warping. In the ImageProcessing class, add the function perspective_ warp():

1 def perspective_warp(self, points, width, height image=None):

2 if image is None:

3 image = self.image

4 pts_source = np.float32([points[0], points[1],

points[3], points[2]])

5 pts_target = np.float32([[0, 0],[width, 0],

[0,height],[width,height]])

6 matrix = cv2.getPerspectiveTransform(

pts_source, pts_target)

7 result = cv2.warpPerspective(

image,matrix,(width,height))

8 return result

Explanations:

Here are the source codes-

1 import cv2

2 import numpy as np

3 import common.Draw as dw

4 import common.ImageProcessing as ip

5 drawing = False

6 final_color = (0, 0, 255)

7 drawing_color = (0, 0, 125)

8 width, height = 320, 480

9 points = []

10 def on_mouse(event, x, y, flags, param):

11 global points, drawing, img, img_bk, iproc, warped_image

12 if event == cv2.EVENT_LBUTTONDOWN:

13 drawing = True

14 add_point(points, (x, y))

15 if len(points) == 4:

16 draw_polygon(img, points, (x, y), True)

17 drawing = False

18 img_bk = iproc.copy()

19 warped_image = iproc.perspective_warp(points,

width, height)

20 points.clear()

21 iproc.show("Perspective Warping", image=warped_image)

22 elif event == cv2.EVENT_MOUSEMOVE:

23 if drawing == True:

24 img = img_bk.copy()

25 draw_polygon(img, points, (x, y))

26 27 def add_point(points, curt_pt):

28 print("Adding point #%d with position(%d,%d)"

29 % (len(points), curt_pt[0], curt_pt[1]))

30 points.append(curt_pt)

31

32 def draw_polygon(img, points, curt_pt, is_final=False):

33 if (len(points) > 0):

34 if is_final == False:

35 dw.draw_polylines(img, np.array([points]),

False, final_color)

36 dw.draw_line(img, points[-1], curt_pt,

drawing_color)

37 else:

38 dw.draw_polylines(img, np.array([points]),

True, final_color)

39 for point in points:

40 dw.draw_circle(img, point, 2, final_color, 2)

41 dw.draw_text(img, str(point), point,

color=final_color,

font_scale=0.5)

42

43 def print_instruction(img):

44 txtInstruction = "Left click to specify four

points to warp image.

ESC to exit, 's' to save"

45 dw.draw_text(img,txtInstruction, (10, 20), 0.5,

(255, 255, 255))

46 print(txtInstruction)

47

48 if __name__ == "__main__":

49 global img, img_bk, iproc, warped_image

50 title = "Original Image"

51 iproc = ip.ImageProcessing(title,

"../res/skewed_image001.jpg")

52 img = iproc.image

53 print_instruction(img)

54 img_bk = iproc.copy()

55 cv2.setMouseCallback(title, on_mouse)

56 iproc.show()

57 while True:

58 iproc.show(image=img)

59 ch = cv2.waitKey(10)

60 if (ch & 0xFF) == 27:

61 break

62 elif ch == ord('s'):

63 # press 's' key to save image

64 filepath = "C:/temp/warp_image.png"

65 cv2.imwrite(filepath, warped_image)

66 print("File saved to " + filepath)

67 cv2.destroyAllWindows()

The source codes are not explained line by line here, because it’s basically the same as the one for drawing polygons discussed before. Execute the codes in WarpImage.py, left click on the image to specify the four points in the same way as we did in drawing polygons, the coordinates are shown in the image. After all four points are collected, it will draw the polygon, and then call perspective_warp() function to transform the specified area and show the resulting image.

Read More

OpenCV and Image Processing - Warp Image

Warp image refers to the process of geometrically transforming an image into different shapes. It involves applying a perspective or affine transformation to the image, which can change its size, orientation, and shape.

This post introduces perspective warping, also known as perspective transformation, which is a type of image warping that transforms an image from one perspective to another. It is a geometric transformation that changes the viewpoint of an image, as if the observer's viewpoint has moved or rotated in space.

It is often used to correct the distortion caused by the camera's perspective when capturing an image. This distortion causes objects in the image to appear different in size and shape depending on their position in the image. For example, objects closer to the camera appear larger and more distorted than those farther away.

The perspective warping process includes defining four points in the original image and mapping them to four corresponding points in the output image. These points are known as the source points and destination points, respectively. Once the corresponding points are identified, a transformation matrix is calculated, which maps each pixel in the original image to its new location in the output image. As shown in Figure below, there is an original image in the left, and the output image in the right. The four source points from the original image are A, B, C and D, the perspective warping process will transform them to the output image in the right, so that A is mapped to A’, B to B’, C to C’ and D to D’.

The real-world use case is, for example, using a camera to take a picture of a document, there are some distortions that make the document looks like the left-side image where A, B, C and D are the four corners of the document. A perspective warping will be able to transform the document into the right-side image, which is corrected and aligned properly. 

Perspective warping is commonly used in computer vision and image processing applications, such as image correction, where in many cases images can be distorted due to the angle of the camera, by applying perspective warping the image can be corrected and aligned properly. Another example is image stitching where multiple images are combined to create a larger panorama, warping can be used to align the images properly so that they fit seamlessly.

Read More

OpenCV and Image Processing - Bitwise Operation contd....

Next is to create (1 – mask), technically this is 255 – mask, because the mask is created in a grayscale channel, which has values ranging from 0 to 255. However, in this case we deal it with the binary channel, the values are either 0 or 255, meaning either 100% opacity or 100% transparency, no other values in between. Although in other cases the values in between could indicate the percentage of transparency.

Figure 3 below shows the result of (1 – mask).

Then apply (1 – mask) to the second image as bitwise AND operation to produce

the background image, the result is shown in Figure 4 below.

The pixels of the bird are removed, and all others remain on the result image. This implements the second part of the formula:

g( i , j ) = mask & f0 ( i , j ) + (1 − mask) & f1 ( i , j )

Finally, the two images are merged by bitwise OR operation, or just simply add them up, the result is shown as Figure 5 below.

This implements the whole formula:

g( i , j ) = mask & f0 ( i , j ) + (1 − mask) & f1 ( i , j )

By comparing it with the one in the last section, the difference is obvious, this one does not have any faded-out effect.

In this case the foreground and background are merged with 100% transparency. In real-world projects, depending on what effects you want to achieve, different methods can be selected.

Below is the code snippet in the ImageProcessing class-

1 def blend_with_mask(self, blend, mask, image=None):

2 if image is None:

3 image = self.image

4 blend = cv2.resize(blend, image.shape[1::-1])

5 mask = cv2.resize(mask, image.shape[1::-1])

6 result = cv2.bitwise_and(blend, mask) +

cv2.bitwise_and(image,(255-mask))

7 return result

Line 1 Define blend_with_mask() function, the blend image and mask image are

passed as parameters.

Line 4 - 5 The mask and blend images must be in the same size as the original image, use cv2.resize() to make them the same dimension, in case they are not.

Line 6 Use bitwise AND on blend image and mask image to produce the foreground

image. 

Use bitwise AND again on the original image with (255-mask) to produce the

background image.

Then add both images using plus operation to make the resulting image.

cv2.bitwise_or() can also be used, they are the same in this case.

The full source codes for this example are in BlendImageBitwise.py file.

1 import cv2

2 import common.ImageProcessing as ip

3 4 def blendTwoImagesWithMask(imageFile1, imageFile2):

5 title = "Blend Two Images"

6 iproc = ip.ImageProcessing(title, imageFile1)

7 iproc.show(title="Original Image1",

image=iproc.image)

8 toBlend = cv2.imread(imageFile2)

9 iproc.show(title="Original Image2", image=toBlend)

10_, mask = iproc.remove_background_by_color(

hsv_lower = (90, 0, 100),

11 hsv_upper = (179,255,255),

12 Image = toBlend )

13 iproc.show(title="Mask from Original Image2",

image=mask )

14 iproc.show(title="(1-Mask)", image= (255-mask))

15 blend = iproc.blend_with_mask(toBlend, mask)

16 iproc.show(title=title, image=blend)

17 cv2.waitKey(0)

18 cv2.destroyAllWindows()

19

20 if __name__ == "__main__":

21 image1 = "../res/sky001.jpg"

22 image2 = "../res/bird002.jpg"

23 blendTwoImagesWithMask(image1, image2)

Line 10 – 12 is to call remove_background_by_color() function in ImageProcessing class to get the mask of the bird image. The value of parameters of hsv_lower and hsv_upper are specific only to this image, different images should have different values to get a mask. 

Read More