User Interaction in OpenCV- Input Values with Trackbars

A Trackbar, or a Slider, is a graphical user interface component that allows the user to adjust a parameter within a range by sliding a knob with the mouse along a horizontal bar, as shown in Figure below. It is commonly used in computer vision applications for real-time parameter tuning, for example, the users can adjust brightness from the range of 0 to 100 by dragging the knob along the bar. 

In OpenCV, cv2.createTrackbar() function is used to create a trackbar. The function takes several arguments, including the name of the trackbar, the name of the window in which it will appear, the initial value of the trackbar, the maximum value, and a callback function that will be called when the trackbar is moved. When a user moves the knob the callback function will be invoked, and the actions defined in the callback function will be performed.

Here are the code snippets to create a sample trackbar -

The range of the trackbar is from 0 to a maximum value of 100, a callback function on_trackbar()is called whenever the knob is moved by the mouse, and it simply prints the current value of the trackbar to the console.

The trackbar is displayed in a window named "Trackbar Window", which is created using the cv2.namedWindow() function, the content of the window is a light gray canvas as we did previously. Inside the main loop, cv2.imshow() is called to display the trackbar window and cv2.waitKey() to wait for a key press. When the user presses the ESC key, it breaks out of the loop and destroys the window using cv2.destroyAllWindows().

When the user interacts with the trackbar by moving the knob, the on_trackbar() function is called with the current value of the trackbar as its parameter. This value can then be used to adjust a parameter in real-time, such as the brightness for an image processing algorithm.

The below example will display a webcam, together with four trackbars for adjusting the brightness, contrast, saturation and hue of the webcam.

Open the UserInputWithTrackbar.py file in the Github repository, the four callback functions are defined below:

1 import cv2

2

3 def change_brightness(value):

4 global cap

5 print("Brightness: " + str(value))

6 cap.set(cv2.CAP_PROP_BRIGHTNESS, value)

7

8 def change_contrast(value):

9 print("Contrast: " + str(value))

10 cap.set(cv2.CAP_PROP_CONTRAST, value)

11

12 def change_saturation(value):

13 print("Saturation: " + str(value))

14 cap.set(cv2.CAP_PROP_SATURATION, value)

15

16 def change_hue(value):

17 print("Hue: " + str(value))

18 cap.set(cv2.CAP_PROP_HUE, value)

Then, define the main function to setup the trackbars and display the webcam-

20 def main():

21 global cap

22

# read from webcam

cap = cv2.VideoCapture(0, cv2.CAP_DSHOW)

23

# set width

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 800)

24

# set height

cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

25

# set initial brightness

cap.set(cv2.CAP_PROP_BRIGHTNESS, 100)

26

# set initial contrast

cap.set(cv2.CAP_PROP_CONTRAST, 50)

27

# set initial saturation

cap.set(cv2.CAP_PROP_SATURATION, 90)

28

# set initial hue

cap.set(cv2.CAP_PROP_HUE, 15)

29

30 cv2.namedWindow('Webcam')

31

cv2.createTrackbar('Brightness', 'Webcam', 100,

300, change_brightness)

32

cv2.createTrackbar('Contrast', 'Webcam', 50, 300,

change_contrast)

33

cv2.createTrackbar('Saturation', 'Webcam', 90,

100, change_saturation)

34

cv2.createTrackbar('Hue', 'Webcam', 15, 360,

change_hue)

35

36 success, img = cap.read()

37 while success:

38 cv2.imshow("Webcam", img)

39

40 # Press ESC key to break the loop

41 if cv2.waitKey(10) & 0xFF == 27:

42 break

43 success, img = cap.read()

44

45 cap.release()

46 cv2.destroyWindow("Webcam")

47

48 if __name__ == '__main__':

49 main()

Execute the code, there are four trackbars shown together with the webcam window, for adjusting brightness, contrast, saturation and hue. The below Figure only shows the trackbars’ area, the webcam screen also shows in the window. You will see the camera screen changes accordingly when each slider is moved by the mouse.

Read More

User Interaction in OpenCV- Crop an Image with Mouse

This example will load an image and use the mouse to draw a rectangle, and then crop the image based on the rectangle as shown below.

When the mouse left button is clicked on the image, the first point is captured, normally it is the top-left of the rectangle however it depends on which direction the mouse is moving, we will resolve it later.

Then hold the left button and move the mouse, similar to previous sections a rectangle is drawn while the mouse is moving to indicate how the rectangle looks like, at the same time the coordinates of the first point and the mouse’s current location are shown, as Figure above.

When the left button is released, the second point is captured, and the final rectangle is drawn, at the same time a separate window is shown with cropped image.

The source code is similar to the previous examples of drawing circles and polygons. Now we only focus on the cropping image part of the source code. As mentioned earlier, when the left mouse button is down the first point is captured and recorded, and when it is released the second point is recorded.

To crop an image, an array operation will simply do the job. 

cropped_image = image[y_start:y_end, x_start:x_end]

The image data is in image array, its first dimension is y coordinate and the second is x. y_start:y_end will select only rows between y_start and y_end, similarly x_start:x_end will select only columns between the two.

Depending on which direction the mouse is moving the first point might not always be the top-left, and the second point might not always be the bottom-right. We resolve this in the crop_image() function by comparing the x and y values of the first and second points, the smaller x and y will be the top-left point, the bigger ones will be the bottom right. The code snippets are show as below:

1 def crop_image(img, pt_first, pt_second):

2 # top-left point

x_tl, y_tl = pt_first

3 # bottom-right point

x_br, y_br = pt_second

4 # swap x value if opposite

if x_br < x_tl:

5 x_br, x_tl = x_tl, x_br

6 # swap y value if opposite

if y_br < y_tl:

7 y_br, y_tl = y_tl, y_br

8 cropped_image = img[y_tl:y_br, x_tl:x_br]

9 cv2.imshow("Cropped Image", cropped_image)

Execute the code in CropImageWithMouse.py file, you can crop the image with the mouse operations, looks something like Figure shown above.

Read More

User Interaction in OpenCV- Draw Polygon with Mouse

Similar to drawing circles, this example will draw polygons with the mouse. OpenCV provides cv2.polylines() function to draw polylines or polygons.

cv2.polylines(img, points, is_closed,color, thickness)

points specify the vertexes (the corner points of a polygon) of the polygons/polylines, is_closed=True will draw a closed polygon, if False, draw the polylines. Other parameters are the same as the cv2.circle() in the previous post.

This example works in this way, every time the mouse left key is clicked, a point is added to the polygon, when the mouse is moving a thin line is drawn between the last point and the mouse current point to indicate where the line will be drawn.

When the right key is clicked the final polygon will be drawn with all points. 

 The source codes are very similar to the previous example of drawing circles, we will not explain it line by line here. The only difference is that an array is used to store the vertexes for the polygon when the EVENT_ LBUTTONDOWN (left button down) events happen, the polygon is drawn based on the array of the vertexes.

When the mouse is moving, a thin line is drawn with drawing_color between the last point in the array and the current mouse point, this thin line is changing along with the mouse moving to indicate the line.

When the mouse right click happens, the EVENT_RBUTTONDOWN event is captured, then the drawing is completed, and the final polygon will be drawn with final_color.

Execute the code in DrawPolygonWithMouse.py and you are able to draw polygons with your mouse, the results are something like Figure shown above.

Read More

User Interaction in OpenCV- Draw Circles with Mouse

This example will use mouse to draw circles with cv2.circle() function provided by OpenCV. Here's the basic syntax of the function:

cv2.circle(img, center, radius, color, thickness)

The parameters: 

This example works in this way, when the left key of the mouse is pressed the center of the circle is decided, then hold the left key and move the mouse, the radius of the circle is continuously calculated while the mouse is moving. When the mouse left key is released the radius is finalized and the final circle is drawn on the canvas, the center coordinates and the radius are also shown on the canvas. While the mouse is moving with the left key down, a thin circle is drawn alone with the mouse moving in order to indicate how the circle looks like.

Here are the codes:

1 import cv2

2 import numpy as np

3 import math

4 import common.Draw as dw

5

6 drawing = False

7 final_color = (255, 255, 255)

8 drawing_color = (125, 125, 125)

9

10 def on_mouse(event, x, y, flags, param):

11 global drawing, ctr, radius, img, img_bk

12 if event == cv2.EVENT_LBUTTONDOWN:

13 drawing = True

14 ctr = x, y

15 radius = 0

16 draw_circle(img, ctr, radius, drawing_color)

17 elif event == cv2.EVENT_MOUSEMOVE:

18 if drawing == True:

19 img = img_bk.copy()

20 radius = calc_radius(ctr, (x,y))

21 draw_circle(img, ctr, radius, drawing_color)

22 elif event == cv2.EVENT_LBUTTONUP:

23 drawing = False

24 radius = calc_radius(ctr, (x,y))

25 draw_circle(img, ctr, radius, final_color, 2, True)

26 img_bk = img.copy()

The source codes are a little bit longer, we break it into three parts, first take a look

at the mouse callback function on_mouse(), it captures three events:

EVENT_LBUTTONDOWN, EVENT_MOUSEMOVE and EVENT_RBUTTONDOWN.

The next part of the source codes is to define several functions, calc_radius() is using a math function math.hypot() to calculate the radius based on the center point and current point; draw_circle() is to draw a circle, which includes drawing a point for the center, and drawing texts to display the center coordinates and radius. print_instruction() is to display the instruction texts on the top of the canvas.

28 def calc_radius(center, current_point):

29 cx, cy = current_point

30 tx, ty = center

31 return int(math.hypot(cx - tx, cy - ty))

32

33 def draw_circle(img, center, r, color,

line_scale=1, is_final=False ):

34 txtCenter = "ctr=(%d,%d)" % center

35 txtRadius = "r=%d" % radius

36 if is_final == True:

37 print("Completing circle with %s and %s" % (txtCenter, txtRadius))

38 dw.draw_circle(img, center, 1, color, line_scale) # draw center

39 dw.draw_circle(img, center, r, color, line_scale) # draw circle

40 dw.draw_text(img, txtCenter, (center[0]-60, center[1]+20), 0.5, color)

41 dw.draw_text(img, txtRadius, (center[0]-15, center[1]+35), 0.5, color)

42 43 def print_instruction(img):

44 txtInstruction = "Press and hold left key to draw a circle. ESC to exit."

45 dw.draw_text(img,txtInstruction, (10, 20), 0.5, (255, 255, 255))

46 print(txtInstruction)

The final part of the source code is the main function, which is similar to previous example codes and straightforward, we will not explain it in detail here.

48 def main():

49 global img, img_bk

50 windowName = 'Mouse Drawing Circles'

51 img = np.zeros((500, 640, 3), np.uint8)

52 print_instruction(img)

53 img_bk = img.copy()

54 cv2.namedWindow(windowName)

55 cv2.setMouseCallback(windowName, on_mouse)

56 while (True):

57 cv2.imshow(windowName, img)

58 if cv2.waitKey(20) == 27:

59 break

60 cv2.destroyAllWindows()

61

62 if __name__ == '__main__':

63 main()

Execute the codes in DrawCircleWithMouse.py, you will be able to draw circles on the canvas with mouse operations, the results are something like Figure shown at the beginning of the post.

Read More

User Interaction in OpenCV

OpenCV provides functions that allow users to interact with images and videos in various ways. In previous sections cv2.waitKey() function has been used to accept user input from the keyboard and the ESC key can be detected by checking the key code returned by the function. And cv2.destroy AllWindows() function closes all the windows that were created using the cv2.imshow(). These are very simple user interaction functions. In addition, OpenCV provides several other functions for user interaction but we will look some commonly used ones.

Mouse Operations

OpenCV provides a facility to control and manage a variety of mouse events, and it provides the flexibility for programmers to manipulate user interactions. It can capture the events generated by mouse operations, such as left-button-down, rightbutton-down, mouse-move and double-click etc. The callback functions can be used to capture these mouse events.

Below code snippets will print out the list of all mouse events that OpenCV supports:

1 import cv2

2

3 for event in dir(cv2):

4 if “EVENT” in event:

5 print(event)

It displays a list of mouse events which is outlined in below table:

This example will show how to use the callback function to capture the mouse events. Load and show an image, when a mouse click happens on the image, the x, y coordinates of the pixel and the blue, green and red values will be printed on a separate window.

1 def mouse_event(event, x, y, flags, param):

2 if event == cv2.EVENT_LBUTTONDOWN:

3 blue = img[y, x, 0]

4 green = img[y, x, 1]

5 red = img[y, x, 2]

6 mycolorImage = np.zeros((100, 280, 3), np.uint8)

7 mycolorImage[:] = [blue, green, red]

8 strBGR = "(B,G,R) =

(" + str(blue) + ", "+str(green)+",

" + str(red) + ")"

9 strXY = "(X,Y) = ("+str(x)+", "+str(y)+")"

10 txtFont = cv2.FONT_HERSHEY_COMPLEX

11 txtColor = (255, 255, 255)

12 cv2.putText(mycolorImage,strXY, (10,30), txtFont, .6, txtColor,1)

13 cv2.putText(mycolorImage,strBGR,(10,50), txtFont,.6, txtColor,1)

14 cv2.imshow("color", mycolorImage)

15

16 if __name__ == '__main__':

17 img = cv2.imread("../res/flower003.jpg")

18 cv2.imshow("image", img)

19 cv2.setMouseCallback("image", mouse_event)

20 cv2.waitKey(0)

21 cv2.destroyAllWindows()

Let's understand the code through following explanation-

Execute the code, the image is loaded and displayed, whenever a mouse click happens on the image, the pixel’s coordinates (x, y) and color values (B, G, R) are displayed in a separate window, and the background color is filled with the color of the pixel, as shown in Figure below-

Read More

OpenCV basics continued...Draw an OpenCV-like Icon

Now let’s use our wrapper functions to draw a complicated image, not exactly same but similar to the OpenCV Icon, like Figure shown below.

A light-gray colored empty canvas of size 360 x 320 is created, same as above.   cv2.ellipse() function will be used to draw the three non-closed circles, a start angle and an end angle can be specified for the ellipse.

cv2.putText() function will be used to draw the texts at the bottom. Below are the code snippets, the axes of the ellipse are defined as (50, 50) so it appears as a circle instead of an ellipse. The center of the three circles and start and end angles of each are defined based on the position of the OpenCV-like icon.

1 def draw_opencv_icon(image):

2 axes = (50, 50)

3 center_top_circle = (160, 70)

4 center_lowerleft_circle = ( center_top_circle[0]-80,

5 center_top_circle[1]+120 )

6 center_lowerright_circle = (center_top_circle[0]+80,

7 center_top_circle[1]+120 )

8 angle_top_circle = 90

9 angle_lowerleft_circle = -45

10 angle_lowerright_circle = -90

11 start_angle = 40

12 end_angle = 320

13 draw_ellipse(image,center_top_circle, axes,

14 angle_top_circle, start_angle, end_angle,

15 color=(0, 0, 255),

16 thickness=40)

17 draw_ellipse(image, center_lowerleft_circle, axes,

18 angle_lowerleft_circle, start_angle, end_angle,

19 color=(0, 255, 0),

20 thickness=40)

21 draw_ellipse(image,center_lowerright_circle,axes,

22 angle_lowerright_circle, start_angle, end_angle,

23 color=(255, 0, 0),

24 thickness=40)

25 draw_text(image, "OpenCV", (10,330), color=(0,0,0),

26 font_scale=2.4, thickness=5)

27

28 if __name__ == '__main__':

29 canvas = np.zeros((360,320,3), np.uint8)

30 canvas[:] = 235,235,235

31 draw_opencv_icon(canvas)

32 cv2.imshow("Canvas", canvas)

33 cv2.waitKey(0)

34 cv2.destroyAllWindows()

Execute the code, the result is shown in Figure at the beginning of the post. 

Read More

OpenCV basics continued...Draw Texts

OpenCV provides functions not only for drawing shapes, but also for texts. cv2.putText() function is used for drawing texts.

Similarly, define a wrapper function draw_text():

1 def draw_text(image, text, org,

2 font_face=cv2.FONT_HERSHEY_COMPLEX, 

3 font_scale=1,

4 color=(255,255,255),

5 thickness=1,

6 line_type=cv2.LINE_AA):

7 cv2.putText(image, text, org, font_face,

8 font_scale, color, thickness, line_type )

Then create a canvas, paint it with light gray color, then call draw_ text() function to draw the texts.

10 canvas = np.zeros((380, 480, 3), np.uint8)

11 canvas[:] = 235,235,235

12 #Draw a text

13 draw_text(canvas, "Hello OpenCV", (50, 100),

14 color=(125, 0, 0),

15 font_scale=1.5,

16 thickness=2)

17 cv2.imshow("Hello OpenCV", canvas)

18 cv2.waitKey(0)

19 cv2.destroyAllWindows()

The result is shown as Figure shown below-

This version of OpenCV supports a limited set of fonts, below table shows the supported fonts.

Read More

OpenCV basics continued...Draw Rectangles, Circles, Ellipses and Polylines Similarly draw

Similarly draw other shapes, now begin with defining our wrapper functions for drawing shapes, like the above draw_line() function.

1 def draw_rectangle(image, top_left, bottom_right,

2 color=(255,255,255),

3 thickness=1,

4 line_type=cv2.LINE_AA):

5 cv2.rectangle(image, top_left, bottom_right,

6 color, thickness, line_type)

7

8 def draw_circle(image, center, radius,

9 color=(255,255,255),

10 thickness=1,

11 line_type=cv2.LINE_AA):

12 cv2.circle(image, center, radius, color, thickness, line_type)

13 

14 def draw_ellipse(image, center, axes, angle, start_angle, end_angle,

15 color=(255,255,255),

16 thickness=1,

17 line_type=cv2.LINE_AA):

18 cv2.ellipse(image, center, axes, angle, start_angle, end_angle,

19 color, thickness, line_type)

20

21 def draw_polylines(image, points,

is_closed=True,

22 color=(255,255,255),

23 thickness=1,

24 line_type=cv2.LINE_AA ):

25 cv2.polylines(image, points, is_closed,

26 color, thickness, line_type)

The results look like Figure shown below-

Read More