Abstract

In recent years, people got more interested in the research of the evolution of human-computer interaction (HCI). Right from the transition of a text-based command line interface to a graphical user interface (GUI) that uses a mouse, many changes have been observed. The ability to manipulate the mouse with hand movements is one of the areas of interest here. At present, due to the cluttered backgrounds and poor lighting, this is a complicated issue. The application demands extremely specific detection and recognition. In this article, a computer vision-based method has been shown for controlling a computer mouse with hand gestures in front of a linked camera. The user may move the mouse and use the left as well as right click functions using this systematic program.

INTRODUCTION

In today’s world, computers have become a day-to-day necessity for humans even for the smallest tasks. To make our work easier, it needs to interact with computer ample number of times a day, to make our work easier. This led to the emergence of a hot topic for research, the Human Computer Interaction. A computer’s mouse is extremely helpful tool that makes it much easier for humans to interact with the machine. But the problem occurs when a person with a physical disability uses the mouse and as a result, while using a mouse can be challenging in situations, it may be impossible for others depending on the severity of the disability. To solve this problem, a system that uses a gesture recognition technology is developed to provide mouse control (GRS). The inconvenience of using a mouse while travelling is another issue that will be addressed while developing GRS, thus a better mechanism must be put in place to manage it. Humans rely heavily on their senses for their communication. Similar to this, a computer’s Graphical User Interface (GUI) heavily relies on the mouse.

By blending both of these approaches in the field of digital signal processing, a more effective interactive system for human-computer interaction is formed[8][9][10]. The quantity of information included in an image signal can be used to control a variety of computer operations. Hand gestures are therefore vital in this subject. In this research, a method  for  controlling  the  mouse  with  hand  gestures  is offered in front of a system camera utilizing computer vision. Python libraries are employed, which are excellent tools for working with images and modifying them as well as automating computer programs like OpenCV and patagium. The user needs only the laptop’s web camera to use this; no colored markers are required. For hand detection, this method does not require any dots or bands on the fingers or wrists. The user intends to scroll the mouse in the direction of the fingers’ movement when three fingers are raised and spaced apart. If the fingers are pressed together, the user can move his hand without moving the mouse if he/she chooses. The user must lift 4 fingers to click left and 5 fingers to click right, respectively. In this article, a thorough study has been done on literature, a detailed description is given in dataset portion, the mathematical formulation of model has been done, along with the methodology and experimentation results has been discussed in this paper. Last but not the least the paper is concluded in an effective manner.