Automating Mouse Movements

Getting the current position of the mouse cursor:

Firstly, we would where currently my mouse cursor is at, for that we can use the .position() function. The function again returns a point object with x and y values that gets the current position of the mouse.

Syntax: pyautogui.position()

Parameters: This function does not take any extra parameters

Return Type: It returns us the position of the mouse cursor in a Point object

Below is the implementation:

Output:

Point(x=1710, y=81)

Moving the cursor and clicking on specific spots:

Now we would try to move the mouse cursor and click at specific spots and perform opening an application and closing it. For moving the mouse pointer we can use .moveto() and specify x,y values along with a duration in which it will perform the operation, and we would use the  .click() function to click on the spot where our mouse pointer is located right now. The code basically moves the mouse cursor to (519,1060) (x,y) values and then simulate a click using the .click()  where the cursor is situated right now, then we again move to the position (1717,352) using the moveTo() and simulate a click again. 

Syntax: pyautogui.moveTo() and pyautogui.click()

Parameters: This moveTo function has two required and one optional parameter, the first two values of x and y are required values while the duration is an extra parameter that kind of animates the movement of the mouse over the no of seconds assigned to the duration parameter. The click method in the example doesn’t take any parameter but an optional pair of parameters can be used to click a particular position on the keyboard.

Return Type: The functions don’t return anything but performs the jobs of the cursor to a specific point and then clicking there programmatically.  

Below is the implementation:

Output:

Now we would explore two more methods namely .moveRel() which helps us to move relative to the position we are at right now and finally we would see how we can simulate a right-click using pyAutoGUI. We start with importing the package and stimulate the cursor to move  498 px & down 998px from its current position. Then we use the click() method to simulate a left-click. Then we move to a specific location using the .moveTo() method. Now we again click on the present position of the cursor but this time we instruct to simulate a right-click instead of left by passing the button=” right” parameter (default is button =” left”). Then we gain to move to a specified location and left click there.

Syntax: pyautogui.moveRel()

Parameters: This moveRel function has also two required and one optional parameter, the first two values of x and y are required values while the duration is an extra parameter that kind of animates the movement of the mouse over the no of seconds assigned to the duration parameter. Also, we used an extra parameter for the pyautogui.click() function, we used button=”right” which simulates a right-click instead of the default left-click.

 Return Type: The functions don’t return anything but perform the jobs of moving the cursor left 498 px & down 998px from it’s current position and then simulate a right-click programmatically. 

Below is the implementation:

Output:

Dragging the cursor to a specific screen position:

Now we would see how can we drag windows using pyAutoGUI. We can use the .dragto() and .dragrel() which are exactly the way the .moveto() and .movrel() works except that in this case, they hold the left click while moving the cursor. In this program we simply, import the modules, then we move to a specified using the .moveTo() function. Then we left-click at the current position of the cursor. Now we again move the cursor to a specified location. Then we use the .dragTo() function to drag (left-click and hold) the to a specific location. Finally, we use the dragRel() function that drags the cursor relative to its current position to 50px right and 50 px down.

Syntax: pyautogui.dragTo() and pyautogui.dragRel()

Parameters: Both the functions has two required and one optional parameter, the first two values of x and y are required values while the duration is an extra parameter that kind of animates the movement of the mouse over the no of seconds assigned to the duration parameter.

Return Type: The functions don’t return anything but perform the jobs of left-click and holding and moves the cursor to (500,500) position and drags the cursor relative to it’s position to 5opx right and 50 px down programmatically. 

Below is the implementation:

Output:

Note: Duration parameter in .moveTo(), .moveRel(), .dragTo() and .dragRel() functions are optional, but it is provided to get a animation effect, without the property functions will execute instantly, and it would be tough to understand. Also we optionally pass x and y values in .click() function which can be used to click at a different location that the location the cursor is currently at.

In this article, we will explore how we can do GUI automation using Python. There are many modules that can do these things, but in this article, we will use a module named PyAutoGUI to perform GUI and desktop automation using python. 

We would explore two sections –

• How to automatically use the mouse pointer to perform certain tasks like moving the cursor, clicking on a certain point on the screen, etc
• Also, we would explore how we can automate the keyboard keystrokes.