Gesture Slider Swiper with Arduino

In this post you’re going to learn how to set an Arduino UNO board as a USB HID keyboard and use gestures to swipe slides during a presentation.

How it works

Here’s a figure that describes exactly how everything works together:

If you like Arduino projects and you want to build more, we recommend getting the Arduino Step-by-step Projects – Build 25 Projects course.

Arduino UNO or Mega as a USB HID keyboard

The Arduino can be configured to emulate a keyboard and perform useful tasks. Attaching a pushbutton to an Arduino digital pin and with a button press, you can write a password, copy+paste, up/down volume, up/down screen brightness, etc…

Although boards like the Arduino UNO can’t do this by default if you flash a new bootloader, you can make the Arduino UNO board turn into a USB HID keyboard.

An Arduino Uno/Mega has two microcontrollers: ATmega328 and 16u2. The 16u2 is normally used for USB to Serial communication. We can also use it as standalone AVR Microcontroller with (or without) USB functions as well.

HoodLoader2 gives you the option to reprogram the 16u2 of a normal Arduino Uno/Mega R3 with custom sketches. This means you can use the 16u2 as a normal USB AVR like an Arduino Leonardo board. You have a full compatible USB-HID core.

You can prepare your Arduino UNO or Arduino Mega to act as a keyboard with the HoodLoader2 bootloader by following the instructions in their official Wiki page.

Supported Arduino boards using Arduino IDE 1.6.7 or higher:

• Uno (needs HoodLoader2 bootloader)
• Mega (needs HoodLoader2 bootloader)
• Leonardo
• (Pro) Micro

Parts required

Here’s a complete list of the components you need for this project:

• Arduino UNO, Mega, Leonardo, ProMicro or any other 8u2/16u2/at90usb8/162/32u2/32u4 compatible board
• 2x Ultrasonic sensor – HC-SR04 
• 2x LEDs 
• 2x 220 ohm resistors 
• Breadboard 
• Jumper wires 

You can use the preceding links or go directly to MakerAdvisor.com/tools to find all the parts for your projects at the best price!

Uploading code

To use your Arduino as a keyboard, you need to install the HID Library.

Installing the HID Library

1. Click here to download the HID library. You should have a .zip folder in your Downloads folder
2. Unzip the .zip folder and you should get HID-master folder
3. Rename your folder from HID-master to HID
4. Move the HID folder to your Arduino IDE installation libraries folder
5. Finally, re-open your Arduino IDE

Copy the following code to your Arduino IDE and upload it to your Arduino board:

 /*
  * Author: Emmanuel Odunlade 
  * Complete Project Details https://randomnerdtutorials.com
  */
 
#include <HID.h>// include the library 

int swipe = 0;//slide
boolean left = false;
boolean right = false;
int maxD = 5;//(in cm) maximum distance from ultrasonic at which the obstruction will be considered a gesture
long int lastTouch = -1;
int resetAfter = 1000;//ms
int afterslideDelay = 500;//
int slideleft_Begin = -1;
int slideNone = 0;
int slideright_Begin =1;

//Declaring the connected pins
int lLed = 7;
int rLed = 6;
const int lEcho = 2;
const int lTrig = 3;
const int rEcho = 4;
const int rTrig = 5;

void setup(){
  pinMode(lLed,OUTPUT);
  pinMode(rLed,OUTPUT);
  pinMode(rEcho,INPUT);  
  pinMode(rTrig,OUTPUT);
  pinMode(lEcho,INPUT);
  pinMode(lTrig,OUTPUT);
  
  Serial.begin(9600);
    // Sends a clean report to the host. This is important because
  // the 16u2 of the Uno/Mega is not turned off while programming
  // so you want to start with a clean report to avoid strange bugs after reset.
  pressRawKeyboard(0, 0);
}

//get distance
//echo is input, trigger is output
unsigned long measureD(int input, int output){
  digitalWrite(output, HIGH);
  delayMicroseconds(10); 
  digitalWrite(output, LOW);
  long range= pulseIn(input, HIGH);
  int distance= range / 29 / 2;// to get distance in cm
  return distance;
}

//
boolean act (int input, int output, int led){
  int d = measureD(input,output);
  boolean pinActivated = false;
  if (d < maxD){
    digitalWrite(led,HIGH);
    pinActivated = true;
  }
  else{
    digitalWrite(led,LOW);
    pinActivated = false;
  }
  return pinActivated;
}

void slideNow(char directn){
  if ('R' == directn)
  {
    // press the right rrow key
    //Serial.println("F");
    pressRawKeyboard(0,RAW_KEYBOARD_RIGHT_ARROW); //modifiers + key
    pressRawKeyboard(0, 0); // release! Important
  }
  if ('L' == directn)
  {
    //press the left arrow key
    //Serial.println("B"); for debug
    pressRawKeyboard(0,RAW_KEYBOARD_LEFT_ARROW); // //modifiers + key
    pressRawKeyboard(0, 0); // release! Important
  }
  delay(afterslideDelay);
  swipe = slideNone;
}

void pressRawKeyboard(uint8_t modifiers, uint8_t key){
  uint8_t keys[8] = { 
    modifiers, 0, key, 0, 0, 0, 0, 0    }; //modifiers, reserved, key[0]
  HID_SendReport(HID_REPORTID_KeyboardReport, keys, sizeof(keys));
}


void loop(){
    left = act(lEcho,lTrig,lLed);
    right = act(rEcho,rTrig,rLed);
    if (left || right){
        lastTouch = millis();
    }
    if (millis() - lastTouch > resetAfter){
      swipe = 0;
      //serial.println(@reset slide& timer);
    }
    if (swipe >= slideNone){
      if ((left)&&(!right)){
        swipe = slideright_Begin;
      }
      if ((right)&&(swipe == slideright_Begin)){
        slideNow('R');
      }
    }
    if (swipe <= slideNone ){
      if ((right)&&(!left)){
        swipe = slideleft_Begin;
      }
      if ((left) && (swipe == slideleft_Begin)){
        slideNow('L');
      }
    }
    delay(50);
 }
 

View raw code

Schematics

Wire your circuit accordingly to the schematic below:

Now, if you move a hand in front of the left ultrasonic sensor, your Arduino sends a Left keyboard command resulting in moving the presentation to the previous slide.

Wrapping Up

I have always been fascinated by natural user interfaces, the idea of things like the sixth sense (which I hope to share a tutorial on someday) and generally all things that makes telekinesis look like a possibility.

Thus I believe there are several updates that could be made to this project. Instead of using ultrasonic sensors and controlling sides what if we attached an Electroencephalogram (EEG) headset and then do more complex stuff like draw shapes and achieve all this within the simplicity frame the Arduino provides.

So the possibilities are endless. Hack it up!