Project Statement

   The aim of this project was to design and implement an embedded system which enables controlling of a robotic arm through the user's body gestures captured by a Kinect for Windows sensor through an Intel's Atom PC.

 

Control of a Robotic Manipulator using Visual Data from a Kinect for Windows Sensor
Equipment

   The list of the equipment employed is as follows: 

 

Description
Coding Language

   At first we started off by coding the Kinect program in C# and the manipulator controller program in C++, however, we then decided to have both codes in C# to achieve a simpler and possible faster and more efficient interface between the two parts.  

Manipulation 

   The Lynx6 robotic platform was chosen to work on. Lynx6 is a 6 degrees of freedom (DoF) robotic arm (also manipulator) designed and produced by Lynxmotion company. The first step in using any robotic manipulator is to calibrate it. Through the calibration of the robotic arm the joint angle limits are measured, the link length are precisely measured, the reachable workspace of the end-effector is worked out, and so on. One of the most important aspects of the calibration is to determine what command is required to take each servo to a desired angle. To achieve this, three sample angles for each joint servo are recorded along with their corresponding servo commands that will take the joints to those angles. These data are then used in the formation of a formula through interpolation that will map any angles within the working range of a servo to their corresponding servo commands. 

   For the purpose of this project where a robotic manipulator with different degrees of freedom for its every joint than the human arm's is to mimic a human arm's gestures, there was need for great deal of modifications and considerations on the raw angular data obtained from the Kinect sensor prior to communicating it to the robotic arm.

   To communicate servo commands to the joints, a serial communication between the robotic arm and the Atom processor is utilized.      

 

http://1.bp.blogspot.com/-YgeTWThniWw/UYOMHUcjbEI/AAAAAAAAAFo/uycFm5ry5k4/s400/lynx5.jpg

Lynx6- a 6 DoF Robotic Manipulator by Lynxmotion Co.

     

Perception

   The Microsoft's Kinect for Windows was chosen to be our sensory platform for capturing human body gestures (most particularly the right or left arm movements). To mimic human body gestures or/and receive commands through user's body movements, a series of information needed be collected from the user. The Microsoft's SDK support for Kinect provides a library of API including a particular API support for skeletal detection. The Skeleton class provided in the Microsoft.Kinect's library and its properties Joints and Position were used to obtain the spatial positions of some of the 20 detectable (supported by the SDK) human body's joints in the stand-up mode. A method was implemented in our code which takes as input arguments the positions of three joints and outputs the joint angle between them. The collected angles are then passed to the robotic manipulation section of the program for further processes to produce appropriate commands to be communicated to the robotic arm's servos at its joints. Examples of the additional processes that need be conducted on the angular data obtained from the Kinect sensor before communicating it over the serial port to the robotic arm are applying the constraints that were worked out through the full calibration. There are many constraints for an arm, be it a robotic arm or a human arm. Some of these constraints are limits of the servos, limited degrees of freedom at a joint, limited end-effector orientation, and constrained reachable workspace.       

http://1.bp.blogspot.com/-UOzMtKVec6o/UYOMjq3_7VI/AAAAAAAAAFw/6V5d3qK2BsQ/s320/KinectForWindows.jpg

Kinect for Windows- a RGB+Depth Image sensor by Microsoft Corp.


Gallery

Demo
 

http://www.youtube.com/watch?v=e4kjBHm4k-g&list=UUzGAfC5Kr-kZSWbPgd1lt3Q&index=3

http://www.youtube.com/watch?feature=player_embedded&v=DoqBsMfx6bQ





Code

 

Brief Program Outline

Note:As this is in early development, the code is not as streamlined as it could be, the code will be refined in the coming weeks.


Future Work
   To implement the inverse kinematics in our design so that by passing the spatial coordinates, the end-effector of the robot will automatically move to that point relative to the manipulator's relative frame. 
   To make the inverse kinematics more efficient, a trajectory-planning algorithm, such as the rapidly-exploring random trees (RRTs), can also be implemented to smooth the transitions.
References

  1. Microsoft Kinect SDK Library: http://msdn.microsoft.com/en-us/library/hh855347.aspx 
  2. "A Mathematical Introduction to Robotic Manipulation": http://www.kramirez.net/Robotica/Material/Libros/  



   

Arash Tavakoli

Send feedback