ROBONAUT Activity Report
July
2003
This month the Robonaut team was busy conducting experiments to advance the autonomous capabilities of Robonaut as well as enhancing the teleoperator interface with the addition of dual arm Force Reflective Hand Controllers (FRHC). Summer researchers from Vanderbilt University, University of Massachusetts, Trinity University and Rice University played an integral role this month in the autonomy and force feedback activities. For one of the autonomy experiments to be conducted, two FRHC's had to be brought on line first. The FRHC's served two purposes this month. The first was to provide forces sensed in the Robot's environment back to the teleoperator. The second purpose was to aid in the learning capabilities of Robonaut. By monitoring the forces applied by the teleoperator (as seen in Robonaut's force/torque sensors), Robonaut's control system can then "learn" what forces are appropriate for grasping, manipulating, lifting and translating various tools and objects in it's work environment. External to the Dexterous Robotics Lab, a keynote presentation was given at the Space Mission Challenges for Information Technology Conference in Pasadena, California as well as a panel presentation at the SIGGRAPH conference held in San Diego, California.
Summer Researchers
AR&SD has hosted a number of university researchers during this summer, each using the Robonaut system as a testbed to apply and test new ideas in robot control. These new technologies span a spectrum of control modes from haptic forms of telepresence to high levels of autonomy and learning. This work has leveraged on Robonaut's ability to be operated both by teleoperation and autonomous control. Coupling Robonaut’s unique upper body with these new control modes will provide the dexterous manipulation needed to assist astronauts in future EVA tasks. The following entries briefly describe each of the university research activities underway.
University: Rice University, Houston Tx
Support: JSC Summer Faculty Program
Researchers: Dr. Marcie O’Malley
John Glassmire
Researchers from Rice have now been working with Robonaut for two years. This summer, FRHC's (fig. 1) were assembled and tested with the Robonaut system to evaluate new technology for telepresence control. A single arm and a dual arm task were both attempted to demonstrate the benefits of force feedback. The first task repeated a cooperative assembly experiment, where Robonaut assisted a human teammate assembling an EVA handrail (fig's. 2 & 3). This testing also took advantage of a Vanderbilt University learning algorithm that collected data when a teleoperator controlled Unit A in response to a teammate’s commands to move the end of the long strut in various directions (fig. 4). The learning algorithm then substituted for the teleoperator and controlled the robot in response to the teammate’s gestures and voice commands to move the strut. The algorithm did an excellent job producing the trajectories normally provided by a teleoperator. The results of this experiment were then compared with previous runs, where the teleoperator had only visual (graphical) displays of force data, but was essentially “numb” due to the absence of force feedback. The results of this work demonstrated that force feedback increased operator task performance, enhanced the operator's involvement in the task, decreased overall forces seen at Robonaut's joints, and decreased overall task times.
The second task used two FRHC's to allow Robonaut to pick up a large, but soft object, using two arms (fig's 5 & 6). Without force feedback, this task has proven to be difficult, either crushing the object or dropping it. This task was unusual in that the robot needed to be able to apply light forces. The FRHC's were found to be an improvement over existing solutions. Significant reductions in peak forces were found, as well as a reduction in the cumulative forces exerted over the course of the task time. This experiment led in to additional learning algorithm development and is described in the Vanderbilt University entry below. The FRHC work was led by Dr. Marcie O'Malley from Rice University, with her graduate student John Glassmire. The Learning work was led by Dr. Alan Peters from Vanderbilt University.
Fig. 1, Dual Force Reflective Handcontrollers (FRHC's)
Fig. 2, Two Agent EVA handrail insertion task
Fig. 3, Teleoperator using single FRHC during handrail insertion task
Fig. 4, Task leader using gestures as part of a "training" process
University: Vanderbilt University, Nashville
TN
Support: DARPA MARS 2020 Humanoids Grant
Researchers: Dr. Alan Peters
Christine Campbell
Kim Hambachen
Researchers from the Vanderbilt Intelligent Robotics Lab have been working with Robonaut for the past 3 years. They have continued to develop and test new autonomy products for Robonaut, including a memory structure (the Sensory EgoSphere) for tracking objects in the robot’s workspace, and now new learning approaches for teaching the robot manual dexterity skills. The general approach is to let a teleoperator “step inside” Robonaut with VR gear used for telepresence control. Past work in telepresence has made these human operators feel like they are the robot. From this perspective, Vanderbilt’s software lets Robonaut watch itself being controlled by the human, and learns from this experience. Last summer, the robot was taught how to reach out and grasp a tool. This summer, the work was extended to assisting astronauts handling large objects and grasping objects with two hands. The latest work lets a teleoperator teach the robot what to do in response to voice commands and arm gestures provided by an adjacent human teammate (fig. 7). Once taught, the robot can autonomously lift objects, and reposition them in response to the human teammate, without each such case having to be custom programmed (fig. 8 & 9). The prior work on memory structures, and the recent work on learning are led by Dr. Alan Peters (fig. 5) from Vanderbilt, with his two graduate students Kim Hambachen and Christina Campbell.
Fig. 5, Dr. Peters "training" the control system for dual arm op's
Fig. 6, Teleoperator using dual FRHC's during ball grasping task
Fig. 7, Task leader using gestures to command autonomous dual arm motions while grasping a ball
Fig. 8, Dropped ball during initial training
Fig. 9, Successful ball manipulation after training was complete
University: UMass, Amherst Ma
Support: DARPA MARS 2020 Humanoids Grant
Researchers: Dr. Roderick Grupen
Dr. Andrew Fagg
Dr. Michael Rosenstein
Robert Platt
Researchers from the University of Massachusetts, Amherst have been working with the Robonaut team for the past two years. Their research involves automating the grasps of dexterous hands using tactile and force data. Last summer the UMass researchers implemented a reflexive grasp system that used Robonaut’s tactile glove (fig. 10) to detect an object and grasp it like the reflex seen in human infants. This summer, the work is in the process of being extended to make grasps of specific objects (such as the pistol grip of the EVA power torque tool) with the added complexity of articulating the tool’s trigger. Another technology under development this month and being tested next month will allow Robonaut to autonomously place that tool on a bolt, torque it tight, and then remove the tool from the bolt head (fig. 11). Combining this skill with the ability to pick up the tool will make Robonaut one of the first, if not the first, robots to autonomously use tools designed for humans. This work is being led by Dr. Roderick Grupen, with Dr. Andrew Fagg, Dr. Michael Rosenstein, and Robert Platt.
Fig. 10, Tactile glove used with smart algorithms to automate grasping
Fig. 11, Teleoperator Manipulating Torque-Tool to gather tactile data for autonomy routines.
University: Trinity University, San Antonio
Tx
Support: JSC Summer Faculty Program
Researchers: Dr. Kevin Nickels
Trinity University researchers have been working with the Robonaut Team for the last two years. One of the key strategies in deploying Robonaut class robots is that the machine should adapt to its environment, without requiring any special accommodations, such as handles, targets or fixtures to allow them to be grasped. While this allows the robot to be inserted into different environments more easily, the design challenges are significant. One of the many challenges is that objects such as EVA tools will never be grasped exactly the same way by the human hand or the Robonaut hand. Humans adapt to each new grasp, picking up a tool, and quickly understanding the grasp with a combination of visual and tactile information. The Trinity work has been to calibrate the eye-hand coordination system of Robonaut to look at an object in its hand and make this adaptation possible, and automatic. The variation in camera, neck, arm and hand mechanical parameters, as well as the uncertainty in each grasp, will be accommodated with a new algorithm that uses the Robonaut vision system to disambiguate the grasp. With large tools, the tip of the object’s uncertainty had been on the order of 10cm. With the new technology developed this summer, that uncertainty has been reduced to the order of 1cm, allowing visual servoing to then refine the motion by guiding tools such as drills onto bolts that can be seen. The work is led by Dr. Kevin Nickels at Trinity University.
Demonstrations
A Robonaut demonstration was given to a group from JSC's Public Affairs Office, students from the Clear Creek Independent School District, and Co-op students conducting work tours this summer in the Robotic Systems Technology Branch.
Colonel Vause (fig. 12), from the US Army Aero Medical Center in San Antonio, Texas was given a demonstration of Robonaut's dexterous capabilities.
Fig. 12, Colonel Vausse shaking hands with Unit A
Papers
A well-received keynote presentation on Robonaut was given at the Space Mission Challenges for Information Technology Conference in Pasadena, California. The paper, "Robonaut's Flexible Information Technology Infrastructure" was included in the conference proceedings.
Robonaut project lead, Dr. Robert Ambrose, attended the SIGGRAPH conference in San Diego, California and delivered a panel presentation on the state of the art in humanoid robotics.