Optical Vicinity Sensor

General

In their usual field of operation service robots must be able to evade unpredictable obstacles. Thus it is necessary to equip them with sensors that allow timely obstacle recognition. Further on knowledge of the absolute position of the robot or its relative position with respect to an object is often required. As odometric methods are slick-prone, special sensors are needed for this task as well. Both of these topics can be adressed by optical vicinity sensing. But it has to be considered, that because of the high amount of computing power needed for this purpose, vision systems often limit the autonomy level of the robot.

This is why at the Institute of Automatic Control an optical sensor system, consisting of stereo camera and processing hard- and software, is being developed. In order to maintain the autonomy of the robot, the focus lies on keeping the requirements for power supply and computational power within the bounds that can possibly be provided on autonomous service robots.

From the gathered image information the structure of the surroundings can be derived. In the next step it is possible to draw conclusions concerning obstacles and possible foot landing areas. Apart from this, it is possible to compensate for drift effects in an inertial measurement unit, by regularly providing reference measurements from the vision system.

The camera system is intended for use on the bipedal robot LISA but in most aspects it is independent of the specific robot. It will also be possible to adapt the system to a robot with a different number of legs or even wheels.

Hardware description

The stereo camera system consists of two Feith CANCams that are attached to an actuated head. This head is currently under construction. It provides two DOF for yaw and pitch movements (looking left-right and up-down). An additional DOF is reserved for changing the base width, that is the distance of the two cameras. Usually the bigger the base width the better is the depth resolution. There are, however, limits to this, when an object comes to close to be seen by both cameras at the same time. In this case, stereo triangulation does no longer work and the base width has to be reduced. The possibility to adapt the camera distance may also be used for telepresence operation, when a human operator is provided with the stereo images. For this purpose it is supposed to be preferable that the camera base width equals approximately the eye distance of the operator. This additional DOF will not be actuated at the start in order to reduce the complexity.

The camera themselves a commercially available monochrome CMOS cameras with a resolution of 1280x1024 pixels. Inside the camera housing reside a Freescale Coldfire MCF5272 microprocessor and a Xilinx Spartan-IIE FPGA. They enable the camera to perform some lowlevel image processing onboard before transfering the data to the main image processing unit on the robot. The transfer takes place via 100Mbit Ethernet.

Main topics

The main topics that are or will be touched in this project and its successors are:

Construction and control of the camera head
Optical image stabilization for longer exposure times
2D calibration of the single cameras
3D calibration and selfcalibration of the stereo camera system
Lowlevel image processing in VHDL
3D model reconstruction from stereo images
Path planning based on vicinity information and mission target
Self localization by use of natural beacons and self generated maps
Sensor fusion for position and motion measurement

Publications

A complete and searchable list of publications with abstracts and citations in bibtex format can be found here.

Gerecke, M., Gerth, W.: Aufbau eines Stereokamerasystems zum Betrieb unter RTOS-UH. In: P. Holleczek, B. Vogel-Heuser (Hrsg.): Echtzeitaspekte bei der Koordinierung Autonomer Systeme. PEARL 2005, Springer Verlag Berlin Heidelberg, 2005, S. 45-54.

The following publications relate to a preceding cooperation project with the TU Munich.

Gerecke, M., Albert, A., Gerth, W.: Vision guided biped walking - trajectories and communication. 5th International Conference on Climbing and Walking Robots, CLAWAR 2002, S. 155-162.
Lorch, O., Albert, A., Denk, J., Gerecke, M., Cupec, R., Seara, J. F., Gerth, W., Schmitdt, G.: Experiments in Vision-Guided Biped Walking. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems EPFL, Lausanne, Switzerland, 2002, S. 2484-2490.


	Optical Vicinity Sensor General In their usual field of operation service robots must be able to evade unpredictable obstacles. Thus it is necessary to equip them with sensors that allow timely obstacle recognition. Further on knowledge of the absolute position of the robot or its relative position with respect to an object is often required. As odometric methods are slick-prone, special sensors are needed for this task as well. Both of these topics can be adressed by optical vicinity sensing. But it has to be considered, that because of the high amount of computing power needed for this purpose, vision systems often limit the autonomy level of the robot. This is why at the Institute of Automatic Control an optical sensor system, consisting of stereo camera and processing hard- and software, is being developed. In order to maintain the autonomy of the robot, the focus lies on keeping the requirements for power supply and computational power within the bounds that can possibly be provided on autonomous service robots. From the gathered image information the structure of the surroundings can be derived. In the next step it is possible to draw conclusions concerning obstacles and possible foot landing areas. Apart from this, it is possible to compensate for drift effects in an inertial measurement unit, by regularly providing reference measurements from the vision system. The camera system is intended for use on the bipedal robot LISA but in most aspects it is independent of the specific robot. It will also be possible to adapt the system to a robot with a different number of legs or even wheels. Hardware description The stereo camera system consists of two Feith CANCams that are attached to an actuated head. This head is currently under construction. It provides two DOF for yaw and pitch movements (looking left-right and up-down). An additional DOF is reserved for changing the base width, that is the distance of the two cameras. Usually the bigger the base width the better is the depth resolution. There are, however, limits to this, when an object comes to close to be seen by both cameras at the same time. In this case, stereo triangulation does no longer work and the base width has to be reduced. The possibility to adapt the camera distance may also be used for telepresence operation, when a human operator is provided with the stereo images. For this purpose it is supposed to be preferable that the camera base width equals approximately the eye distance of the operator. This additional DOF will not be actuated at the start in order to reduce the complexity. The camera themselves a commercially available monochrome CMOS cameras with a resolution of 1280x1024 pixels. Inside the camera housing reside a Freescale Coldfire MCF5272 microprocessor and a Xilinx Spartan-IIE FPGA. They enable the camera to perform some lowlevel image processing onboard before transfering the data to the main image processing unit on the robot. The transfer takes place via 100Mbit Ethernet. Main topics The main topics that are or will be touched in this project and its successors are: Construction and control of the camera head Optical image stabilization for longer exposure times 2D calibration of the single cameras 3D calibration and selfcalibration of the stereo camera system Lowlevel image processing in VHDL 3D model reconstruction from stereo images Path planning based on vicinity information and mission target Self localization by use of natural beacons and self generated maps Sensor fusion for position and motion measurement Publications A complete and searchable list of publications with abstracts and citations in bibtex format can be found here. Gerecke, M., Gerth, W.: Aufbau eines Stereokamerasystems zum Betrieb unter RTOS-UH. In: P. Holleczek, B. Vogel-Heuser (Hrsg.): Echtzeitaspekte bei der Koordinierung Autonomer Systeme. PEARL 2005, Springer Verlag Berlin Heidelberg, 2005, S. 45-54. The following publications relate to a preceding cooperation project with the TU Munich. Gerecke, M., Albert, A., Gerth, W.: Vision guided biped walking - trajectories and communication. 5th International Conference on Climbing and Walking Robots, CLAWAR 2002, S. 155-162. Lorch, O., Albert, A., Denk, J., Gerecke, M., Cupec, R., Seara, J. F., Gerth, W., Schmitdt, G.: Experiments in Vision-Guided Biped Walking. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems EPFL, Lausanne, Switzerland, 2002, S. 2484-2490.

	Institute of Automatic Control - URL http://www.irt.uni-hannover.de/forschung/asr/cam_en.html Responsible: Institute of Automatic Control, Biped Team, last modification 01.09.2009 © Leibniz Universität Hannover 1998 - 2009