Table of Contents

Introduction

Part 1

Chapter 1. Calibration of Vision Sensors

1.1. Introduction

1.2. General formulation of the problem of calibration

1.3. Linear approach

1.4. Non-linear photogrammetric approach

1.5. Results of experimentation

1.6. Bibliography

Chapter 2. Self-Calibration of Video Sensors

2.1. Introduction

2.2. Reminder and notation

2.3. Huang-Faugeras constraints and Trivedi’s equations

2.4. Kruppa equations

2.5. Implementation

2.6. Experimental results

2.7. Conclusion

2.8. Acknowledgement

2.9. Bibliography

Chapter 3. Specific Displacements for Self-calibration

3.1. Introduction: interest to resort to specific movements

3.2. Modeling: parametrization of specific models

3.3. Self-calibration of a camera

3.4. Perception of depth

3.5. Estimating a specific model on real data

3.6. Conclusion

3.7. Bibliography

Part 2

Chapter 4. Localization Tools

4.1. Introduction

4.2. Geometric modeling of a video camera

4.3. Localization of a voluminous object by monocular vision

4.4. Localization of a voluminous object by multi-ocular vision

4.5. Localization of an articulated object

4.6. Hand-eye calibration

4.7. Initialization methods

4.8. Analytical calculations of localization errors

4.9. Conclusion

4.10. Bibliography

Part 3

Chapter 5. Reconstruction of 3D Scenes from Multiple Views

5.1. Introduction

5.2. Geometry relating to the acquisition of multiple images

5.3. Matching

5.4. 3D reconstruction

5.5. 3D modeling

5.6. Examples of applications

5.7. Conclusion

5.8. Bibliography

Chapter 6. 3D Reconstruction by Active Dynamic Vision

6.1. Introduction: active vision

6.2. Reconstruction of 3D primitives

6.3. Reconstruction of a complete scene

6.4. Results

6.5. Conclusion

6.6. Appendix: calculation of the interaction matrix

6.7. Bibliography

Part 4

Chapter 7. Shape Recognition in Images

7.1. Introduction

7.2. State of the art

7.3. Principle of local quasi-invariants

7.4. Photometric approach

7.5. Geometric approach

7.6. Indexing of images

7.7. Conclusion

7.8. Bibliography

List of Authors

Index

First published in France in 2003 by Hermes Science/Lavoisier entitled Perception visuelle par imagerie video © LAVOISIER, 2003

First published in Great Britain and the United States in 2009 by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd
27-37 St George’s Road
London SW19 4EU
UK
www.iste.co.uk
John Wiley & Sons, Inc.
111 River Street
Hoboken, NJ 07030
USA
www.wiley.com

© ISTE Ltd, 2009

The rights of Michel Dhome to be identified as the author of this work have been asserted by him in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Cataloging-in-Publication Data

[Perception visuelle par imagerie video. English] Visual perception through video imagery / edited by Michel Dhome.

p. cm.

Includes index.

ISBN 978-1-84821-016-5

1. Computer vision. 2. Visual perception. 3. Vision. I. Dhome, Michel.

TA1634.P4513 2007

006.3'7--dc22

2007028789

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN: 978-1-84821-016-5

Introduction

Artificial vision with a main objective of automatic perception and interpretation of the universe observed by a system containing one or several cameras is a relatively new field of investigation. It leads to a surprisingly large range of problems, and most of these are not currently resolved in a reliable way. Although a general theory is not close to being reached, significant progress has been made recently, theoretically as well as methodologically.

In the visible world, images of luminance are the result of two physical processes: the first one is linked to reflectance properties of the surface of observed objects, while the second one corresponds to the projection of these same objects on the light sensitive plate of the sensor used. From a mathematical standpoint, in order to interpret the observed scene, we must solve an inverse problem, i.e. infer the surface geometry (3D) of objects present, from the purely 2D content of the image or from logged images.

This reputedly complex problem in the context of computer vision is solved by man with surprising ease. However, the human vision system operation is clearly not founded on a single concept. Examining the implemented processes during short or long distance vision is sufficient proof. In the first case, the existing disparity between left and right retinal images makes it possible for man to obtain indepth information by triangulation (stereoscopy) relating to its close environment which is vital in particular to manually capture objects. In the second case, when looking at long distance, or even more so when contemplating a picture, stereoscopy is obviously no help in interpreting the observed scene. Even under these conditions, however (total lack of direct 3D information), man is able to estimate the form and spatial position of objects he observes in the vast majority of cases. This requires mental processes, from 2D information extracted from a luminance image, able to infer 3D information. These are based on the unconscious use of prior knowledge relating to the principle of retinal image composition and the form of 3D objects surrounding us. The surprising capabilities of the human vision system are because this knowledge is continuously enhanced from early childhood.

In the last few decades, researchers in the artificial vision community have attempted to develop perception systems that would work from data emanating from video cameras. This book presents a few tools emerging from recent advances in the field.

In Part 1, the reader will find three chapters dedicated to calibration or self-calibration of video sensors. Chapter 1 presents a finite estimation approach of the intrinsic parameters of a video camera, greatly inspired by the world of photogrammetry. It is based on the interpretation of images from a calibration test chart which is not generally known with great precision. Chapter 2 addresses the complex self-calibration problem from a series of matching points between different images from a single scene. The recommended method is based on an elegant and simple decomposition of Kruppa equations. Chapter 3 explores the self-calibration problem of cameras with specific movements making the implementation of simplified development of the main matrix possible.

Part 2 mainly involves the estimation of the relative object/sensor position by introducing prior knowledge (CAD object model). The reader will discover how to treat the localization problem of a rigid object observed by a monocular system. The formalism presented is then extended to understand cases as different as multi-ocular localization and hand-eye calibration, and research on the posture of articulated objects such as robotic arms.

Part 3 addresses volume information inference in two chapters. Chapter 5 discusses the reconstruction problem of a fixed scene observed by a multi-ocular system. The notions homo-log points, epipolar geometry, fundamental matrix, essential matrix and trifocal tensor are first introduced as well as different approaches for obtaining these entities. The problem of dense matching between image pairs is addressed before a few reconstruction examples are shown. Chapter 6 discusses the notion of active dynamic vision. It is the control of the path of a camera embedded in a robotic arm in order to reconstruct the surrounding scene. An underlying problem involves the definition of optimal movements necessary for the reconstruction of different primitives (points, straight lines, cylinders). Finally, perception strategies are proposed in order to ensure a complete reconstruction taking inter-object blanking into consideration.

The recognition of forms in images is the heart of Part 4. Chapter 7 is dedicated to proposing tools for the identification in a database of the images containing visual elements identical to those contained in a request image; the differences of acquisition between images which could involve the point of view, conditions of illumination and global composition of the scene observed. The methods presented are based on a common principle: the use of quasi-invariants associated with local descriptors. These are tested against large image databases.

Part 1

Chapter 1

Calibration of Vision Sensors

1.1. Introduction

Calibration of a vision system consists of determining a mathematical relation existing between the three-dimensional (3D) coordinates of the reference markers of a scene and the two-dimensional (2D) coordinates of these same reference markers projected and detected in an image.

Determining this relation is an uphill task in vision, particularly for reconstruction, where it is necessary to infer 3D information from data extracted from the 2D image. In reality, the field of application is broader and calibration proves to be essential since it is necessary to establish a relation between an image and the 3D world: recognition and localization of objects, dimensional control of parts, and reconstruction of the environment for the navigation of a robot.

A complete analysis of the calibration of a vision system must take into account all the photometric, optic and electronic phenomena present in the image acquisition chain.

In general, a system that enables the calibration of a camera is made up of:

– a calibration test card (grid or standard object), generally consisting of reference markers from which 3D coordinates are accurately inferred in its local reference coordinate system;

– an image acquisition system for the digitization and storage of test card images;

– an algorithm of correlating 2D reference marker detected in the images with their counterparts of the test card;

– a calculation algorithm of transformation matrix perspective of the camera, relating the calibration test card mark with that of the image.

Generally, the problem of calibrating CCD cameras unfolds in two ways, namely, the geometric study of calibration (calculation of the projection matrix) and a radiometric calibration (uniformity of brightness in an image). The first problem is widely covered, whereas the latter is less studied.

In this chapter, we will approach the problem of geometric calibration of a video sensor in a didactic way and will solve it first using linear methods and then using non-linear methods. Often, written works talk about weak or strong calibration; the distinction is at the level of an overall estimate of the projection matrix (weak case) or in each parameter estimation which forms this matrix (strong case). The continuation of this work will deal with the problem of strong calibration of vision sensors.

1.2. General formulation of the problem of calibration

The process of calibration is a monitored process often requiring the operator’s attention. This is a static process, which is carried out offline, before using the camera for a precise visual task. Once the camera is calibrated, its parameters must remain fixed throughout its use. Each time we wish to modify the focus, the focal distance, and even the opening of lens, the camera will have to be recalibrated. Many works in the last 20 years have made it possible to obtain fairly complex and precise methods for assimilating a vision system in a metrological collection. The reader can refer to the following link for an exhaustive view of the major publications in the field: .

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!