Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric...

161
Multimodal Biometrics for Identity Documents 1 State-of-the-Art Research Report PFS 341-08.05 (Version 2.0) Damien Dessimoz Jonas Richiardi Prof. Christophe Champod Dr. Andrzej Drygajlo {damien.dessimoz, christophe.champod}@unil.ch {jonas.richiardi, andrzej.drygajlo}@epfl.ch June 2006 1 This project was sponsored by the Foundation Banque Cantonale Vaudoise.

Transcript of Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric...

Page 1: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Multimodal Biometrics for Identity

Documents 1

State-of-the-Art

Research Report

PFS 341-08.05

(Version 2.0)

Damien Dessimoz Jonas RichiardiProf. Christophe Champod Dr. Andrzej Drygajlo

{damien.dessimoz, christophe.champod}@unil.ch

{jonas.richiardi, andrzej.drygajlo}@epfl.ch

June 2006

1This project was sponsored by the Foundation Banque Cantonale Vaudoise.

Page 2: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Contents

1 Introduction 1

2 Definitions 42.1 Identity and identity documents . . . . . . . . . . . . . . . . . . 4

2.1.1 Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1.2 Identity Documents . . . . . . . . . . . . . . . . . . . . . 5

2.2 Biometrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2.1 General Definition . . . . . . . . . . . . . . . . . . . . . . 8

2.3 Use of biometrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.1 Biometric System . . . . . . . . . . . . . . . . . . . . . . . 92.3.2 Properties of biometric systems . . . . . . . . . . . . . . . 11

2.4 Processing steps for biometric data . . . . . . . . . . . . . . . . . 132.4.1 Biometric operations using the processing steps . . . . . . 15

2.5 Performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . 162.6 Evaluation protocols for Biometric Systems . . . . . . . . . . . . 19

2.6.1 Evaluation scenarios . . . . . . . . . . . . . . . . . . . . . 192.6.2 Evaluation steps . . . . . . . . . . . . . . . . . . . . . . . 19

2.7 Biometric systems architecture . . . . . . . . . . . . . . . . . . . 222.7.1 Smart Cards . . . . . . . . . . . . . . . . . . . . . . . . . 242.7.2 Template storage . . . . . . . . . . . . . . . . . . . . . . . 252.7.3 Processing locations . . . . . . . . . . . . . . . . . . . . . 26

2.8 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.8.1 Integration of biometrics with cryptographic techniques . 282.8.2 Cryptographic techniques for identity documents . . . . . 29

2.9 Vulnerability of biometric systems . . . . . . . . . . . . . . . . . 292.10 Multibiometrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.10.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . 312.10.2 Fusion scenarios and levels . . . . . . . . . . . . . . . . . 312.10.3 Fusion methods . . . . . . . . . . . . . . . . . . . . . . . . 332.10.4 Operational modes . . . . . . . . . . . . . . . . . . . . . . 34

3 Standards 363.1 BioAPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.2 CBEFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.3 ANSI X9.84 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.4 ISO/JTC1/SC37 . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.5 ICAO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.6 Wavelet Scalar Quantization . . . . . . . . . . . . . . . . . . . . . 41

i

Page 3: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

3.7 JPEG2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4 Legal framework, privacy and social factors 444.1 Legal framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.1.1 Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . 454.1.2 European Community - Council of Europe . . . . . . . . . 474.1.3 United States of America . . . . . . . . . . . . . . . . . . 474.1.4 France . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.1.5 Germany . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4.2 Biometrics as sensitive personal data . . . . . . . . . . . . . . . . 494.3 Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.4 Privacy protection . . . . . . . . . . . . . . . . . . . . . . . . . . 524.5 Public perception of biometrics . . . . . . . . . . . . . . . . . . . 54

5 Modality: Face 565.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.2 Overview of algorithmic approaches . . . . . . . . . . . . . . . . 57

5.2.1 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . 575.2.2 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . 575.2.3 3D recognition . . . . . . . . . . . . . . . . . . . . . . . . 58

5.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595.4 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.4.1 Ergonomics and acquisition environment . . . . . . . . . . 625.4.2 Face acquisition for identity documents . . . . . . . . . . 62

5.5 Computational resources . . . . . . . . . . . . . . . . . . . . . . . 635.6 Open source systems . . . . . . . . . . . . . . . . . . . . . . . . . 645.7 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.7.1 2D facial databases . . . . . . . . . . . . . . . . . . . . . . 645.7.2 3D facial database . . . . . . . . . . . . . . . . . . . . . . 65

5.8 International competitions . . . . . . . . . . . . . . . . . . . . . . 665.8.1 Competitions on FERET . . . . . . . . . . . . . . . . . . 665.8.2 Competitions on XM2VTS . . . . . . . . . . . . . . . . . 675.8.3 Competitions on BANCA . . . . . . . . . . . . . . . . . . 675.8.4 3D face verification competitions . . . . . . . . . . . . . . 67

6 Modality: Fingerprint 696.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696.2 Overview of algorithmic approaches . . . . . . . . . . . . . . . . 70

6.2.1 Human matching process . . . . . . . . . . . . . . . . . . 706.2.2 Automatic matching process . . . . . . . . . . . . . . . . 71

6.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746.4 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

6.4.1 Optical sensor . . . . . . . . . . . . . . . . . . . . . . . . 776.4.2 Solid-state sensor . . . . . . . . . . . . . . . . . . . . . . . 786.4.3 Ultrasound sensor . . . . . . . . . . . . . . . . . . . . . . 786.4.4 Ergonomics and acquisition environment . . . . . . . . . . 786.4.5 Fingerprint acquisition for identity documents . . . . . . 79

6.5 Computational resources . . . . . . . . . . . . . . . . . . . . . . . 796.6 Open source systems . . . . . . . . . . . . . . . . . . . . . . . . . 796.7 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

ii

Page 4: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

6.8 International competitions . . . . . . . . . . . . . . . . . . . . . . 816.8.1 Fingerprint Vendor Technology Evaluation 2003 . . . . . 816.8.2 Studies of One-to-One Fingerprint Matching with Vendor

SDK Matchers . . . . . . . . . . . . . . . . . . . . . . . . 816.8.3 Fingerprint Verification Competition . . . . . . . . . . . . 826.8.4 Second Fingerprint Verification Competition . . . . . . . 826.8.5 Third Fingerprint Verification Competition . . . . . . . . 826.8.6 Fourth Fingerprint Verification Competition . . . . . . . . 83

7 Modality: Iris 847.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 847.2 Overview of algorithmic approaches . . . . . . . . . . . . . . . . 847.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867.4 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

7.4.1 Ergonomics and acquisition environment . . . . . . . . . . 877.4.2 Iris acquisition for identity documents . . . . . . . . . . . 88

7.5 Computational resources . . . . . . . . . . . . . . . . . . . . . . . 887.6 Open source systems . . . . . . . . . . . . . . . . . . . . . . . . . 887.7 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 887.8 International competitions . . . . . . . . . . . . . . . . . . . . . . 89

8 Modality: On-line signature 908.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908.2 Overview of algorithmic approaches . . . . . . . . . . . . . . . . 91

8.2.1 Dynamic Time Warping . . . . . . . . . . . . . . . . . . . 918.2.2 Hidden Markov models . . . . . . . . . . . . . . . . . . . 928.2.3 Neural networks . . . . . . . . . . . . . . . . . . . . . . . 928.2.4 Euclidean distance . . . . . . . . . . . . . . . . . . . . . . 938.2.5 Regional correlation . . . . . . . . . . . . . . . . . . . . . 93

8.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 938.4 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

8.4.1 Electronics in the writing surface . . . . . . . . . . . . . . 948.4.2 Electronics in the pen . . . . . . . . . . . . . . . . . . . . 948.4.3 Electronics both in the pen and the surface . . . . . . . . 958.4.4 Ergonomics and acquisition environment . . . . . . . . . . 968.4.5 Online signature acquisition for identity documents . . . . 96

8.5 Computational resources . . . . . . . . . . . . . . . . . . . . . . . 968.6 Open source systems . . . . . . . . . . . . . . . . . . . . . . . . . 978.7 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978.8 International competitions . . . . . . . . . . . . . . . . . . . . . . 97

9 Modality: Speech 999.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 999.2 Overview of algorithmic approaches . . . . . . . . . . . . . . . . 100

9.2.1 Dynamic time warping . . . . . . . . . . . . . . . . . . . . 1009.2.2 Vector quantisation . . . . . . . . . . . . . . . . . . . . . 1009.2.3 Statistical methods: HMM . . . . . . . . . . . . . . . . . 1009.2.4 Statistical methods: GMM . . . . . . . . . . . . . . . . . 1009.2.5 Neural networks . . . . . . . . . . . . . . . . . . . . . . . 1019.2.6 Support vector machines . . . . . . . . . . . . . . . . . . . 101

iii

Page 5: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

9.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1019.4 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

9.4.1 Ergonomics and acquisition environment . . . . . . . . . . 1039.4.2 Voice acquisition for identity documents . . . . . . . . . . 103

9.5 Computational resources . . . . . . . . . . . . . . . . . . . . . . . 1039.6 Open source systems . . . . . . . . . . . . . . . . . . . . . . . . . 1049.7 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1049.8 International competitions . . . . . . . . . . . . . . . . . . . . . . 106

10 Multimodality 10710.1 Multimodality and Identity Documents . . . . . . . . . . . . . . 10710.2 Multimodal biometric systems and databases . . . . . . . . . . . 108

10.2.1 Single biometric, multiple sensors . . . . . . . . . . . . . . 10810.2.2 Multiple biometrics . . . . . . . . . . . . . . . . . . . . . . 10910.2.3 Single biometric, multiple matchers, units and/or repre-

sentations . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

11 Integration to identity documents 11511.1 ICAO technical specifications . . . . . . . . . . . . . . . . . . . . 11511.2 NIST recommendations . . . . . . . . . . . . . . . . . . . . . . . 11611.3 European Community - European Council, legal specifications . . 11611.4 Biometric identity documents projects . . . . . . . . . . . . . . . 119

11.4.1 Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . 11911.4.2 France . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12011.4.3 Belgium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12111.4.4 Italy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12211.4.5 Spain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12211.4.6 Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . 12311.4.7 Great-Britain . . . . . . . . . . . . . . . . . . . . . . . . . 12311.4.8 Germany . . . . . . . . . . . . . . . . . . . . . . . . . . . 12611.4.9 United States of America . . . . . . . . . . . . . . . . . . 12611.4.10 International Labour Organisation (ILO) . . . . . . . . . 127

12 Summary and future work 12812.1 Usage of biometrics . . . . . . . . . . . . . . . . . . . . . . . . . . 12812.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12812.3 Systems architecture and information security . . . . . . . . . . . 12912.4 Privacy and public perception . . . . . . . . . . . . . . . . . . . . 12912.5 Choice of modality . . . . . . . . . . . . . . . . . . . . . . . . . . 13012.6 Multimodality for identity documents . . . . . . . . . . . . . . . 13112.7 Acquisition and evaluation protocols . . . . . . . . . . . . . . . . 131

Bibliography 133

iv

Page 6: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 1

Introduction

The issues associated with identity usurpation are currently at the heart ofnumerous concerns in our modern society. Establishing the identity of individ-uals is recognized as fundamental to the numerous administrative operations.Identity documents (IDs) are tools that permit the bearers to prove or confirmtheir identity with a high degree of certainty. In response to the dangers posedby theft or fraudulent use of identity documents and security threats, a widerange of biometric technologies is emerging, covering e.g. face, fingerprint andiris. They are also proposed to enforce border control and check-in procedures.These are positive developments and they offer specific solutions to enhance doc-ument integrity and ensure that the bearer designated on the document is trulythe person holding it. Biometric identifiers - conceptually unique attributes -are today portrayed as the panacea for identity verification.

In many countries, identity document is increasingly associated with biomet-rics. Most modern identity cards include chips embedding biometric identifier.Under the impetus of the United States of America, a large number of coun-tries (all EU countries) are developing biometric passports. ICAO (InternationalCivil Aviation Organization, a United Nations specialised agency) issued spe-cific recommendations for travel documents inviting its members to use facialimages and optionally fingerprint or iris as a biometric modalities. The Swissgovernment is currently conducting a pilot study by testing and evaluating thepassport of next generation developed according to ICAO recommendations.

This project has been triggered by the frenetic technological promises andclaim of simplicity of biometric technology applied to identity documents. Webelieve that the deployment of such technology is a complex task that shouldreceive proper attention. This research initiative (MBioID) addresses the fol-lowing germane question: What and how will biometric technologies be deployedin identity documents in the foreseeable future? This research proposes to lookat current and future practices and systems for establishing and using identitydocuments and evaluate their effectiveness in large-scale deployments.

Today, most research is focused on studying various biometric modalitiesindependently. This renders comparisons between various biometric solutionsdifficult, thus a multi-modal approach will be favored in this initiative.

1

Page 7: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

This report constitutes the first milestone of the MBioID project. At theoutset of the initiative, it was felt that all relevant information should be gath-ered in a review document, in order to establish the current state-of-the-art. Insuch a rapidly evolving field, this step was of paramount importance to conductresearch both for the elaboration of acquisition and evaluation protocols and forthe establishment of a multimodal biometric research database. Discussion ofthe cost and whether or not the deployment of biometrics in this context willensure adequate security or improve border controls is one that requires politicalinvolvement. Such considerations have been left out of this report.

This report is organised as follows:

Chapter 2 presents the main biometric definitions that are required when ap-prehending such a field. Emphasis is put on the generic properties (including atypology of errors), processing steps and architectures (including cryptography)that are shared regardless of the biometric attribute considered. Multimodalbiometrics is also defined. It allows developing and reaffirming a common lan-guage that is sometimes missing in the literature and insist on the importantdistinction between using biometrics for verification purposes as opposed tousing biometrics for identification purposes. Verification implies a 1 to 1 com-parison between the acquired feature of a person claiming an identity and onetemplate corresponding to that identity. Identification aims at finding an un-known individual in a database of N persons.

Interoperability for biometric passport is a key component to ensure all pass-ports issued by each country are readable by the readers placed at borders.Chapter 3 offers a review of the main standards developed in the biometricindustry and standardisation organisations (such as ISO) to ensure interoper-ability.

Biometrics and associated issues such as privacy and personal data protec-tion are bound to get unprecedented levels of attention. Biometric informationis personal data. Adding a biometric measure to identity documents cannot beenvisaged without assessing the legal framework and the potential impacts onprivacy and without raising questions regarding the relationship between thestate and the citizen and the proportionality of the state’s actions. What willbe collected, why and how are questions that the state needs to address in atransparent manner. Following the 27th International Conference of Data Pro-tection and Privacy Commissioners (September 2005), a resolution on the useof biometrics in passports, identity cards and travel documents calls for the im-plementation of effective safeguards and the technical restriction of the use ofbiometrics in passports and identity cards to verification purposes. Chapter 4looks internationally at the tendencies on these issues. The option has beentaken to leave out a full historical perspective, even though such a perspectiveis decisive to fully apprehend what is at stake. For that, we invite the readerto refer to the collective work edited by Caplan and Torpey 1 and to the recent

1Caplan J and Torpey J, Documenting Individual Identity - The Development of StatePractices in the Modern World. Princeton: Princeton University Press, 2001.

2

Page 8: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

issue of the Cahiers de la Securite (2005, No56, Police et identification).

Chapters from 5 to 9 summarise the state of affairs in terms of technologicaldevelopment for a range of biometric modalities (face, fingerprint, iris, on-linesignature and speech) that can potentially play a role in identity documents.The choice of these modalities has been based on ICAO recommendations andavailabilities. No doubt that the future may bring additional biometric solu-tions, for example ear morphology. Each technology has specific strengths andweaknesses that are reviewed. The framework of analysis is analogous for allchosen modalities. It encompasses an overview of the algorithmic approach, as-sessments of performance, available sensors, computational resources requiredand availability of systems and databases. The aim is to offer a structured com-pilation of information that is not actually found in such a format.

At present, biometrics in identity documents are mainly investigated withone biometric modality. Given that the accessibility of the biometric feature isnot perfect (e.g. a few percent of the population cannot be enrolled using finger-prints or iris because of specific activities, age, ethnic background or disability)multimodality can be viewed as a mechanism to avoid penalising individuals whodo not have the required biometrics. The opportunities and constraints offeredby a combination of biometric features in this area are discussed in Chapter 10with an emphasis on the description of existing multimodal databases.

Chapter 11 tackles the issues associated with the integration of biometricfeatures in identity documents, by reviewing the ICAO and NIST (NationalInstitute of Standards and Technology) technical requirements and presentingtoday’s state of play in various countries and organisations.

To conclude this review document, we propose in Chapter 12 a short sum-mary and outline of the future work that will be undertaken under the MBioIDinitiative. The deployment of biometrics in identity documents cannot be seenas a single technological challenge. The issues associated with it are technologi-cal, legal or societal, and call for a larger debate. Taking performance issues forexample, the rate of failures to enrol forces to deal with exceptions procedureand discuss its fairness when compulsory enrolment is suggested. Choosing anadequate biometric technology for that task will indeed be guided by consider-ing rates of false matches and non-matches. But setting the acceptable limitswill require political decisions that are outside the strict technological arena.

We take this opportunity to thank the University of Lausanne, the EPFLand the Foundation BVC for the financial support to the research initiative.

3

Page 9: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 2

Definitions

2.1 Identity and identity documents

2.1.1 Identity

The term of identity is defined as “the quality or condition of being the same insubstance, composition, nature, properties, or in particular qualities under con-sideration” [1]. The personal identity is thus a data set that allows to recognizea person and to distinguish her from another one, and that can establish theidentity of this person. Identity always refers to a reference group or community.Indeed, our identity is function of this group or community in which we are insome point in time. According to Edmond Locard [166], this data set is at thebasis of the identification process and allows to distinguish the identical fromthe similar.

“En police scientifique et en droit, l’identite est l’ensemble des carac-teres par lesquels un homme definit sa personnalite propre et se dis-tingue de tout autre. Dans ce dernier ordre d’idees, etablir l’identited’un individu, est l’operation policiere ou medico-legale appelee iden-tification. Un homme peut etre semblable a plusieurs autres, ou a unautre, au point d’amener des erreurs; il n’est jamais identique qu’aun seul, a lui meme. C’est a discriminer avec soin les elementsde ressemblance des elements d’identite que consiste le probleme del’identification”.

For Paul Kirk [158], this individualisation process is also essential to crimi-nalistics:

“The real aim of all forensic science is to establish individuality, orto approach it as closely as the present state of the science allows.Criminalistics is the science of individualization.

An identity is also defined as “a presentation or role of some underlyingentity” [66]. In the case of a a human being, this entity can have some physicalfeatures such as its height, weight or DNA, called attributes. The identity of thisentity has also some attributes, such as a username, a social security numberor particular authorisations and permissions. The term legal identity , usually

4

Page 10: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

assigned to every citizen, can be introduced here, referring to the fact thatall human beings should be known and individualized by their registry office[182]. In the case of identity documents, it is this legal identity, associated to aparticular entity, which can be verified.

Three approaches are available to prove a person’s identity [194] and toprovide ”the right person with the right privileges, the right access at the righttime” [291]. These identity proving approaches, which establish the genuinenessof the identity, are:

Something you have The associated service or access is received throughthe presentation of a physical object (keys, magnetic card, identity document,. . . ), in possession of the concerned person.

Something you know A pre-defined knowledge, as a password normally keptsecret, permits to access a service.

Something you are Measurable personal traits, such as biometric measures,can also be used for identity prove.

A combination of these approaches makes the identity proof more secure.In day to day activities, the combination of possession and knowledge is verywidespread. The use of the third approach, in addition to the others, has signifi-cant advantages. Without sophisticated means, biometrics are difficult to share,steal or forge and cannot be forgotten or lost. This latter solution provides thusa higher security level in identity prove.

2.1.2 Identity Documents

An identity document (ID) is ”something written, inscribed, etc., which furnishesevidence or information upon a subject” [1]. This piece of documentation is de-signed to prove the identity of the person carrying it. The document contains adata set that allows a person to be recognized and distinguished from anotherone. According to the Swiss law, an identity document, a passport or an iden-tity card [70] 1, certify the Swiss nationality and the identity of the owner [69] 2.

The International Civil Aviation Organization (ICAO), an agency of theUnited Nations, recommends, in its technical report [125], the deployment ofMachine Readable Travel Documents (MRTDs), containing also biometric mea-surements. The characteristics selected by this organisation to be used in thenew generation of identity documents (facial, fingerprint and iris images) haveto be stored on a contactless chip with specific technical constraints3. Thechoice of using MRTD was imposed by the willingness to improve the registra-tion (entries and exits) at the borders. The use of biometrics came after, withthee purpose of facilitating the verification process and increasing security. Fig-ure 2.1 presents an example of a machine readable part, line surrounded, of a

1Swiss federal prescription on identity documents available online athttp://www.admin.ch/ch/f/rs/1/143.11.fr.pdf.

2Swiss federal law on identity documents available online athttp://www.admin.ch/ch/f/rs/1/143.1.fr.pdf.

3Please note our remarks on contactless storage in Section 2.7.

5

Page 11: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Swiss identity document. A more complete description of the ICAO standardsis presented in Section 3.5.

Figure 2.1: Machine readable part of a Swiss identity document.

According to the ICAO Doc 9303 4 [124], a secure issuance process is de-scribed as follows:

- Criminal background checks in respect of document applicants and officialsinvolved in the passport issuance process.

- Secure access to facilities and computer networks dedicated to document is-suance.

- Detection and elimination of duplicate documents (in the registry).

- Identity verification of applicants who claim ownership of issued documents.

The current issuance process of an identity document without biometricinformation, as well as temporary documents, in Switzerland can be describedas follows 5:

- The applicant has to register personally his request with the competent mu-nicipal authorities.

- The applicant has to furnish several breeding documents to prove who he is,such as an old identity document, origin act or individual certificate offamily status for single person and family book for married persons.

- The competent municipal authorities collect personal data, which will haveto appear on the identity document, and biometrics, such as currently arecent photograph, fulfilling special conditions.

- Issuance accepted or denied.

4ICAO Doc 9303 available after ordering at http://www.icao.int/mrtd/publications/doc.cfm.5Description of the Swiss issuance process available at

http://www.schweizerpass.admin.ch.

6

Page 12: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

After this process, the owner of this new document can prove and establishhis identity and his nationality, and can use it for general purposes and dailyactivities in the society.

The issuance process of an identity document with biometric information inSwitzerland, set up for the pilot project, will follow these steps 6:

- The applicant has to register personally his request with the competent mu-nicipal authorities, furnishing the same breeding documents as the currentissuance process.

- The applicant has to go in a biometric enrollment center five days or at mostthirty working days after the registration. A facial image will be acquiredand stored in the Swiss identity document’s information system, calledISA.

- The new passport will be recieved at least after 30 working days. The appli-cant has the possibility to check in a biometric enrollment center if thepassport is operational. Furthermore, the applicant has the possibility toconsult in biometric enrollment centers the data stored in the chip.

In other countries, the issuance process is similar to the Swiss one. For ex-ample, Figure 2.2 describes the issuance process in the United States of Americafor a visa, including collection of biometric data.

Figure 2.2: United States of America’s identity document issuance process [52].

It is important to warn against potential breach of security at all the stagesof this issuance process. Indeed, a security breach for the breeding documents

6Description of the Swiss issuance process for biometric identity document available athttp://www.schweizerpass.admin.ch. Section 11.4 presents a complete description of thenew Swiss biometric passport.

7

Page 13: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

to be supplied can make useless a security increase, such as the use of biomet-rics, in the identity documents. So the documents establishing the identity inthe first steps have to be genuine. It is then important that the administrativepersonnel involved in this process possess robust knowledge in the detection offorgery and have at their disposal original documents for comparison, as wellas specialised knowledge on security elements integrated into documents to besubmitted.

In this technical report, the term of Identity Documents is related to traveldocuments used for verifying the identity of the owner, as a visa, an identitycard or a passport. Some documents are also available, for other purposesthan border checking, for verifying the rights of the owner to some loans andauthorisations, such as driver license, medical and allowance card and specificaccess to a protected place.

2.2 Biometrics

2.2.1 General Definition

Every human being has experience in recognizing a familiar person, in dailyactivities, by using his/her specific characteristics, like voice, face, gait, hand-writing and signature. Some people, more than others, have even the ability torecognize unknown persons, after having seen or heard them.

The difficulties associated with person identification and individualisationhave already been highlighted by the pioneers of forensic sciences. AlphonseBertillon developed in the eighteenth century an anthropometric identificationapproach, based on the measure of physical characteristics of a person [24].The term of biometrics used today is the scientific follow-on of this approach,abandoned in these days in favor of fingerprinting.

Biometrics is a term that encompasses ”the application of modern statisti-cal methods to the measurements of biological objects” [1]. However, by lan-guage misuse, the term biometrics usually refers to automatic technologies formeasuring and analyzing biological and anthropological characteristics such asfingerprints, eye retinas and irises, voice patterns, facial patterns, and handmeasurements, especially for identity prove. Biometrics refers ”to identifyingan individual based on his or her distinguishing characteristics” [34]. Note thata legal definition of the term biometrics does not exist at the moment [110].Actually, such technologies are used in several domains, person authorizationexamination in e-Banking and e-Commerce transactions or within the frame-work of access controls to security areas. Ideally the biometric characteristicsused should satisfy the following properties [280] 7:

Robustness Over time, the characteristic should not change (Permanence),and thus have a low intra-class variability.

Distinctiveness Over the population, a great variation of the characteristicshould exist (Uniqueness), and thus have large inter-class variability.

7The corresponding terms used earlier by [64] for these ideal properties are indicated inbrackets.

8

Page 14: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Availability Ideally, the whole population should possess the characteristic(Universality).

Accessibility The characteristic should be easy to acquire (Collectability).

The characteristics could be physiological or behavioral [194]. Characteris-tics, which can be measured on a part of the body at some point in time (pas-sive), are physiological biometrics. On the other hand, characteristics, which arelearned or acquired over time (active), are called behavioral. These last char-acteristics are produced by an individual with a special effort, and are hencedependent to some degree on his state of mind. For example, fingerprint, handgeometry and face are physiological biometrics, while dynamic signature, gait,keystroke dynamics and lip motion are behavioral onces. Biometric character-istics such as voice could even be viewed as a combination of physiological andbehavioral traits [34, 146]. Indeed, the voice depends on physical features suchas vibrations of vocal cords and vocal tract shape, but also on behavioral fea-tures, such as the state of mind of the person who speaks. Another class ofcharacteristics can be added to the classification of biometrics [194]: the bio-

logical ones. The main biological characteristic is the Deoxyribonucleic Acid(DNA). It can be detected in biological material, such as blood, skin and salivaand is often cited as the ultimate biometrics [34, 146]. Biometrics of this cat-egory are never used, nowadays due to technological constraints, for instantautomatic identity prove process, while those of the two other classes are oftenused for such purpose.

In this report, the term modality will be used to name these characteristicsseparately.

2.3 Use of biometrics

2.3.1 Biometric System

A biometric system is essentially a pattern-recognition system [146, 223]. Sucha system involves three aspects [141]: data acquisition and preprocessing, datarepresentation, and decision-making. It can thus compare a specific set of physi-ological or behavioral characteristics to the characteristics extracted beforehandfrom a person, and recognise this last one. The digital representation recordedin a database, which describes the characteristics or features of a physical trait,is defined as a template. It is obtained by a feature extraction algorithm. Thecomplete process is described in Section 2.5. Biometric systems are traditionallyused for three different applications [202]: physical access control for the pro-tection against unauthorized person to access to places or rooms, logical accesscontrol for the protection of networks and computers, and time and attendancecontrol.

An authentication procedure, a way“to let the system know the user identity”in information technology [173], can be performed in two modes by a biometricsystem [34, 146]:

9

Page 15: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Identification This method consists in selecting the correct identity of anunknown person from a database of registered identities (Figure 2.3). It is calleda ”one to many” matching process, because the system is asked to completea comparison between the person’s biometrics and all the biometric templatesstored in a database. The system can take either the ”best”match, or it can scorethe possible matches, and rank them in order of similarity [291]. Two modesare possible, positive and negative identification, as described in a taxonomyof uses in [280] (Figure 2.4 summarizes the differences between them). Thepositive identification tends to determine if a given person is really in a specificdatabase. Such a method is applied when the goal is to prevent multiple usersof a single identity. A negative identification determines if a given person is notin a “watchlist” database. Such a method is applied for example when the goalis to identify persons registered under several identities.

Figure 2.3: Identification task [173].

Figure 2.4: Summary of the taxonomy of uses [280].

Verification This method consists in verifying whether a person is who he orshe claims to be (Figure 2.5). It is called a ”one to one” matching process, asthe system has to complete a comparison between the person’s biometric andonly one chosen template stored in a centralized or a distributed database, e.g.directly on a chip for an identity document. Such a method is applied when the

10

Page 16: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

goal is to secure and restrict specific accesses with obviously cooperative users.

Figure 2.5: Verification task [173].

The application environments of biometric systems are variable, and thusa taxonomy has been proposed in this field [280]. They are classified in sixcategories:

Overt vs. covert If the user is aware about the acquisition of his biometriccharacteristics, the application is overt and declared; if not, the use is calledcovert.

Habituated vs. non-habituated If the user presents his biometric charac-teristic(s) every day, the application is considered as habituated (after a shortperiod of time); if the frequency of use is low, the application is considered asnon-habituated.

Attended vs. non-attended If the user is observed and guided by super-visors during the process, the application is called attended; if not, the use iscalled non-attended.

Standard vs. non-standard environment If all the conditions can becontrolled and if the use takes place indoors within standard conditions, theapplication deployment is called within standard environment; if not, the use iscalled in non-standard environment.

Public vs. private If the users are customers of the system, the applicationis public; if the users are employees, the application is called private.

Open vs. closed If the system used works on completely proprietary formats,the application is called closed; if the system can exchange any data with otherbiometric systems used in other application, the use is called open.

2.3.2 Properties of biometric systems

Besides the basic properties that a biometric has to satisfy (see Section 2.2.1),some additional properties have to be considered in a biometric system [34, 64,146]:

11

Page 17: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Performance All the factors that influence and affect the accuracy and thecomputational speed of a biometric system.

Acceptability The population should accept the fact that the characteristicis taken from them.

Circumvention The ability of a system to resist against potential threatsand spoof attacks.

Exception handling The ability to complete a manual matching process inthe case of an impossibility of features’ extraction and modality use for certainpersons.

System Cost All the costs of the system components, in adequate and nor-mal use.

Some limitations, related to the properties described above, have been no-ticed when only a unique modality is used in biometric systems [146]:

Acquisition of noisy data The biometrics itself can be perturbed by noise,for example a scar on a fingerprint. The acquisition process can also be thereason of the background noise, for example, a fingerprint sensor maintained ina non-appropriate state, changing face’s illumination during the acquisition, orbackground noise when a speaker recognition system operates.

Intra-class variability The data acquired during the enrollment can be dif-ferent from the data acquired at the authentication process, because of changesin the technical characteristics of the sensor, an incorrect human-machine in-teraction, or simply day-to-day variability of the modality, affecting thus thematching process.

Distinctiveness While an ideal biometric trait should satisfy the property ofuniqueness, the features extracted from real trait always possess some interclasssimilarities, making this property less specific. The term of inter-class variabilitycan be used to describe this phenomenon.

Non-universality The desired property of universality of a biometrics meansthat everybody should have the specific trait. However, it is not the case. Thereare some people for which the system is not able to extract the features of theirbiometric, either due of the poor quality of the features, or because they cannot present their trait at all. According to [146], about 4% of the populationhas not fingerprint ridges of sufficient quality to be enrolled. Recent researchof the National Institute of Standard and Technology (NIST) tried to estimatethe proportion of the population who have fingerprints that are hard to match[115], and the results were not so alarming. On a set of 6’000 frequent users ofthe US-VISIT programme (with 10 samples of each right and left index), theauthors noticed that none of the subjects had fingerprint ridges that were alwayshard to match and less than 0.05% of the subjects had fingerprint ridges thatwere usually hard to match.

12

Page 18: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Spoof attacks Biometric systems have to resist against spoof attacks anddetect impostors who want to circumvent the system. Biometrics that can beacquired surreptitiously are particularly vulnerable in that regard (voice andface can easily be covertly acquired, but with some degree of inventiveness sig-nature and fingerprints can be as well). Biometric systems using physiologicalbiometrics such as fingerprint and iris have also recently been proved likely tobe spoofed [184, 185]. Spoof attacks can occur by using artificially created bio-metrics, by attacking via input port and at database [251], or by producing anydata such as a noised facial image that allow to establish a fake identity [160].Anti-spoofing measures can be the use of passwords or smart cards, a supervis-ing acquisition process during the transactions, the use of multimodal biometricsystems, the use of quality measures of the input signal and the use of livenessdetection methods. Systems which have not any liveness detection countermea-sure, whether it is for fingerprint or iris, can be fooled by using either artificialgummy fingers, or simple eye images printed on paper. For detecting spoof at-tacks through fingerprint sensors for example, distortion [12], perspiration [83]or odor [16] can be used. It should also be mentioned that some biometric char-acteristics are revocable, and other not. Indeed, there is a “life-long linking ofthe biometric characteristic to the data subject” [7]. The revocation means thepossibility to change or modify the biometric characteristic if this latter is com-promised by a spoof attack (i.e. in the case of an ”identity theft”). The signaturemodality is the only biometric characteristic that can be modified completely ifit is compromised. However, when fingerprints are used for identification or ver-ification purposes, the possibility of choosing another finger for the recognitionprocess remains, which is also the case for iris and hand geometry/palmprintmodalities, but only to some extent.

2.4 Processing steps for biometric data

For all biometric modalities, biometric data will be transformed according tothe processing steps described below and summarised in the activity diagram ofFigure 2.6.

Capture or acquisition The biometric data (voice, on-line signature, finger-print, . . . ), also called biometric presentation, is digitised via the input device(microphone, pen tablet, fingerprint scanner, . . . ) and stored in memory.

Preprocessing The signal-domain acquired data is prepared for feature ex-traction. This is typically used for normalising the signal-domain data andremove biases or sources of corruption in a systematic fashion. For speech, thiswill for instance include DC component removal as well as silence detection andremoval. For signatures, this stage would include translating the signature tostart at (0, 0) coordinates and resampling the signature. For fingerprints, thismay include rotation normalisation and thinning (skeletarisation).

Feature extraction Discriminative features are extracted from the preprocesseddata. Although features are very different for each biometric modality, the gen-eral underlying principle remains the same: this processing step typically reduces

13

Page 19: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

acquire signal

pre-process signal

extract features

post-process features

create template

store template

compute threshold store threshold

[enrollment]

get user template and threshold

[verification]

match template decide

get user templates 1..N match templates output N-best scores

[identification]

Figure 2.6: Processing steps for enrollment, verification, and identification.

the dimensionality of the input data to create a feature-level representation ofinput patterns that will be used by the classifier to perform pattern recognition.Typical examples of features include Mel Frequency Cepstral Coefficients or Per-ceptual Linear Prediction coefficients for speech, tangent angles and velocitiesfor on-line signature, and minutiae locations for fingerprint: ”In general, featureextraction is a form of non-reversible compression, meaning that the originalbiometric image cannot be reconstructed from the extracted features” [280].

Postprocessing Features are normalised to remove bias or adapt them tothe classifier. An example of removing feature-domain bias is cepstral meansubtraction for speech, where transmission channel effects can be compensatedfor. Additionally, certain classifiers such as neural networks or support vectormachines work best when their inputs have comparable dynamic ranges.

Template creation User models, also called templates, are created fromtraining feature sets to obtain a generic representation of a user that will beused for future comparisons. Many algorithms and procedures can be used de-pending on feature and model class. For speech or signatures this can involvetraining Gaussian mixture models (GMMs) using an iterative procedure.

Background model creation A background model, also called world modelor anti-model, is needed by some biometric algorithms to provide normalisationfor user presentation scores. They represent an “average” of the users fromthe population of the system. They are typically created by pooling togetherfeatures of many different users.

Template storage Once their parameters are estimated, user models arestored in a secure location for use in later biometric operations.

Template matching A biometric presentation is compared with a particularuser’s biometric template. This typically results in a presentation score whichis somehow related to how likely it is that this particular user is the source of

14

Page 20: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

that presentation. Depending on model and classifier types, this processing stepwill vary. For instance, GMM classifiers will use a likelihood-based score. Fora given presentation, match scores are typically computed as the ratio of thescore of the presentation with respect to a particular user’s model to the scoreof the presentation with respect to the background model. Thus, this representsa kind of hypothesis testing, where the hypothesis is “is it more likely that thispresentation was produced by this particular user rather than anyone else in thebackground population?”.

Threshold computation Several presentations belonging to a particular userand several presentations not belonging to that particular user (impostor pre-sentations) are matched to that user’s model to determine a hard limit (thethreshold) below which a presentation will not be considered as belonging tothe user. Thresholds can be user-independent (system-wide) or user-dependent,which is largely reported to give lower error rates. Again, many threshold com-putation procedures exist but most do work in the presentation score domain.Not all biometric modalities need a threshold, for example fingerprint matchingrequires no threshold.

2.4.1 Biometric operations using the processing steps

The processing steps described above will be used in the following higher-levelbiometric operations.

Enrollment A user is added to the biometric system. A certain number ofbiometric presentation of a particular user are acquired, preprocessed, trans-formed into features, and postprocessed, then used to train a user model andadapt (retrain) the world model if necessary. The user model along with im-postor presentations may be used to obtain a threshold for that user. The newmodel is then stored, along with the threshold for that user if needed.

Verification The claim to a user’s identity causes the presented biometricdata to be compared against the claimed user’s model. Thus, the biometric datais acquired, preprocessed, transformed into features, and postprocessed, beforebeing matched with the claimed user’s model and the resulting score beingcompared with the stored threshold computed for the claimed user or a genericthreshold value.

Identification A database of user models is searched for the most likely sourceof the biometric presentation. Thus, the biometric data is acquired, preprocessed,transformed into features, and postprocessed, before being matched with all theuser models of interest. The user model that obtains the highest score withrespect to the presentation is suggested to be the source of the presentation.

15

Page 21: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

2.5 Performance metrics

The performance of biometric systems is measured, qualified and expressed usingdifferent rates. Some of these rates are also used to measure quantitatively thefive properties, described in Section 2.3.2, for biometric systems [280].

During the enrollment, acquisition difficulties can appear with some people[34, 273], and can be quantified by rates called Failure to Acquire (FTA) andFailure to Enroll (FTE).

FTA Percentage of users for which the system has not the ability to presentand acquire a usable biometric sample during the enrollment and the trans-actions. FTA hence cover FTE and the quality assessment of templates. Forexample, the images’ quality of a fingerprint can be assessed by algorithms thatwill allow or otherwise refuse the creation of a template or the extraction offeatures (see Section 6.6 for more details on an algorithm developed by NIST).

FTE Percentage of users for which the system has not the ability to generatea template of sufficient quality for the enrollment because of limitations of thetechnology. (The FTE measures the availability of a biometric modality [280].)

Another measure that can dictate the performance of the enrollment processis the Time to Enroll rate, defined as:

TTE Duration of the enrollment process from capture the features of the phys-iological or behavioral trait to the creation of the biometric template.

Perfect error-free matches are never generated by systems [240]. In additionto the fact that uniqueness may not be achievable as such, the outcome of theenrollment process is influenced by factors [223], such as the acquisition condi-tions and the intra-variability of the features extracted from the modality. Asthe template and the physiological or behavioral trait are never exactly identi-cal, the system has to estimate and quantify the similarity between them andthen, according to the matching criterion, the threshold, a decision is taken bythe system. This result may not be in adequacy with the truth of the matterand the system may generate two decision errors regarding a user’s (or someoneelse) claim (verification mode), False Rejection Rate (FRR) and False Accep-tance Rate (FAR), often called type I and respectively type II errors. If no claimis made (identification mode), two matching errors are generated, False MatchRate (FMR) and False Non-Match Rate (FNMR) [176]. The FNMR measuresthe robustness, while the FMR measures the distinctiveness of a biometric sys-tem [280]. Accordingly, we have the following definitions:

FRR Percentage of users who claimed a specific identity or on which a claim(such as “He is on a watchlist”) was claimed or exclaimed, for which the systemhas either falsely rejected them during decision process, or does not have theability at all to acquire their biometrics (whereas it was the case during theenrollment). Hence FRR also cover the FTA/FTE aspects outside enrollment.

16

Page 22: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

FAR Percentage of users who claimed a specific identity, or on which a claim(such as “He is on a watchlist”) was claimed or exclaimed, and that the systemhas falsely accepted them for this claimed identity during the decision process(FTA/FTE should have no bearing here).

FNMR Frequency of occurrance of non match rate (i.e the submitted tem-plate does not match the one from person already enrolled).

FMR Frequency of occurrance of match rate (i.e. the submitted templatematches the one from another person already enrolled).

In negative identification systems, the Binning Error Rate (BER) and thePenetration Rate (PR) can be measured [176]. Indeed, some algorithmic ap-proaches partition the data in subspaces, in order to minimise the number ofsamples to compare, and thus the duration of the matching process. But ”themore partitioning of the database that occurs the lower the penetration rate,but the greater the probability of a partitioning error” [176]. That will impacton FNMR.

BER Number of matching template-samples pairs that the system has placedin different bins, with regard to the number of pairs assessed.

PR Average o number of comparisons needed - under the binning scheme -between each sample and the database, divided by the size of this latter.

FRR and FAR are dependent on FMR, FNMR, FTA, BER and PR [176].Only if a single successful match influences the acceptance, the equations rep-resenting these dependences are:

FAR = PR × FMR × (1 − FTA) (2.1)

FRR = FTA + (1 − FTA) × BER + (1 − FTA) × (1 − BER) × FNMR(2.2)

Other rates which can be used with those two mentioned above, is the EqualError Rate (EER) and the Half Total Error Rate (HTER).

EER This rate refers to the threshold setting where FNMR and FMR (orFRR and FAR) are equal. This rate is often assigned as a summary performancemeasure.

HTER This rate refers to half of the addition of FRR and FAR.

A threshold is said to be set a priori if it is estimated only on the trainingdata, and a posteriori if it is estimated based on results with test data. It isnot a trivial problem to set the threshold a priori, and the EER figure with ana-posteriori threshold may be hard to reach with a threshold set a priori.

17

Page 23: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

One way to represent graphically FNMR and FMR is to use a DetectionError Trade-Off (DET) curve [179]. The error rates are plotted on both axesand the relative performances of several recognition systems can be better dis-tinguished. Figure 2.7 presents an example of DET curves, and thus the perfor-mances, of some biometric technologies on specific evaluation protocols [176].

Figure 2.7: DET curves presenting the performances of the main biometrictechnologies [176].

Table 2.1 presents some figures for different modalities, which are taken fromvery different tasks, and therefore should not be construed as a direct comparisonbetween biometric modalities.

Modality FMR (%) FNMR (%) ReferencesFace 1 10 [213]Fingerprint 0.01 2.54 [172]Iris 0.00129 0.583 [131]On-line signature 2.89 2.89 [298]Speech 6 6 [233]

Table 2.1: Performance examples of biometric systems.

A final way of estimating the performance of a biometric system is to focuson the matching process duration, called Time to Match Rate .

TTM Duration of the matching process since the end of the capture, until thesystem’s decision.

18

Page 24: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

2.6 Evaluation protocols for Biometric Systems

2.6.1 Evaluation scenarios

An evaluation of a biometric system should ideally be completed by independentorganisations [214]. The complete detailed procedure, the evaluation protocol,the testing procedure, the performance results and representative examples ofthe data set, should be made available for the scientific community, in orderto repeat the evaluation. Recommendations are also proposed to undertakesuccessful evaluations, such as testing biometric systems on data not previouslyseen, completing not too hard or not too easy evaluations. Three evaluationtests can be conducted [214]: the technology evaluation, the scenario evaluationand the operational evaluation.

Technology evaluation It consists in comparing algorithms for a specifictechnology in typical conditions, in order to establish the current state of theart and to measure the progress of each kind of approach. These evaluationsare the most frequent tests conducted on biometric systems. International com-petitions for fingerprint recognition (FVC), face recognition (FERET, FRVT),speaker recognition (NIST) and online signature (SVC) are examples of suchevaluations 8.

Scenario evaluation It consists in measuring the system performance in asimulated application, combining sensors and algorithms. The scenarios testedshould model real-world conditions as closely as possible. In [175], an exampleof a scenario evaluation is presented. Six modalities were used (face, fingerprint,hand geometry, iris, vein and voice) and the tests were conducted on 200 subjectsover a three-month period. The objectives were to measure the performance ofspecific biometric systems, to promote and encourage evaluation tests.

Operational evaluation It consists in measuring the performance of a spe-cific algorithm in a specific environment and on a specific target population.

2.6.2 Evaluation steps

An evaluation test can be characterized in 5 steps [176]: planning the evaluation,data collection, analysis, uncertainty of estimates and reporting of performanceresults.

Planning the evaluation The first step consists in determining what theevaluation will demonstrate and how it will be done. For an appropriate datacollection procedure, all the information about the systems to be tested areneeded. Then, the factors influencing the performance should be controlled,in order to observe or to minimize their effects. The subjects on which theexperiments will be conducted should be fully informed and their identity should

8Chapters 5 to 9 contain a detailed description of international competitions for somebiometric modalities.

19

Page 25: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

never be released. The acquisition for enrollment and testing should be time-lapse. The test size will influence the accuracy of the results, the larger thesamples, the more accurate the estimates will be. Two rules of thumb can beused for finding the test size: the Rule of 3 and the Rule of 30. The ruleof 3 allows to assess for a given number of comparisons N independent andidentically distributed returning by chance no errors, which lowest error ratecan be estimated. For a 95% confidence level, the error rate p is 9:

p ≈ 3/N (2.3)

The rule of 30 proposes that the confidence interval (for 90% confidence) forthe true error rate is within ± 30% of the observed error rate, when at least 30errors are observed 10.

In order to increase the number of transactions (genuine and impostor) andkeep the number of volunteers reasonable, the biometric characteristic of eachsubject can be acquired multiple times. The test size should nevertheless beas large as practicable and the number of samples should correspond to therequirements of the rules of 3 and 30. After collecting and analysing the data,the uncertainty in the observed error rates should be estimated, in order todetermine if the data set was large enough.

Even if the data collection is demanding, several operational evaluationsshould not be conducted simultaneously. However, the same data set can be usedfor technology evaluations, while for scenario evaluations, the presentations’order of the subjects to the sensors may be randomised, for decreasing theinfluence that the volunteers became familiar with the sensors.

Data collection During the data collection of the corpus (collected biometricsamples) and the database (information about the samples and the volunteers),errors should be avoided, for example an incorrect use of the system or theacquisition of corrupted samples, a wrong PIN for a specific subject or thepresentation of the wrong body part during the enrollment.

Ideally, the system should automatically allow the recording of the biometricinformation and log enrollments and transactions, such as the claimed identity,the quality and the matching scores. This approach will permit a full cross-comparison of the samples, thus increasing the number of impostor scores, animprovement evaluation of new algorithms based on the same set of samples,an examination of the transactions log for checking errors in the corpus and thedatabase, and a decrease of transcription errors. If the system evaluated returnsonly a binary decision, DET curves should be plotted in changing the securitysettings of the system. If the system allows only online testing, each subjectshould complete transactions at each of these security settings.

The enrollment, conducted only once for each subject, has to be made inadequacy with the evaluation scenarios. The conditions should be the same foreach enrollment in a technology evaluation, similar to the target applicationconditions and consistent throughout the process in a scenario evaluation, and

9For a confidence level of 90%, this equation becomes p ≈ 2/N.10For a confidence level of 90%, the confidence interval is within ± 10% of the observed

error rate, when at least 260 errors are observed.

20

Page 26: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

should not be controlled in an operational evaluation. The quality of the bio-metric information should be checked during the enrollment. The elapsed timeand/or the number of attempts for enrolling should be pre-determined, in orderto determine the FTE regarding these criteria. The quality threshold of theacquisition procedure will also affect the failure to acquire rate in scenario andoperational evaluation.

The elapsed time between the enrollment process and the technology eval-uation, influencing the template ageing phenomenon, should be determined re-garding the required application: the longer the elapsed time, the more difficultthe matching process will be. In scenario evaluation, this elapsed time shouldbe as close as the relevant target application, while in operational evaluation,this frequency of use should be balanced with frequent and non frequent sub-jects. In order to evaluate the template ageing and the fact that the user getsaccustomed to the system, multiple samples should be acquired over time indifferent sessions.

Whether or not the samples acquired during the evaluation matched withany template already enrolled, these samples should be included in the corpus.

The presentation and channel effects have to be either uniform or randomlyvarying across the subjects during the enrollment and the testing process intechnology and scenario evaluations, while these effects should not be controlledin the operational evaluation.

Online transactions, samples of a subjects’ set compared against the tem-plates of an other part of the database (randomly selected), or offline transac-tions, a full cross-comparison between every sample and every non-correspondingtemplate, can be conducted for evaluating impostor attacks. Some conditionsare nevertheless required. With partitioning systems, the templates againstwhich the samples will be compared online should belong to the same bin (i.e.same general shape for fingerprints and same gender for voice). The use of abackground database (large database containing biometric samples from differ-ent environment or population than the reference population) for online impos-tor transactions is not recommended, as these transactions should be made inthe same environment as the genuine transactions. When the templates aredependent, the subjects used for the online impostor transactions should notbe enrolled in the database. In an offline generation of impostor transactions,the samples of each subject are compared to a subset of the database, whichdoes not contain the corresponding template. The offline impostor transactionsshould also not contain within-individual comparisons, such as between fingersof a same subjects. These latter transactions should thus not be included in theimpostor transactions, as “within-individual comparisons are not equivalent tobetween-individual comparisons” [176].

Analysis The analysis of the evaluation can be completed using the ratespresented in Section 2.5, such as FTE, FTA, FMR, FNMR, FAR, FRR, BERand PR.

Uncertainty of estimates The uncertainties in the performance of biometricsystems can be generated by two errors types. The natural variations, calledsystematic errors, can be decreased by increasing the test size. The bias in testprocedures, called random errors, can be evaluated and measured in changing

21

Page 27: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

the environment conditions. For estimating the variance of the performancemeasures, some assumptions have to be made:

- The subjects used in the tests are representative of the target population.

- The transactions of different subjects are independent.

- The transactions are independent of threshold.

- The error rates vary across the population.

- The observed errors are not too limited in numbers.

Some equations are proposed in [176] for measuring the variance of falsematch and false non-match transactions, whether there were single or multipletransactions for each subjects.

For confidence intervals estimation, the errors observed can be approximatedby parametric distributions or by non parametric methods, such as bootstrap.Bootstrap values of FMR, obtained by sampling with replacement (more thanone occurrence of the same item available) from original samples allow the cre-ation of confidence intervals on the distribution of these values. At least 1000bootstrap samples for 95%, and 5000 for 99%, are generally required for a correctestimation of confidence intervals.

Reporting performance results In addition to all performance metricsmeasured during the analysis, the reporting should disclose all information aboutthe system, the type and the size of evaluation, the demographics of the sub-jects, the test conditions, the elapsed time between enrollment and testing, thequality and decision thresholds used during the acquisition, the factors influenc-ing the evaluated performance, the test procedure, examples of abnormal cases,the estimated uncertainties and the deviation from the requirements of [176].

2.7 Biometric systems architecture

Biometric systems can be implemented in different ways depending on the ap-plication scenario. These different implementations can be described in termsof the processing steps defined in 2.4. Essentially, distributed architectures splitprocessing tasks between different hosts (or computers, or machines) over awired or wireless network and typically make use of a verification or identifica-tion server, while centralised architectures keep all biometric processing tasks onone host, typically at the access point. Centralised architectures can be imple-mented on standard personal computers or embedded microprocessors such assmart cards (see 2.7.1). In the case of smartcards, a host computer to which thesmartcard reader is attached can perform some processing steps such as featureextraction.

Biometric systems architectures can also be described in terms of where thetemplate storage and matching take place. Figure 2.8 provides an overview ofdifferent storage architectures according to these two processing steps.

We stress that biometric template and/or raw biometric data storage shouldbe handled with the utmost care, having security and privacy in mind first and

22

Page 28: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

biometric systems

distributed storage centralised storage

push (features) pull (templates)

match on cardmatch on server match on client

Figure 2.8: Taxonomy for biometric systems storage architecture.

foremost. Unlike a PIN number, a compromised biometric template cannot bechanged (although for fingerprints it is possible to switch to other fingers).

Centralised databases containing personal information such as biometricdata or templates are very likely to become focal points of attack due to theirhigh value, and contribute to increasing (biometric) “identity theft” rather thandeterring it. Numerous recent stories about large consumer database serve tohighlight the problems inherent with centralising valuable data for a large num-ber of users. For example, ChoicePoint, possibly the largest US-based iden-tification and credential verification company, has over a period of one yearleaked information about 145’000 people to fraudsters from its database [188].More recently, on April 12th, 2005, LexisNexis admitted to a breach concerningapproximately 310’000 US citizens11, which was earlier announced to concern“only”32’000 people. To make matters worse, LexisNexis’ subsidiary Seisint hadbeen in charge of running the MATRIX (Multistate Anti-Terrorism InformationExchange) database together with the state of Florida. Even more recently, Cit-igroup lost track of backup tapes containing data for 3.9 million customers12.

Therefore, distributed storage architecture, where biometric templates and/ordata is only stored on the identity document, seems the most reasonable wayto prevent such large-scale security breaches, as the effort expounded to fraud-ulently access one person’s biometric template would need to be repeated foreach user.

Due to concerns about easy covert acquisition and subsequent possible re-covery of the secret cryptographic key [38] or indeed, any data stored, the useof Radio-Frequency IDentification (RFID) chips or other contactless media forstoring biometric data in passports should be avoided. In 2005, the CalifornianSenate proposed a bill (called the“Identity Information Protection Act of 2005”)to forbid the use of contactless chips in identity documents [255]:

The act would prohibit identification documents created, mandated,purchased, or issued by various public entities from containing a con-tactless integrated circuit or other device that can broadcast personalinformation or enable personal information to be scanned remotely,except as specified.

The proposal to store biometric data on a RFID chip has also come undercriticism from human and civil rights organisations worldwide [225].

Furthermore, the Swiss Federal Data Protection Commissioner has recom-mended in his Annual report, that adequate measures ought to be taken in order

11http://www.lexisnexis.com/about/releases/0789.asp.12http://www.informationweek.com/story/showArticle.jhtml?articleID=164301046.

23

Page 29: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

to avoid any illegal processing of personal data if RFID are used [228]. Indeed,the persons have to be informed“about the objective of the data processing andhave to know which data are processed, when, where and how”.

2.7.1 Smart Cards

Smart cards are most frequently made of flexible plastic (PVC), and embed anintegrated circuit micromodule containing read-only memory (ROM), randomaccess memory (RAM), application memory (often electrically erasable pro-grammable read only memory or EEPROM), microprocessor, and input/outputcontroller. In addition, a card operating system or chip operating system (COS)is present, which provides basic functions such as data management, access con-trol, and cryptographic operations.

Three prominent COS are JavaCard 13, MULTOS 14 and the ISO 7816COS [133], but countless other COS (including the short-lived Windows forsmartcards) exist in the industry. Both JavaCard and MULTOS allow appli-cations on the card to be changed if needs be, facilitating software upgrades.JavaCard applications are developed in Java, while MULTOS applications canbe written in C or native language for the underlying hardware. Both these COSprovide the concept of a firewall between on-card applications to prevent oneapplication from accessing data reserved to another application. Convergenceis under way in the form of the Global Platform specifications 15, which drawsfrom both Visa’s Open Platform specification and the MULTOS specification.

In recent years, smartcard hardware has become quite powerful and versatile,with for example the ST Microelectronics ST22 family offering 32 bits RISC-typemicroprocessors with 32 KB of RAM, 364 KB of ROM and 256 KB of EEPROM,as well as hardware encryption functions, and supporting JavaCard 2.1 programswithout recompilation. Thus, it seems that generic-purpose smartcards wouldprovide sufficient computing power to run biometric applications.

Lately, standards have emerged to integrate smartcards and biometric data,such as the JavaCard biometric API 16 and the MULTOS Biometry C API [177],which is compatible with the JavaCard biometric API.

Figure 2.9: Integrated fingerprint sensor-smartcard assembly from RiTech In-ternational ltd. (RiTech website).

13http://java.sun.com/products/javacard/.14http://www.multos.com/.15http://www.globalplatform.org/.16http://www.javacardforum.org/Documents/Biometry/biometry.html.

24

Page 30: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Numerous biometric-enabled smart cards are proposed by many vendors,and sensor-on-card solutions (Figure 2.9) exist for the fingerprint modality in awide array of form factors (USB sticks, PDAs, smartphones, mice...). Match-on-card is an attractive solution also because it offers very good scalability -no matter how many users are issued with biometric identity documents, thematching time will be the same as it is performed locally.

2.7.2 Template storage

For parametric classifiers17, a model of the user’s data will need to be stored toenable verification.

Template storage is an important issue because its compromission wouldenable an attacker to generate very effective synthetic attacks. Furthermore,template data can be reversible to some level, meaning the original biometricor some information about the user can be gained from the template. As anrelatively benign example, for a GMM-based speaker model, formant frequenciescan be estimated [77], meaning the gender or accent of the user can be inferred,albeit not easily. For face images, user templates (weights) based on eigenfacesare perfectly reversible from the eigenfaces and the mean face, which means theoriginal face image can be perfectly reconstructed (or a very good approximationif only the first few eigenfaces are used, as these contain most of the distinctivefeatures). Therefore, template information must be considered sensitive dataand stored and protected accordingly.

It should be noted that some classifiers (typically those used with the fin-gerprint modality) do not need background models and thus will require lessstorage space.

Templates can be stored in plain or encrypted form (see section 2.8), usinga variety of media.

Printed storage Printed storage comprises 1-dimensional and 2-dimensionalbarcodes. These typically have low storage capacity compared to other formsof storage (up to about 500 bytes per square inch for 2D), but are sufficient formost biometric template storage. They may be inappropriate to store multiplebiometric templates unless care is taken to choose an efficient encoding for thetemplates. They are well suited to be embedded in identity documents. Axtels’QR code18 has a maximum storage capacity of 2953 bytes, or about 2.88 kB.Their DataMatrix and PDF417 barcodes have similar or lower storage capacities.Datastrip’s 2DSuperscript19 barcodes can store from 90 to 9615 bytes (8.41 kB)depending on the size of the barcode area. De La Rue International has proposeda high-density 2D barcode which can store up to 32 kB of data [264].

Optical storage Optical storage such as compact discs or digital versatilediscs benefits from large storage capacities (in the low GB range) but sufferfrom relatively low transfer rates and large size. For these reasons they are notwell suited for embedding in identity documents.

17Non-parametric classifiers such as k-nearest neighbours store examples of the biometricpresentation instead and use no explicit modelling.

18http://www.axtel.com/QRCode.htm.19http://www.datastrip.com/english/products_detail.asp?id=268.

25

Page 31: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Figure 2.10: Example 2D QR barcode (Axtel’s website).

Solid-state storage Solid-state storage refers to integrated circuit chips orthe memory found in smart cards, and a large variety of removable storage mediaused in digital photography such as CompactFlash, SmartMedia, Memory Stick,etc. Their capacity (32 kB is common for algorithm storage, and 256 kB can beobtained in small form factors, for example see Fig. 2.11), low access times andsmall form factors make them well suitable embedding in identity documents.The interface needed to read data from the chip can necessitate contact or not.In the case of RFID contactless transmission, it should be noted that problemswill arise when an RFID biometric passport is appended with a RFID biometricvisa and that this solution is currently not feasible [55].

Figure 2.11: Solid-state micro-controller from Samsung (S3CC9EF) with 256kB storage and ISO 7816 compliance (Samsung website).

Magnetic storage Magnetic storage refers to magnetic substrate whose sur-face can be encoded to store digital information, as is the case in hard disksor DAT archive tapes. Their large capacity is counterbalanced by their relativesusceptibility to mechanical problems due to moving parts. While some solu-tions like IBM’s Microdrive go down to relatively small form factors, their costmay present an obstacle to widespread use.

Because solid-state and printed storage offer low footprints and sufficientstorage, they are well suited for use in biometric identity documents.

The ICAO has defined a standard for storing biometric data, known as “log-ical data structure” or LDS [126]. It comprises a series of data elements such asdate of birth, name, and optionally20 face or other biometrics.

2.7.3 Processing locations

Depending on the hardware where the processing take place, we define severalsolutions and their acronyms in Table 2.2. We assume a limit of 32 kB memory

20For ID issuing authorities, the use of biometrics is optional, but if a biometric is included,it must follow the LDS.

26

Page 32: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

for the smartcard solution and 32 kB for 2D barcode storage, so the DOC, FOCand TOC options also apply to 2D barcodes. This table is made to the best ofour knowledge.

Acronym Meaning 2D face FP Iris Sig. Speech

DOC data on card√ √ √ √

a ×FOC features on card

√ √ √ √ √

TOC template on card√ √ √ √ √

MOC match on card√ √ √ √ √

SOC sensor on card × √ × × ×

Table 2.2: Processing locations in a smart card-based architecture.

aDepends on signature duration.

Recently, three major biometric companies and one smart card supplier havepresented a multimodal match-on-card solution, with facial, iris and fingerprintdata [23]. The images of this three biometric data are stored on one high-specification Jav-based smart card and the matching process is performed onthe card itself.

2.8 Cryptography

Cryptography is the art and science of making messages very difficult to read tounauthorised parties. Encryption refers to the operation by which a plaintextmessage is converted to ciphered form, making it very difficult to read, whiledecryption refers to the operation by which the original plaintext message is re-covered from the ciphertext. Conversion to and from ciphertext occurs throughthe use of a secret piece of information called the key, which is a bit string of acertain length (a ballpark figure being from 64 to 4096 bits).

Two broad algorithms families can be distinguished in cryptography: sym-metric algorithms, where the same key is used for encryption and decryption,and public-key algorithms, where different keys are used for encryption anddecryption. Symmetric algorithms can be denoted as follows [250]:

Ek(M) = C

Dk(C) = M, (2.4)

where E is the encryption function using key k, M is the plaintext message,C is the cyphertext and D is the decryption function. Public-key algorithmscan be denoted as follows:

Ek1(M) = C

Dk2(C) = M, (2.5)

where k1 is the public key, which is publicly available and can be stored ona public-access key server, and k2 is the private key known only to the intendedrecipient of the message. Often, private keys are protected by passwords, butcan also be protected by biometric access control (see Section 2.8.1).

27

Page 33: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Another application of cryptography is for producting one-way hash func-tions. Hash functions, also called message digests or cryptographic checksums,are mathematical functions that can be applied to messages to produce uniqueand shorter output strings. Since no two different messages should produce thesame hash value if the hash function is collision-free, hashes allow interestedparties to verify that a message has not been modified.

Digital signatures allow a user to sign a message (or a hash thereof) to proveits authenticity. One amongst many possible algorithms based on public-keycryptography involves encrypting a hash of the message with the user’s privatekey. By encrypting the hash with her private key, the user has proved heridentity because only she knows the password to unlock the private key. At theother end, the recipient can verify the authenticity of the message by generatinga hash for it using the same hash function as the sender, then decrypting thehash sent with the public key of the sender. If the locally generated hash andthe sent hash match, the message has not been modified and has genuinely beensigned by the sender.

2.8.1 Integration of biometrics with cryptographic tech-niques

Private keys of users are typically protected against misuse by a password orpassphrase. If the correct password is provided, the key is released and can beused to decrypt a message. If the password is compromised, an unauthoriseduser may be able to use the private key and decrypt the message.

Thus, a first possibility for integrating biometrics and cryptography is to usebiometric matching to protect the secret key. This means biometric verificationand key release are two separate processes. A problem of this approach is thata biometric template still has to be stored to verify the identity claims. Thismechanism has been applied to smartcard-based systems for fingerprints [63].

A second possibility is to use the biometric data directly in cryptographic keygeneration. This can be done by combining biometric data and other randomdata to create a private key. It has been applied to fingerprints [259], face [105],palm [71], and iris [112].

One of the main issues to solve in this context is that for cryptography,all bits must match exactly otherwise the data cannot be decrypted (avalancheproperty); unfortunately, two biometric presentations never match bit-for-bit(which is one motivation for using statistical models). Possible solutions involvethe use of error-correcting codes, as for instance in [112], where the authors usetwo error correction codes that can withstand up to 20% bit errors.

Vielhauer et al. [275] have used computations based on global features of thesignature to form a 24-components hash vector. This could likely be used togenerate keys. While to the best of our knowledge no other research deal withthis issue, it is likely that the signature modality is suitable for biometrics-basedkey generation.

Tuyls and Goseling have proposed a generic cryptographic framework forprivacy-protective biometric authentication [269].

28

Page 34: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

2.8.2 Cryptographic techniques for identity documents

The ICAO has issued a technical report concerning protection of data storedon biometric identity documents using cryptographic techniques [157]. A briefsummary is provided in [156].

The ICAO technical report proposes passive and active authentication. Inpassive authentication, the chip holding the data does not perform any compu-tation. A data security object contains a hash of the contents of each data groupof the LDS (see Section 2.7.2), to ensure that they have not been tampered with.

In active authentication, the chip in the identity document has some capacityto perform computations and a challenge-response mechanism is put in place toensure the chip itself has not been substituted.

The report also defines key management standards and other aspects ofimplementation.

Juels, Molnar and Wagner [148] provide a good overview of security andprivacy risks related to the use of ICAO-based passports.

2.9 Vulnerability of biometric systems

Standard information security concerns apply to the biometric identity docu-ment case, perhaps even more stringently because of the very confidential natureof the data processed. Security issues with distributed systems (denial of service,sniffing, man-in-the-middle, replay attacks...) are also present in distributedbiometric architectures.

Tuyls and Goseling [269] propose an algorithm that protects privacy andachieves secure biometric authentication, using the following reasonable assump-tions about distributed biometric systems security:

• Enrollment is performed at a trusted Certification Authority (CA). TheCA adds a protected form of the user data to a database.

• The database is vulnerable to attacks from the outside as well as from theinside (malicious verifier).

• During recognition an attacker is able to present synthetic biometric sourcesat the sensor.

• All capturing and processing during authentication are tamper resistant,e.g. no information about raw biometric data or features can be obtainedfrom the sensor.

• The communication channel between the sensor and the verification au-thority is assumed to be public and authenticated, i.e. the line can beeavesdropped by an attacker.

All these assumptions can be taken as given in real-world deployment ofbiometric identity documents, except that the sensor/feature extraction devicemay not be fully tamper-resistant.

Centralised architectures such as match on card, where all processing stepsare performed internally, offer the highest system security because the user’sbiometric data never leaves the card. Existing industrial processes for smart

29

Page 35: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

cards can be adapted to the biometric case [211].

As mentioned by Albrecht 21, “the security of biometric systems dependslargely on the protection of the reference data and the comparison mecha-nisms” [6]. Indeed, the data acquired during the enrollment and the verifi-cation/identification transactions may be genuine. Furthermore, the acquisitionprocess has to be secured, in order to avoid any interception, replacement with-out the consent of the subject.

The main places of a biometric system in which attacks may occur [34, 230]are as follows (Figure 2.12):

Figure 2.12: Possible vulnerable points of a biometric system [34, 230].

1. Presenting fake biometrics at the sensor: A true biometric represen-tation is presented to the device, but obtained in an unauthorized manner,such as a fake gummy finger, an iris printout or a face mask.

2. Resubmitting previously stored digitized biometrics signals (re-play attack): The digitised biometric signal, which was previously en-rolled and stored in the databases, is replayed to the system, circumventingthus the acquisition device.

3. Overriding the feature extraction process: A pre-selected templateis produced in the features extraction module using a Trojan horse.

4. Tampering with the biometric feature representation: During thetransmission between the feature extraction and the matching module, afraudulent feature set replaces the template acquired by the device.

5. Attacking the enrollment center: The enrollment module is also vul-nerable to spoof attacks, such as those described in the previous points 1to 4.

6. Attacking the channel between the enrollment center and thedatabases: During the transmission, a fraudulent template replaces thetemplate produced during the enrollment.

21Member of the German Federal Office for Information Security (BSI) and Head ofTeleTrustT Working Group Biometrics, Germany.

30

Page 36: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

7. Corrupting the matcher: A pre-selected score is produced in the match-ing extraction module using a Trojan horse.

8. Tampering with stored templates: A template, previously stored inthe database (distributed or not), can be modified and used afterward ascorrupted template.

9. Attacking the channel between the stored templates and thematcher: During the transmission between the databases and the match-ing module, a fraudulent template replaces the template previously stored.

10. Overriding the final decision: The result of the decision module canbe modified and replace the output obtained previously.

11. Attacking the application: The application is also a point of attackand all existing security system should be used to reduce the vulnerabilityat this level.

2.10 Multibiometrics

2.10.1 Generalities

In real-world applications, some limits of monomodal biometric systems havealready been reported (see Section 2.3.2). Indeed some biometrics have only lit-tle variation over the population, have large intra-variability over time, or/andare not present in all the population. To fill these gaps, the use of multimodalbiometrics is a first choice solution [146, 242]. In [119], Hong, Jain and Pankantidemonstrated empirically the performance improvement in integrating multiplebiometrics. It seems that multibiometrics systems increase their robustness [238]and are more reliable [146, 241]. Indeed, multimodality approaches provide ap-propriate measures to resist against spoof attacks, as it is difficult to counterfeitseveral modalities at the same time, to circumvent a system. They also providean adapted solution to the limitations of universality, as even if a biometrics isnot possessed by a person, the other(s) modality(ies) can still be used.

2.10.2 Fusion scenarios and levels

As described in Section 2.3 and 2.4, biometric systems have four main compo-nents: sensor, feature extraction, matching-score and decision-making modules.The combination of single biometrics and the fusion of several modalities can becompleted at every stage of this process. Multimodal systems can be designedto operate in five different ways [146, 242] (Figure 2.13). Some of them may notinvolve multiple modalities but imply a fusion at some points. They are givenhere for sake of completeness.

Single biometric, multiple sensors The same biometric is acquired by dif-ferent sensors and combined to complete and improve the recognition process.

Multiple biometrics Different biometrics of a same person are acquired andcombined to complete and improve the recognition process. This approach isthe only well-named multimodal biometric fusion scenario.

31

Page 37: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Figure 2.13: Combination schemes [146].

Single biometric, multiple units The same biometric, but different units(e.g. two different fingers), are acquired and combined to complete and improvethe recognition process.

Single biometric, multiple representations The same biometric unit isacquired several times by a same sensor and combined to complete and improvethe recognition process.

Single biometric, multiple matchers The same biometric is acquired onesby a single sensor and different approaches of features extraction and matchingare combined to complete and improve the recognition process.

From the scenarios described above and the four important components ofbiometric systems, the combination and the fusion of the information acquiredin each stage is possible (Figure 2.14).

Features extraction This fusion mode consists in combining the featuresextracted from biometrical traits, into a unique features vector, if the modalitiesor the features extracted are independent between them.

Matching score level This fusion mode consists in combining the scores,which describe the similarities between the biometrics acquired and their tem-plates, obtained by each biometric system. This mode requires a scores’ normal-ization, as the scores have to belong to a common domain before the combination

32

Page 38: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[241, 242]. A two-step process can be completed: statistical estimation of thescores distribution and translation into a common domain.

Decision level This fusion mode consists in combining the decisions taken byeach biometric system, to obtain a final decision.

Figure 2.14: Fusion levels possibilities [241].

The earlier the fusion is operated, the more biometric systems are believedto be effective [242]. Indeed, fusion at a features extraction level is expectedto provide better accuracy. However, the difficulties to complete this kind offusion, as state of the art biometric systems generally do not allow access to thisstage, the next level, the matching score level, is usually preferred. But reliableand robust biometric systems require suitable decision-level fusion approaches[238].

2.10.3 Fusion methods

In this Section, some fusion methods will be described. In the matching scorelevel, methods such as simple sum rules [242], weighted averaging [146], product

rules, k-NN classifiers, decision trees and Bayesian methods [241] can be usedto combine scores obtained by biometric systems. Such approaches providesignificant performance improvement. In [238], Roli et al. have subdivided the

33

Page 39: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

decision fusion methods in two main categories: fixed and trained rules. Fusionstrategies such as majority voting and sum rule are fixed rules, and shouldperform well when different systems with similar performance are combined. Onthe other hand, techniques such as weighted averaging and behavior knowledge

space are trained rules and should perform well when systems with differentperformances are combined. User-specific parameters can be used to improvefurther the performance of biometric systems, with techniques such as user-specific thresholds and weights [145, 242].

2.10.4 Operational modes

Furthermore, from an operational aspect, biometric systems can perform inthree different modes: (a) parallel, (b) serial and (c) hierarchical mode [146, 242](Figure 2.15).

Figure 2.15: Classifier schemes [173].

Parallel mode This operational mode consists in completing the combinationof the modalities simultaneously.

Serial mode This operational mode consists in completing the combinationof the modalities one after the others, as it permits to reduce at the start theinitial population before the following modality is used. The decision could thus

34

Page 40: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

be taken before all the remaining biometrics are acquired, reducing considerablythe processing duration (Time to Match rate).

Hierarchical mode This operational mode consists in completing the com-bination of the modalities in a hierarchical scheme, like a tree structure, whenthe number of classifiers is large.

35

Page 41: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 3

Standards

International standards relating to biometrics are maturing quickly, and manyare already available. They support interoperability and data exchange betweenapplications and systems, thus avoiding problems and costs stemming from pro-prietary systems. See [30] for a quick overview of future biometric standardswhich are still not approved at an international level, but may be soon approved.

For identity documents such as passports, international standards are essen-tial so that biometric verification can be performed.

An example of interoperability is the capacity of all ePassports issued by eachcountry to be readable by the readers placed at borders. Recently a passport- passport readers interoperability test was conducted in Japan on 8-10 March2005 [29]. The main objective of the test was“to ensure that all passports can beread at all borders points, regardless of who has manufactured the passports andthe readers”, accordingly to the ICAO specifications and requirements. 16 readervendors and 30 passport vendors (providing about 100 different passports) haveparticipated to these tests. The tests demonstrated that on average, 82.9% ofall passports could be read, while each reader could read 79.5% of all passports.Furthermore, Java-based solutions needed the slowest time to read the datacontained on the chip in a passive authentication mode: 2 seconds with a 20Kbpicture on the chip. If Basic Access Control is used besides, the reading timeincreased up to 20 seconds.

In November 2005, another passport interoperability test was conducted inSingapore 1. The organisers had 140 ePassports specimens and 45 readers forthe test, but only the results from 95 ePassports and 29 readers were published.22 readers out of 29 had an interoperability percentage higher than 90% andonly 1 reader out of the 7 remaining had an interoperability percentage below79% (about 55%). At the end of May 2006, another international passportinteroperability test will be conducted and will focus principally on cross overtests and conformity tests 2.

Another example of interoperability is the efficiency of biometric system towork with interchange formats. The MIT (Minutiae Template Interoperabil-ity Testing) project 3 will test and impove the interoperability of fingerprint

1See http://www.securitydocumentworld.com for more information.2See http://wg8.de/interoptest-berlin for more information.3See http://www.MITproject.com for more information.

36

Page 42: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

biometrics. The main objectives of the MIT project are:

- define criteria for interoperability testing;

- develop a database of fingerprint images to enable testing;

- develop a test bed enabling the automated and repeatable testing of fin-gerprint minutiae interoperability, and also investigation of how factorssuch as image quality are important for interoperability;

- incorporate an improvement step, whereby the interoperability of thetested systems can be improve;

- provide for testing if interoperability of future systems from further ven-dors.

3.1 BioAPI

The BioAPI specification version 1.1 [10] intends to promote interoperabilitybetween vendors. The BioAPI 1.1 specification defines two APIs: an Appli-cation Programming Interface (API), which exposes framework functionality tothe application, and a Service Provider Interface (SPI), which exposes biometricfunctionality to the framework. Biometric Service Providers (BSP), essentiallyrecognition engines for a given modality also responsible for managing theiruser interface, implement SPI functionality. The API exposes three high-levelmethods: enroll, verify, and identify. These in turn rely on lower-levelprimitives: capture, which acquires the biometric signal, process, which ex-tracts the features and performs pre/post-processing, match, which comparesdata to a user model, and createTemplate, which trains user models from sup-plied data. BioAPI defines a number of function signatures in C which must beimplemented for the BSP or application to be BioAPI compliant.

More recently, the newer BioAPI specification (BioAPI v2.0) has been pub-lished [139]. It extends and generalises the BioAPI 1.1 framework. An interest-ing change is that it breaks down BSPs into sub-units called Biometric FunctionProviders (BFPs). These are

• Sensor BFPs, which manage sensors

• Archive BFPs, which manage access to template databases (whether storedlocally on a smartcard-type device or remotely on a database).

• Processing-algorithm BFPs, which can perform pre-processing and featureextraction

• Matching-algorithm BFPs, which actually perform the match and returna score

To these functional level correspond “unit” levels, which replace the conceptof “device” found in BioAPI 1.1. Thus, it is now possible for a BioAPI 2.0 BSPto use several sensors (sensor units).

As much of the SC37 standardisation efforts (see section 3.4) are based onBioAPI 2.0, it is likely that BioAPI 1.1 applications and components will migratetowards BioAPI 2.0

37

Page 43: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

3.2 CBEFF

The Common Biometric Exchange File Format (CBEFF) [219] is a standard toexchange biometric data between systems and organisations. CBEFF does notprovide for a standard template format, but merely for a storage format, andthus is not meant to address template sharing between applications. The BioAPIBiometric Information Record (BIR) conforms to the CBEFF specification.

This format specifies that biometric data must be encoded in three parts:the Standard Biometric Header (SBH), the Biometric Specific Memory Block(BSMB) which contains the biometric data payload, and the Signature Block,which is typically a hash of BSMB and parts of the header.

The SBH contains information such as encryption flags (if the payload isencrypted), CBEFF or patron version number, BSMB format owner and type,data length, and so on for each modality available to the application.

The BSMB is a memory block whose format can be defined by the formatowner. It may contain raw (signal), intermediate (features or pre-processeddata), or templates, to be used for enrollment or matching. The BSMB mayalso contain non-biometric data.

3.3 ANSI X9.84

The ANSI X9.84 standard [11], Biometric Information Management and Secu-rity for the Financial Services Industry, describes the security features neededto implement biometric verification for financial services. The Core SecurityRequirements are summarised here:

1. The integrity of biometric data and verification results must be guaranteedbetween any 2 components using software techniques such as hashes andphysical measures such as tamper-resistent assemblies.

2. The source and receiver of biometric data and verification results must beauthenticated, again using software or physical methods where appropri-ate.

3. The confidentiality of biometric data may be ensured between any 2 com-ponents.

X9.84 also describes secure enrollment, verification, storage, transmission,and termination procedures.

3.4 ISO/JTC1/SC37

The International Organization for Standardization (ISO) and the InternationalElectrotechnical Commission (IEC), created in early 1980’s the Joint TechnicalCommittee One (JTC1), which has several active subcommittees, amongst otherthree subcommittees interesting for the biometric area 4. The SC17, for Cardsand Personal Identification, the SC27 for IT Security Techniques and the SC37for Biometrics. All these committees have working groups, and for instance the

4JTC1 homepage at http://www.jtc1.org.

38

Page 44: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

six SC37 WGs are in the areas of harmonized biometric vocabulary (WG 1),biometric technical interfaces (WG 2), biometric data interchange formats (WG3), biometric functional architecture and related profiles (WG 4), biometrictesting and reporting (WG 5) and cross-jurisdictional and societal aspects (WG6).

The biometric data interchange formats, concerning fingerprint, face and irisimages, proposed by the ISO/IEC JTC1/SC37, are presented in this section.The parts 2, 4, 5 and 6 are already approved to date [30].

Part 1: Framework This part is still not available at the time of the publi-cation of this report.

Part 2: Fingerprint Minutiae Data The ISO/IEC SC37 working draft onfingerprint minutiae data specifies the main requirements about the extraction ofminutiae features from fingerprint images and defines the data format for storageand transport and for a card-based use [134]. The minutiae are specified by alocation (horizontal x and vertical y positions, with the origin at the upper leftcorner of the image), a direction (angle between the tangent and the horizontalline, starting on the right and going counter-clockwise) and a type (ridge ending,ridge bifurcation and other). The placement of the minutiae is function either ofwhere the three legs of the thinned valley (ridge ending) or of the thinned ridge(ridge bifurcation) intersect, or of the ridge skeleton end or bifurcation point.The latter will be used in the record formats, while the card formats will use oneof the two localizations. For the matching process, the type ”other” can matchwith all the types and the ”ridge ending” and ”ridge bifurcation” type can matcheither with itself, or with the type ”other”. All these features are intended to beembedded in a CBEFF -compliant structure in the Biometric Data Block.

Part 3: Fingerprint Pattern Spectral Data The ISO/IEC SC37 workingdraft on fingerprint pattern spectral data specifies the main requirements aboutthe exchange of local or global spectral features from fingerprint images andis characterized by two main steps [135]. Re-sampling of the data to a lowerresolution, and dividing a portion of the image into a grid of (non-)overlappingcells for creating the fingerprint pattern spectral interchange data. Each cell canbe decomposed into a 2D spectral representation (Discrete Fourier Transform)with complex spectral components characterized by a wavelength in the x andy direction, an amplitude and a phase. All these features are intended to beembedded in a CBEFF -compliant structure in the Biometric Data Block.

Part 4: Fingerprint Image Data The ISO/IEC SC37 working draft onfingerprint image data specifies the main requirements about the exchange offingerprint images within a CBEFF data structure [136]. The scanner used toacquire to fingerprint image should have a minimum resolution of 500 dpi, witheach pixel gray level quantized to 8 bits. Latent print scanners should have aminimum resolution of 1000 dpi. Such images with high resolution should becompressed with the JPEG 2000 standard. The signal-to-noise ratio ought tobe equal to or greater than 125. The dynamic gray-scale range of image dataought to 200 gray levels for at least 80% of the captured image, and 128 graylevels for at least 99% of the captured image. The fingerprint center should be

39

Page 45: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

located approximately in the center of the image capture area. For multi-fingercapture, half of the fingers should be located to the left, while the other halfshould be to the right of the image center.

Part 5: Face Image Data The ISO/IEC SC37 working draft on face imagedata specifies the main requirements about a face image format for recognitionapplications [137]. It specifies the record format within a CBEFF data structure,scene constraints, photographic properties and digital attributes of the facialimages. Indeed, the pose angles with respect to the frontal view of the subjectas well as the geometric characteristics of the full frontal image are defined. Theminimum image width should be 240 pixels, which corresponds to an imageheight of 320 pixels. The distance between the two eyes’ centers should beabout 120 pixels. The image minimum resolution should be about 300 dpi. Thestandard passport photo size (width x height) is 416 x 536 pixels at 300 dpi,and the average size of the uncompressed image is about 669kB (about 71kBafter compression with JPEG standards). Some guidelines for travel documents’facial images are also presented: subject looking directly at the camera withopened eyes, appropriate brightness and contrast for the image without anyshadows or flash reflections, a light-coloured background, image printed at ahigh quality resolution, face taking up 70-80 % of the photograph, with thetop of the shoulders visible, showing clearly the eyes with no flash reflection ifwearing glasses, without covering the face and wearing hats or caps, and haveneutral expression. This working draft contains also data areas for 3-dimensionalinformation, for example x, y and z coordinates of some feature points, such aseye location and nose length and position. But an amendment was submittedto the SC37/WG3 in order to include, additionally: to these feature points, the3D image information and the range data in the CBEFF Header. These datawill be stored using a canonical or cylindrical coordinate system.

Part 6: Iris Image Data The ISO/IEC SC37 working draft on iris imagedata specifies some requirements about the exchange of iris images and proposestwo interchange formats, a raw data (compressed or not) and a polar image,intended to be embedded in a CBEFF-compliant structure in the CBEFF Bio-metric Block [138]. Each pixel of the raw image shall be represented by 256 graylevels (8 bits). During the acquisition, near-infrared wavelengths (700-900 nm)should be use for the illumination. The minimum requirement for the imageresolution is that the iris diameter is between 100 and 149 pixels, but for highquality images, this diameter should be about 200 pixels or more. For the con-trast, a minimum of 90 gray levels should separate the iris and the sclera, while aminimum of 50 gray levels should separate the iris and pupil. Furthermore, 70%of the iris should be visible on the image and at least 70 pixels should separatethe vertical and horizontal edges of the image and the iris. The signal-to-noiseratio should be equal to or greater than 40 dB. For the lossless compressionand the compressed formats, they should be in accordance respectively with theJPEG-LS and JPEG compression standards. Some pre-processing steps can beapplied to the image, such as rectilinear image pre-processing (image rotationangle and uncertainty) and polar image pre-processing (boundary extraction,iris occlusions, scan type, orientation correction and polar conversion). Somerequirements are also indicated for the presentation of the iris during the ac-

40

Page 46: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

quisition process: vertical positioning of the head, opening the eyes as wide aspossible, presenting an iris with a pupil size of 7 mm or less, removing eyeglassesand contact lenses.

Part 7: Signature/Sign Behavioural Data] This part is still not availableat the time of the publication of this report.

Part 8: Finger Pattern Skeletal Data This part is still not available atthe time of the publication of this report.

3.5 ICAO

The International Civil Aviation Organization (ICAO) and the InternationalOrganization of Standardization (ISO) joined their competences for publishingthe Doc 9303 specifications in three separate parts with common structure [124]:Part 1 for Passports, Part 2 for Visas and Part 3 for Official Travel Documents(Cards). These parts are structured as follows:

- The first two sections contain an introduction about the organisations ICAOand ISO, some definitions and references.

- The third section contains technical specifications for all MRTDs, as physicalrequirements, security safeguards and a standardized layout.

- Specifications unique to the MRTD under discussion.

- Specifications for additional size variant to the MRTD under discussion (onlyfor Part 2 and Part 3).

The ICAO made some recommendations about the deployment of biomet-rics in machine readable travel documents [125]. Here are presented the mostimportant requirements regarding biometrics 5:

- Incorporation of an “optimally-compressed” facial image, and additionally,according to the will of states, a fingerprint and/or an iris image. The mainadvantages observed by the ICAO for the use of facial recognition in thispurpose, is that this approach is socially and culturally accepted, alreadyused for identity documents issuance and control, and non-intrusive.

- The minimum storage sizes per image in the Logical Data Structure (LDS)are about 12 to 20 kB for the facial image, about 10 kB for the fingerprintimage and about 30 kB for the iris image.

3.6 Wavelet Scalar Quantization

The FBI proposed the Wavelet Scalar Quantization (WSQ) image compressionalgorithm as a way to standardize the digitization and compression of gray-scalefingerprint images [44]. To archive the large FBI fingerprint database (more than40 million fingerprint cards!), an efficient compression algorithm was required

5Section 11.1 presents the main technical specifications for biometric Identity Documents.

41

Page 47: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

which retained the fidelity of ridge details [121]. The fingerprint images scannedat 500 dpi (8 bits of gray-scale) are compressed with the WSQ algorithm and an“archival-quality images at compression ratios of about 15:1” are thus produced[44]. The specifications of the WSQ encoder/decoder are as follows [44] (Figure3.1):

WSQ Encoder It consists in a discrete wavelet transform (DWT) decompo-sition, a scalar quantization, and a Huffman entropy coding.

WSQ Decoder It has to be able to decode these three processes and all vari-ants of them that are allowed under the general specifications.

Figure 3.1: Overview of the WSQ algorithm [44].

The specifications for a WSQ certification complying by the FBI’s recom-mendations are as follows: the commercial vendors’ implementations have to betested and thus have to belong to a category of encoders and to a single decoderwith sufficient generality that any compressed images can be decoded [44]. Thecertification guidelines for this standardized compression algorithm are availableon-line 6.

3.7 JPEG2000

The Joint Photographic Experts Group (JPEG) 7 proposed since 1988 a popularimage compression standard, which efficiency was increased with the adoptionof the new JPEG-2000 standard, used amongst other for digital archiving appli-cations. Previously, the JPEG standard used Discrete Cosine Transforms, while

6Certification guidelines available at http://www.itl.nist.gov/iad/vip/fing/cert_gui.html.7JPEG homepage at http://www.jpeg.org/.

42

Page 48: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

wavelet technology are used for the new generation of compression standard,which can achieve much higher compression ratios [174]. Some features of thisnew algorithm are presented in [178]: state-of-the-art bit-rate compression per-formance; progressive transmission by quality, resolution, component, or spatiallocality; lossy and lossless compression; random access to the bitstream; pan andzoom; compressed domain processing; region of interest coding by progression;limited memory implementations.

The JPEG 2000 standard is organized in twelve parts, amongst which sixwere adopted as ISO standards [174].

Part 1 This part defines the core JPEG 2000 image coding system was adoptedas an ISO standard (royalty and license-fee free, but not patent free).

Part 2 This part defines various extensions to the base-line core system (part1).

Part 3 This part, adopted as an ISO standard, specifies a file format formotion sequences of JPEG 2000 images.

Part 4 This part, adopted as an ISO standard, tests the conformance to thepart 1, and thus the test’s procedures for the encoder and the decoder processes.

Part 5 This part, adopted as an ISO standard, consists in a reference software,which implements in C and Java the core JPEG 2000 image coding system,described in part 1.

Part 6 This part, the Compound image file format adopted as an ISO stan-dard, provides a framework for encoding compound document images.

Part 7 Abandoned.

Part 8 This part consists in security aspects, which specified standardizedtools and solutions, such as encryption, source authentication, conditional ac-cess, and ownership protection.

Part 9 This part, the interactive protocol, defines solutions for delivering im-age and metadata

Part 10 This part defines the encoding of 3D data.

Part 11 This part specifies additional tools for protection and errors detectionfor wireless multimedia applications.

Part 12 This part, the ISO Base Media File Format, is a common initiativebetween JPEG and MPEG, to create a base file format for future multimediaapplications.

43

Page 49: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 4

Legal framework, privacyand social factors

At the 26th international conference of the data protection and privacy com-missioners, the Swiss Federal Data Protection Deputy Commissioner [276] hashighlighted that

The collection and processing of biometric data must be conductedonly in accordance with the requirements of data protection regula-tions and especially with the basic principles (lawfulness, good faith,purpose-link, data security, proportionality and rights of the personsconcerned).

Here, we summarise these basic principles as per [276]. In the private sector,biometric data can in principle only be used with the approval of the personconcerned, and the approval must be free, specific, and informed (lawfulness).

The process of collection and processing of biometric data should be trans-parent, and not happen without the knowledge of the subject (good faith prin-ciple).

The purpose-link principle states that if a less privacy-invasive techniquesuch as verification instead of identification can achieve the stated goal (e.g.access control), it should be used.

The proportionality principle means that personal data can only be collectedif they are necessary with respect to the purpose for which they should be col-lected and processed. As such, the question should be asked whether the desiredgoal could not be achieved without using personal data. Applied to biometrics,this means that putting in place an identification system is not necessary wherea verification system is sufficient, and that anonymisation and encryption meth-ods that would allow authentication without identification should be preferred.The proportionality principle is applied differently according to the country,and each country’s data protection commissioner is consulted on a case-by-casebasis.

According to the data security principle, the security of biometric systemsdata is essential. In case of “identity theft”, the victim will be hard put todemonstrate that she has not commited the wrongdoings perpetrated by the

44

Page 50: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

usurper. Thus, security measures should be put in place starting with the dataacquisition process.

4.1 Legal framework

As mentioned in Section 2.2.1, no legal definition of biometrics exists at themoment, neither in Europe, nor at the international level [110]. Even if securitytakes more and more importance, the right for the private life and the respectof the human body are in force in most countries. This right for private lifemeans also the protection of any sensitive personal data, as it is the case withbiometrics. Legal issues for some countries will be presented in this part, be-ginning with Switzerland, the European Community and then with the USA,which has influenced most the international legal framework since 2001.

4.1.1 Switzerland

In Switzerland, even if there is no special law on biometrics, the Swiss FederalConstitution 1 protects the right to privacy and against the misuse of personaldata (Art. 13). The Swiss Federal Data Protection Act 2 has for main objec-tive to protect the personality and the fundamental rights of those individualsabout whom undergoes a data processing. The Swiss Federal Data ProtectionCommissioner 3 has warned (in his annual reports, data protection act and in-formation sheets and guides) about the introduction of biometric systems. Heargued that the use of such systems has to respect amongst other things theproportionality and decisiveness rules. In 2002 4, biometric information wasalready considered as personal sensitive data, and the use of biometric systemought only to be accepted and approved when technical security problems re-lated to its introduction were resolved. In his last annual report 5, the Swissfederal data protection commissioner has mentioned that the use of biometricshas to respect some principles accordingly to data protection rules, in order toavoid risks about liberties and fondamental rights [228]. These considerationsare as follows:

- Biometrics can be used if no other less intrusive mean is available.

- Biometrics can be used if the main objective is the protection and security ofdata.

- The finality of the processing has to be respected.

- Clear information has to be given to the concerned persons.

- The collection process of the biometric data should be explained to the con-cerned persons (overt collection).

1Swiss Federal Constitution available online at http://www.admin.ch/ch/f/rs/1/101.fr.pdf.2Swiss Federal Data Protection Act available online at

http://www.admin.ch/ch/f/rs/c235_1.html.3Swiss Federal Data Protection Commissioner homepage: http://www.edsb.ch/.4Press release at http://www.edsb.ch/d/doku/pressemitteilungen/2002/2002-07-01c.htm.5The 2004/2005 annual report of the swiss federal data protection commissioner is available

online at http://www.edsb.ch/e/doku/jahresberichte/2005/index.htm.

45

Page 51: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

- Alternative choices have to be planed for people who have not the possibilityto use a particular biometric.

- The identification of biometric data should only be conducted by comparingdata acquired directly from the concerned person.

- The original biometric data have to be destructed after the enrollment proce-dure.

- Methods using biometric templates, instead of raw data, stored in decen-tralised databases should be privileged.

- A centralised database of biometric data, which can also be found outsidethe system as traces left by individuals, such as fingerprints, can only becreated in the case of a “dominating interest” (i.e. security).

- Without such an interest, other modalities, such as hand geometry, have tobe chosen.

- Adequate measures, such as encryption of data or the use of templates, haveto be taken in order to prevent any diversion of the finality if biometricdata that can be found as traces are stored in a centralised database.

- Biometric data should not be used as a unique universal identifier.

- Information about the health of the person should not be gathered from thebiometric data.

- Anonymous authentication should be privileged in order to avoid the revealingof the person’s identity.

- In a verification mode, the biometric data should not be used for other pur-poses, except if the legal framework has allowed this finality.

- A biometric system should be securised with additional identification / veri-fication means, such as an access code.

- Starting from the enrollment, the biometric data have to be encrypted, as wellas the communication in the network.

- A periodical re-enrollment is necessary in order to verify the reliability of thestored biometric data, and to avoid the ageing of the biometric informa-tion.

- The persons ought to have the possibility to control how their biometric dataare used.

- Certification and audit procedures should be conducted with biometric sys-tems. Furthermore, the risks related to their use have to be evaluatedbefore the introduction of such systems.

46

Page 52: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

4.1.2 European Community - Council of Europe

In 1981 the Convention 108 for the protection of individuals with regard toautomatic processing of personal data [72] 6, was adopted by the Council of Eu-rope. Its main objective is to protect personal data in general. The EuropeanParliament and the Council of Europe proposed the Data Protection Directive95/46/EC 7, which has for main objectives the protection of individuals withregard to the processing of personal data and the free movement of such data.The core values of this directive are as follows [7]: “reduction of the processingof personal data to the unavoidable extent, maintain the highest transparencypossible, institutional and individual control of processing of personal data as ef-ficient as possible”. The G29, a consultative group of the European Communitycomposed by representatives of national authorities of control, has warned aboutthe danger of the conservation of biometric information in databases [108] 8.

4.1.3 United States of America

Since 2001, the USA have thought about ways to introduce legislation to en-hance control of citizens and protect their territory. Three important laws havethus been introduced. The US Patriot Act 9 has for main objectives “to deterand punish terrorist acts in the United States and around the world, to en-hance law enforcement investigatory tools, and for other purposes”. The USPublic Law 107-71 (the Aviation and Transportation Security Act) 10 has formain objective the use of emergent technology in aviation security, such as theaccess control system for airports employees. The US Public Law 107-173 (theEnhanced Border Security and Visa Entry Reform Act of 2002) 11 has for mainobjective the enhancement of the border security of the United States, such asthe introduction of biometric information in travel documents. In the recent USNational Intelligence Reform Act of 2004 (US Public Law 108-458 / S.2845 ) 12,the use of biometric technology is mentioned as a way to increase the securityof the United States (e.g. the use of a biometric entry and exit data system forverifying the identity of passengers in airports and for collecting the biometricexit data, the development of an integrated biometric screening system, the useof biometrics to improve the security of travel documents and pilot licenses, theestablishment of competitive centers of excellence at the national biometric labo-ratories and the promotion of research and development of biometric technologyapplications to aviation security). The US-VISIT program 13 belongs to thesesecurity measures for enhancing the control at US borders, and has for mainobjective to verify the identity of visitors with visas by collecting amongst otherbiometric information: the two index fingers and a photography of the face. The

6Available online at http://conventions.coe.int/Treaty/EN/Treaties/Html/108.htm.7Data Protection Directive 95/46/EC available online at

http://www.dataprivacy.ie/6aii.htm.8Available online at http://europa.eu.int/comm/justice_home/fsj/privacy/workinggroup/index_en.htm.9US Patriot Act available online at http://www.epic.org/privacy/terrorism/hr3162.html.

10Aviation and Transportation Security Act available online athttp://www.access.gpo.gov/nara/publaw/107publ.html.

11Enhanced Border Security and Visa Entry Reform Act of 2002 available online athttp://www.access.gpo.gov/nara/publaw/107publ.html.

12National Intelligence Reform Act of 2004 available online athttp://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=108_cong_bills&docid=f:s2845pp.txt.pdf.

13US-VISIT homepage at http://www.dhs.gov/dhspublic/display?theme=91&content=3768.

47

Page 53: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

fingerprints collected are compared against a watchlist, against a database ofindividuals already enrolled by the program, and against the biometric informa-tion contained on the identity documents for verification purposes. The US RealID Act, approved by the house of representatives in the beginning of February2005, has for main objective to“establish and rapidely implement regulations forState driver’s license and identification document security standards” 14. Thisact will impose the use of anticounterfeiting features, machine-readable tech-nology for all driver’s licenses and other identity cards. Furthermore, this actrequires that States’ Department of Motor Vehicles databases should be linkedand all the information shared with the federal authority.

It is important to highlight that the United States has no data protectionlaws, only codes of conducts or recommendations, no data protection commis-sioner and that the term of privacy doesn’t exist in the U.S. Constitution [290].

4.1.4 France

In France, as there is no legal definition of biometrics, only the French DataProtection Board CNIL 15 can allow the use of biometric information, accord-ingly to the French Data Protection Act 16. Several guidelines regarding the useof biometric technology for protecting privacy was proposed by the CNIL, suchas using preferably a decentralized database and the respect of proportionalityand decisiveness rules. In the future, the CNIL will evaluate, within the INESproject (creation of a new biometric identity card called CNIE 17), the problemsgenerated by a centralized database for identity documents and strike an ap-propriate balance between the risks and the advantages of such an application.The CNIL has given its approval for the French Biometric Passport.

4.1.5 Germany

In Germany, the Federal Data Protection Act 18 has for main objective to protectthe individual against the impairment of his/her right to privacy through thehandling of personal data. Indeed, the right to informational self-determinationwhich is “the power of the individuals to basically decide for themselves whethertheir personal data should be divulged and used” [7], is at the base of dataprotection in Germany. The Federal Data Protection Commissioner ensuresthat the Data Protection Act is implemented. Furthermore, as all Europeancountries, Germany has to implement the Data Protection Directive 95/46/ECin federal law. The German Federal Office for Information Security (BSI) 19

also plays a role in the domain of data protection: this federal institution hasthe responsability to“support the federal data protection commissioner in issuesrelating to the technical implementation of statutory requirements” [7].

14Real ID Act available online at http://thomas.loc.gov/cgi-bin/query/z?c109:H.R.418:.15CNIL homepage: http://www.cnil.fr.16French Data Protection Act available online at http://www.cnil.fr/index.php?id=301.17Chapter 11)presents more information about the INES Program.18Federal Data Protection Act available online at http://www.bfdi.bund.de.19Bundesamt fur Sicherheit in der Informationstechnik ’s home page: http://bsi.bund.de.

48

Page 54: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

4.2 Biometrics as sensitive personal data

A study in the Netherlands concluded that biometric information must be con-sidered as personal data, if the template is associated to other personal data[273] (which is most likely to be the case). Furthermore, if racial and religiousinformation can be obtained from the collected data, such as facial images, thesemust also be considered as sensitive. Canada’s standing committee in citizen-ship and immigration reports likewise that biometric data is information of theperson, rather than information about the person [96]. The swiss federal dataprotection act specifies in article 3 that any information relating to an identifiedor identifiable person is considered as personal data and that any data relatingto religious, health or racial origin is considered as sensitive data. Raw biomet-ric data is considered as sensitive personal data according to these definitions asit can always be linked to the source and can include sensitive elements [197].

It should also be noted that raw biometric data, and in some cases biometricfeatures, may contain medical information. For example, fingerprints can allowto some degree inference about ethnicity, chromosomal aberrations [195, 274],and mental disorders (skin and brain develop from the same ectoderm duringgestation) [272], and retina scans can reveal information concerning diseasessuch as diabetes, brain tumors, or AIDS [35].

A recent draft report of the Council of Europe considers that biometrical dataconstitute a specific category of personal data [73] 20. Accordingly, these datahave to be used only for ”specific, explicit and legitimate” purposes, and involveadequate protection measures. The appropriate architecture of the biometricsystem has then to be chosen depending on its purpos. This report recommendsthat the complete information about the purpose of the system should be madeavailable to the person whose biometric will be enrolled. This person shouldalso be entitled to access and rectify data set upon request. Accordingly tothe International Conference of Data Protection Commissioners cited in [227],a complete transparency of the biometric system is necessary, especially if thesystem is designed for a large-scale use.

4.3 Privacy

The term of privacy , defined as ”the state or quality of being private”[1], appearsalso largely in the literature in relation to biometrics. Privacy is ”the ability tolead your life free of intrusions, to main autonomous, and to control access toyour personal information”[223]. This term also includes an aspect of autonomy,as well as ”a control over the quantity of information abroad” [290]. An elegantdefinition of privacy is provided in [9]:“[...] privacy is control over how and whenwe are represented to others”.

The International Biometric Group 21 subdivides privacy risks in two gen-eral categories [130]. The personal privacy concerns people who find the useof biometrics as invasive and inherently offensive. The informational privacyconcerns the impact of a misuse (unauthorized collection, use, retention anddisclosure) of biometric data.

20As best practices, the BioVision project recommends also “to always treat biometric dataas personl data” [6].

21IBG homepage at http://www.biometricgroup.com.

49

Page 55: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

The relation between a biometric system and privacy can be described as acontinuum according to the BioPrivacy framework [130], from privacy enhancingto privacy intrusive:

Privacy-Protective A biometric system is privacy-protective if it is used toprotect or restrict the access to personal information or if this system is a meanfor establishing a trusted identity.

Privacy-Sympathetic A biometric system is privacy-sympathetic if it is usedin a limited way, or if the system is designed to ensure the protection fromunauthorized access and usage.

Privacy-Neutral A biometric system is privacy-neutral if its use has lowprivacy impact.

Privacy-Invasive A biometric system is privacy-invasive if it is used withoutknowledge or consent, or if it is used for undisclosed purposes.

Regarding biometrics, using a part of a human being (“something you are”)during a transaction is considering as “giving information about himself!” [290].Additionally, because biometric technology is recent, people may have of a neg-ative a priori for its use and may thus be reticent to the introduction of suchsystems. The implications for privacy, every time biometrics is used, is a veryimportant element that people want to know about [280].

The inlet of new technologies has always required setting-up new laws andpolicies [290]. Indeed, each time a new technology emerges, the legal frameworkbecomes outdated and has to be rethought according to this new reality or to setsocietal acceptable boundaries for its usage. It is exactly what is happening nowwith the introduction of biometrics in daily activities and in its implementationin identity documents. Legal experts have to explore from their perspectives“what is required to safeguard the public interest and to ensure optimal resultsfor society”, without excluding scientific experts from this analysis [290].

As with all emerging technologies, there is potential for abuse [240]. Themain danger is that the information transmitted with or without permission ismisused (in respect to the use allowed for a defined application). The mainconcerns regarding biometrics are as follows [240]:

- Information gathered without permission.

- Information used for other purposes.

- Information disseminated without permission.

- Creation of a complete picture about people.

- Real-time surveillance and profiling.

These concerns are legitimately caused by the fact that the biometrics col-lected by a system can be linked to personal information or can allow to trackthe movements of a person [280] or proceed to background checks against lawenforcement databases. What is also highlighted by Wayman et al., is that this

50

Page 56: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

new technology ”can only link a person to a biometric pattern and any iden-tity data and personal attributed presented at the time of enrollment in thesystem” and that an authentication’s anonymity can be guaranteed when nolink is possible between the biometrics and the personal information. Indeed,if no other personal information is stored during the enrollment process at thesame time as the biometrics, this latter cannot be linked to the concerned per-son. However, this latter scenario is very unlikely. The Convention 108 of theCouncil of Europe also considers that biometrics linked with any informationare personal data [72]. Figure 4.1 highlights some differences between biometricand non-biometric identifiers [280]. With respect to this figure, we should pointout the following: 1) Point 1 depends entirely on your definition of “personalinformation”. As mentioned in section 4.2, a convincing argument can be madethat biometric data is in fact personal information. Additionally, some modal-ities like face are extremely easy to steal covertly (just use a digital camera).2) Point 3 is correct stricto sensu, but no real system, save those based onlyon Match On Card, are likely to be used without being linked to a database ofidentities. To assume otherwise is naıve.

Figure 4.1: Biometric and non-biometric identifiers [280].

All the actors involved in the biometric world agree that its use shouldenhance the security and the efficiency in any transactions [240, 290]. As statedin Section 2.1.1, an appropriate and intelligent use of biometrics is the mostsecure way to prove one’s identity. Regarding this enhancement of security byusing biometrics, we can wonder if it has to be made to the detriment of people’sliberty and if we should trade our liberty for security [40]. The use of biometricsystem requires a proportionality rule regarding the finality [110, 223]. Indeed,the pursued end of the use of a biometric system and the dangers generatedby the constitution of databases, and thus possibilities of misuses, has to beproportional [110]. The scale of deployment of any biometric system determinesthe relationship between biometrics and privacy [130].

Several methodologies have been developed to gauge the impact on privacy

51

Page 57: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

of computer systems deployment. A generic one that can be applied to thecase of biometrics for identity documents is proposed by [65] and others underthe name “privacy impact assessment”. It proposes 18 categories of questions(data retention, accountability, consent...) which need to be asked when puttingforward a proposal for a new system.

The International Biometric Group developed a tool to evaluate the impactof a deployment of a specific biometric system, in a specific environment [130].Ten questions are proposed to find out the appropriate precautions and protec-tions that are applicable in a particular situation. Each question describes theprivacy risks involved.

1. Is the system deployed overtly or covertly?

2. Is the system optional or mandatory?

3. Is the system used for verification or identification?

4. Is the system deployed for a fixed period of time, or it is indefinite?

5. Is the system deployed in the private or the public sector?

6. In what capacity is the user interacting with the system, as individ-ual/customer or as employee/citizen?

7. Who owns the biometric information, the enrollee or the institution?

8. Where is the biometric data stored, personal storage or database storage?

9. What type of biometric technology is being deployed, behavioral or phys-iological information?

10. Does the system utilize biometric templates, biometric images, or both?

For example, an overt match-on-card verification system has lower privacyrisks than a covert identification system, with a centralized database. Even ifall the risks involved are not defined with these ten questions, a privacy riskstatement can be correctly postulated, for evaluating the potential of misuse ofany system.

4.4 Privacy protection

The fears about the use of biometric systems are legitimate, but all excessiveopinions are counterproductive. Indeed, those who claim that any use of bio-metrics is privacy restrictive, as well as those who claim that biometrics has tobe used in any transactions, block the discussions, and thus these opinions arenot appropriate [240].

The right equilibrium has to be found, thanks to the proportionality rule.The 25th International Conference of Data Protection Commissioners 2003 22

has adopted five resolutions for protecting people’s privacy and enhancing dataprotection some of which apply to the biometric identity document case:

22The 25th International Conference of Data Protection Commissioners homepage:http://www.privacyconference2003.org.

52

Page 58: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

1. Resolution on improving the communication of data protection and pri-vacy information practices.

2. Resolution concerning the Transfer of Passengers’ Data.

3. Resolution on Data Protection and International Organisations.

4. Proposed resolution on Automatic Software Updates.

5. Resolution on Radiofrequency Identification.

In Switzerland, the Federal Data Protection Commissioner has not yet takenposition on the issue of biometric for identity documents, but is expected to do soin the coming year. Trials of biometric technologies in the Zurich County (Air-port) should result in recommendations being made public from either countyor federal data protection offices.

The Swiss Federal Data Protection Commissioner has however taken positionon the issue of fingerprint for employees’ time clocks [228]. The use of biometricdata for time clocks, stored on personalised smart cards is in adequacy with theswiss legal framework on data protection. Furthermore, a centralised databasewith only minutiae information, related to the identity of the person, can alsobe in adequacy with the swiss legal framework, upon condition that securitymeasures are applied, as specified in the Swiss Federal Data Protection Act 23.

In the literature, some solutions are presented to protect the privacy whenbiometric systems are used. Basic principles are needed to enhance both libertyand security [240]: the environment should be overt instead of covert; the systemshould use a verification process instead of a identification process; the systemshould operate with local storage instead of a centralize database; the systemshould be “opt in” instead of mandatory; the matching process should be basedon a template instead of a stored image; the pseudonymity should be guaranteed;the system should be well protected against misuse; a routine secondary reviewshould be conducted; and a suitable secondary identification system should bepresent if the primary does not work. Rosenzweig et al. also postulated that bio-metrics could not be an absolute security measure: “the system cannot be seenas the ultimate security tool, and thus the perfect solution; its simply anothertool in a layered approach to security”. Ratha et al. have proposed another wayto protect privacy in biometric environment [230]. The biometric signal shouldbe modified, in “applying repeatable noninvertible distortions to the biometricsignal”, for avoiding the problem of compromised biometrics. For Prabhakaret al., the solution for the enhancement of the privacy is as follow [223]: theapplication should use a decentralized recognition process and the match-on-cardprocess could be an appropriate solution when fingerprint technology is used.In agreement with ethical considerations, some statements concerning privacycan be made [40]: the privacy of information entrusted should be protected bythe developers of the system; some information about the systems and theiruses should be transmitted to the public and the technology should be used ina socially responsible way, and thus minimize the possibility of misuse.

In adopting such protections, the enhancement of privacy and security canbe guaranteed. ”An accountable and responsible use of biometric systems can

23Section 4.1 presents more information about the Swiss legal framework and Data Protec-tion Act.

53

Page 59: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

in fact protect individual privacy” [223].

An example of a privacy protection policy can be found in the recommenda-tion of the Ontario information and privacy commisioner for the Toronto welfareentitlement system, which uses fingerprints [54]:

• requiring the biometric, in this case, the finger scan, to be encrypted;

• restricting the use of the encrypted finger scan only to authentication ofeligibility, thereby ensuring that it is not used as an instrument of socialcontrol or surveillance;

• ensuring that an identifiable fingerprint cannot be reconstructed from anencrypted finger scan stored in the database;

• ensuring that a latent fingerprint (i.e., picked up from a crime scene),cannot be matched to an encrypted finger scan stored in a database;

• ensuring that an encrypted finger scan cannot itself be used to serve as aunique identifier;

• ensuring that an encrypted finger scan alone cannot be used to identifyan individual (i.e., in the same manner as a fingerprint can be used);

• ensuring that strict controls are in place as to who may access the biometricinformation and for what purpose;

• requiring the production of a warrant or court order prior to grantingaccess to external agencies such as the police or government departments;

• ensuring that any benefits data (i.e., personal information such as historyof payments made, etc.) are stored separately from personal identifierssuch as name, date of birth, etc.

In the field of biometric identity documents, important means have to be in-vested by governments in order to protect the data acquired during the issuanceprocess, especially with the biometric data. Indeed, an independent institution,such as the CNIL (the French Data Protection Board) in France, should bemandated by each government to permanently control the data collection andthe access to these information.

4.5 Public perception of biometrics

The introduction of biometrics on the scale of a country, let alone internationally,is not without potential impact on society.

Face recognition is applicable both to identification and verification. Sensors(digital cameras) are readily available that can acquire high quality face imagesat a great distance, without the subject’s knowledge or consent. This makesthis modality a good choice for covert surveillance operations. Country-widebiometric enrollment for identity documents will provide the responsible agency

54

Page 60: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

with a very large biometric database of faces, the use of which must be controlledwith the utmost care24.

Ideally, the criteria for being put on and off a watchlist, places of legal de-ployment for biometric surveillance, and other issues will be debated in Federalor County Parliaments and clearly set in law. By and large, this has not hap-pened in Switzerland (see Section 4) and there is little public awareness of theissues at hand. A mainstream Swiss magazine (l’Hebdo) has recently deploredthis state of affairs and warned about generalised surveillance [14].

In the USA, commentators from all sides of the political spectrum (from theCato Institute [76] to the American Civil Liberties Union [261]) have warnedabout possible mission creep of face recognition technologies and the need fortransparency in the matter. The example in Tampa, Florida [261] is particularlyillustrative of what might happen when no clear policy is set on usage of facerecognition, as that city’s own Police Department decided to include not onlywanted people on the watchlist, but also those from which“valuable intelligence”can be gathered, and people “based upon [their] prior criminal activity andrecord”. The reader is further referred to [5] for a classic article on the societalimplications of face recognition in public places.

Fingerprint sensors imply touching a surface which has been touched beforeby other users. Depending on their culture, some users may not feel comfortablesharing germs with other previous users. This may be a problem, for example,for Japanese people, which are accustomed to greetings involving not a hand-shake but bowing.

24In this respect, the post-9/11 blurring of provinces between intelligence and police forcesin the USA can be seen as cause for concern in terms of privacy [51].

55

Page 61: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 5

Modality: Face

5.1 Introduction

The problem of face recognition can be defined as matching a scene image orsequence of scene images (video) with a stored template of the face. In thissection, we focus on face recognition on single scene images. In face verificationliterature, training data is often called gallery data, and testing data is calledprobe data. Two important pre-processing tasks in automated face recognitionare

face segmentation or detection which refers to the action of approximatelyextracting the position of the face out of the scene image, and

illumination normalization which is used to correct for differences in illu-mination conditions between enrollment and deployment conditions. A surveyis presented in [253].

The ICAO technical report on machine readable travel documents [125] pro-poses that 2D face be the main modality for identity documents:

[. . . ] face is the biometric most suited to the practicalities of traveldocument issuance, with fingerprint and/or iris available for choiceby States for inclusion as complementary biometric technologies.

In opposition to 2D face recognition using in most cases normal intensityimages, 3D face recognition consists in using the three-dimensional shape ofthe face [41] as well as the texture, acquired by one or several sensors. Theuse of additional information, as the depth and surface curvatures, can clearlyincrease the performance and the accuracy of such recognition systems. Suchan approach ”overcomes limitations due to viewpoint and lightning variations”[186]. The survey of 3D face recognition in this report is largely based on 3Dface recognition surveys [41, 248], and on the comments of Prof. Thomas Vetter(Graphics and Vision Research Group, University of Basel 1).

1Graphics and Vision Research Group homepage at http://gravis.cs.unibas.ch/.

56

Page 62: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

5.2 Overview of algorithmic approaches

Numerous algorithms exist for segmentation and recognition, which we brieflyreview below. This section is largely based on [301], to which the reader isreferred for a comprehensive review of the field of face recognition.

5.2.1 Segmentation

Segmentation is critical to successful face recognition, and inaccurately locatedfaces contribute significantly to recognition errors. Some algorithms [268] canbe used both for segmentation and recognition. A method which proposes tobuild models of faces and non-faces, then measures the distance between theinput and the distributions, and finally trains a multi-layer perceptron on thesetypical distances has been proposed in [263]. The correct detections ranged from79.9% to 96.3%. Various combinations of multiple neural networks classifiers areused in [243] to classify scene regions into face or non-face, using image windowsof different sizes. This achieved between 77.9% and 92.5% detection rates withvarying numbers of false detections on 130 frontal face test images.

5.2.2 Recognition

Zhao [301] divides face recognition algorithms into three categories: holisticmethods, which use the whole face image for recognition, feature-based methods,which use local regions such as eyes or mouth, and hybrid methods, which useboth local regions and the whole face.

Many holistic face recognition methods, as well as image analysis methods,are based on eigenspace decomposition [256]: face images are represented asvectors by concatenating the pixels of the image line-by-line. Then, an averagevector is computed that represents a mean face. Also, a difference vector iscomputed for each user to quantify the differences to the mean face. Then, thecovariance matrix of the difference vectors is computed. Finally, principal axescan be obtained by eigendecomposition of the covariance matrix. The first Neigenvectors (mostly called eigenfaces in face recognition literature) presentingthe highest eigenvalues will be retained and represent the most significant fea-tures of faces. Figure 5.1 shows an example of eigendecomposition of a faceimage. This procedure is also known as principal component analysis (PCA)decomposition. Finally, each user model is represented as a linear combination(weighted sum) of coefficients corresponding to each eigenface. It should benoted that, given the mean face and eigenfaces, these models are reversible.

In [268], faces are compared by projecting them into eigenface componentsand using an Euclidean distance measure. Results are provided for a 16-usersdatabase of 2500 images in various conditions. This technique has been thesubject of many improvements, for example [196] have proposed modeling boththe principal subspace containing the principal components and its orthogonalcomplement.

A successful feature-based method is elastic bunch graph matching [288],which tolerates deformations of the faces. This uses local features (chin, eyes,nose, etc.) represented by wavelets and computed from different face images ofthe same subject.

57

Page 63: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

20 40 60

20

40

60

Figure 5.1: First 9 eigenfaces extracted from Yale database (see below).

Hybrid methods are currently an active area of research, and aside fromstandard multiple classifier methods it is not clear how and whether to combineholistic and local features [301].

5.2.3 3D recognition

For 3D facial recognition systems, the main algorithmic approaches can be clas-sified as follows [41]. The first one is the Principal Component Analysis-basedalgorithm, where each eigenvector is a principal direction and the correspondingeigenvalue is a principal curvature 2. In [45], PCA is applied on planar models ofcurved surfaces (bending-invariant canonical forms). Such an approach has bet-ter performance than 2D eigenfaces and is invariant to significant deformationof the face, as facial expressions.

With the Extended Gaussian Images-based algorithm, a mapping surfacepoints onto a unit sphere, called the Gaussian sphere, can be obtained froma three dimensional object, such that all points have a corresponding surfacenormal direction [292]. The Extended Gaussian Image (EGI) is an alternativerepresentation, for which a weight is associated to each point on the Gaussiansphere equal to the area of the surface having the given normal. The correlationof the EGI of the convex regions or of the curvature-based segmentation iscomputed from a range image for the comparison. In [26], surface matching-and a central and lateral profiles matching-based algorithms are used for 3Dfacial recognition. In [262], Gaussian curvature is used for the detection oflandmarks in a 3D model, while [62] use point signatures to describe, moreefficiency than 3D coordinates, ”the structural neighborhood” of the landmarkson the 3D model. Surface-based approaches, such as EGI and profiles-based, use

2Section 5.2.2 presents more information about PCA.

58

Page 64: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

global methods for the recognition process [248]. Surface-based approaches, asGaussian curvature and point signature, use local methods for the recognitionprocess [248].

Some other approaches are used for 3D facial recognition, such as Haus-dorff distance matching, Iterative Closest Point (ICP) matching, Local FeatureAnalysis (LFA)-based algorithm, Dynamic-Link-Architecture-based paradigm,Hidden Markovs Models-based algorithms on depth maps and intensity images[41, 122, 248]. In [33], a morphable face model was created from 3D facial databy combinating linearly a large number of 3D face scans. This model is used forcreate a 3D face model from a 2D image and for giving the face a natural aspect(shape and texture). The commercial A4-Vision 3D facial recognition systemuse a Surface Semantic Analysis-based approach and extract local surface cur-vature characteristics, in order to calculate similarity scores [2].

The templates obtained in 3D face recognition systems usually range from1000 to 3500 bytes [203] and the one obtained with the A4 Vision system is 4092bytes [2].

5.3 Performance

The performance of face recognition systems is very dependent on the applica-tion [301], and good results in an evaluation campaign or in vendor specificationsdo not mean that these will be obtained in the field. A striking example of thisis found in a German government report [47], which tested 4 systems in realisticdeployment conditions. Quoting from the report (Phase II refers to testing inmismatched conditions):

[. . . ] All in all, two out of the four systems tested had a false rejectionrate (FRR) of 64% and 68% respectively in Phase I and 75% and73% in Phase II. The two other systems, with FRRs of 90% and98% (Phase I) and 99% and 99.7% (Phase II), hardly recognised anyof the subjects, and the weaker of these two systems was in fact sounreliable that it was only available for use on a few of the days.Recognition performance was not nearly as good as the promotionalmaterial of the system vendors would lead one to believe. [. . . ]

Failure To Enroll should also be taken into consideration. In laboratoryconditions, the same report tested 3 algorithms on databases of 5000 to 50000people. The smallest database gave a FTE of 7.6% (378 persons), 7.0% (351persons) and 0% respectively for each algorithms, while the largest databasegave FTEs of 7.6% (3783 persons), 7.3% (3672 persons), and 31.5%3 (15723persons). A US Army Research Lab report [155] gives 51% rank 1 identificationperformance for a test population of 270 users over 13 weeks.

However, in laboratory conditions face recognition algorithms can displayusable error rates, as will be shown in Section 5.8.

A recent study conducted in Norway demonstrated that face recognition maynot obtain acceptable performance in identity verification for large-scale appli-cations [159]. With a training data set of 1536 subjects and a test data sets

3This latter result was corrected after software updates but no data is available in thereport.

59

Page 65: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

of 1426 subjects (test data set II) and 10000 images from several thousands ofnational passport photos (test data set III), the authors obtained the followingresults. At a false acceptance rate of 1%, 97% of the subjects in the test data setII and 99.99% subjects of the subjects in the test data set III, generated one ormore false acceptances. At a false acceptance rate of 0.1%, the majority of thesubjects in the test test data set II did not generate any false acceptace, while92% of the subjects in test data set III generated more than one acceptances.

An improvement of 3D face recognition systems is necessary to increasetheir performances, and thus to be able to use them widely in applications [41].Indeed, this technology perform not as well as expected, while there is no reasonthat the performances are lower than 2D technologies, given that an additionalinformation, the third dimension, is used. The current limitations are classifiedin three categories: sensor, algorithms, and methodology.

Sensor The main problem in such a technology is the bad quality of theinput data used for 3D facial recognition systems. Particular efforts have tobe made in reducing frequency and severity of artifacts, in increasing depth offield, spatial and depth resolution and in reducing acquisition time. Holes andspikes, the most common artifacts, are present in face regions, even under idealillumination. Actually, the depth of field, from about 0.3 to 1 meter, is alsoa limitation relative to the need of cooperative subjects. The acquisition timeshould also be reduced, especially for structured-light systems. Furthermore, itis important to notice that such sensors were not initially developed for facialrecognition and are thus not yet mature for this recognition process.

Algorithms The main difficulty appearing during the comparison process isto make reference points correspond between the samples. Indeed, this regis-tration process is crucial for the alignment of the samples, in order to comparethem accurately. Furthermore, some state-of-the-art algorithms were not ableto be insensitive to size variations and to handle changes in facial expression.This latter is a ”major cause of performance degradation that must be addressedin the next generation of algorithms” [41].

Methodology and Datasets A limitation to evaluate the performance of3D face recognition systems is the lack of appropriate datasets. Using explicit,distinct training, validation and test sets could also improve the experimentalmethodology in this area.

In 2005, an operational evaluation of 3D and 2D facial recognition technolo-gies was performed for the Immigration & Check points Authority (Singapore)in order to “assess the effectiveness of the 3D technology and compare it to 2Dtechnology” [31]. 1018 subjects were enrolled for this trial: acquisition of a 2Ddigital photo and a structured-light image (A4Vision 3D technology). No FTEwas noted for the 3D system, even if 1.7% of the subjects were enrolled man-ually. In the verification mode, only 7.3% (837 attempts out of 4834 from 827subjects) of all the attempts failed to verification. In the identification mode,the 3D system achieved at a FAR of 0.0047%, a FRR of 0.103%, when the (rec-ommended) threshold was set at 80%, while the 2D system achieved at a FAR of

60

Page 66: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

0.12, a FRR of 9.79%, when the (recommended) threshold was set at 70%. Theresults of this trial demonstrated also that better performance can be achievedif the subjects acquire more than one attempt for the verification/identificationmodes and that the failures were mainly due to incorrect presentations of thesubjects in front of the camera.

5.4 Sensors

Any low-cost camera (“webcam”) is usable for 2D face recognition. However,best results will be obtained with cameras that have auto-focus, low-noise sen-sors, and optics which introduce minimal aberrations. As much as possible,cameras with similar optical characteristics should be used both for enrollmentand testing [47].

To obtain a 3D image of a face, in standarised conditions or not 4, threemain approaches are available: stereo-based, structured-light and laser scanner.The acquisition process can be completed by several sensors placed all aroundthe face or by a single sensor from a single view point.

Stereo-based This passive approach consists in using two object’s imagestaken by separate cameras, vertically arranged in such a way that their angleof sight varies only about 8-15 degrees [186]. These cameras are calibratedand a true metric reconstruction is obtained. An image pyramid is constructedand at each level, match reliable interest are pointed, for producing a densecorrespondence map, refined at every level. The final map is transposed intoa 3-D mesh. Approximately 9 seconds are necessary to capture such a 3Dimage, from 2D images. Currently, sufficient information about the facial’sshape cannot be extracted with such an approach [122]. When using rangeimages, called passive range acquisition, such an approach is not able to producein real time 3D range images [45].

Structured-light This active approach consists in using a projection of aknown pattern to recover 3D coordinates [25], working either in monochromelighting, or in invisible near-infrared range. The original lighting is distorted bythe object and after reconstruction, the shape of the face is obtained. Rangeimages can also be used in this approach for acquiring facial shape [3]: a se-quence of stripe patterns is projected on the object and captured by a camera.Each pixel acquired is coded and the shape is obtained by triangulation, as theparameters of the projector and the camera are known. Such an approach isalso called active range acquisition [45]. A sequential projection of a series ofblack and white stripes, called coded light can also be used and is more robustand accurate than the structured light approach [45].

Laser scanner This approach allow a cylindrical representation of the headstructure [33], with parameters as surface points sampled at 512 equally-spacedangles and at 512 equally spaces vertical steps. The RGB-colour values can also

4Examples of standardized conditions: illumination, position and orientation of thehead,. . .

61

Page 67: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

be stored in a texture map.

The commercial A4-Vision 3D facial recognition system uses the structured-light approach, in invisible near-infrared range, for acquiring the data and recon-structed the 3D surface with surface reconstruction and optimization algorithms[2]. An example of 3D models from a single person, obtained by using the A4-Vision 3D facial recognition system, is presented in Figure 5.2.

Figure 5.2: 3D Models from a single person obtained by using the A4-Vision3D facial recognition system.

5.4.1 Ergonomics and acquisition environment

For best recognition performance, as many parameters should be kept con-stant between enrollment and field conditions. These include: distance fromthe camera to the head, centering of the face in the image (ideally no part ofthe face should be cut), yaw, pitch, and roll of the head, illumination angleand source, background colour and texture. For these reasons, outdoor facerecognition is still a challenging problem [301]. Distance from the camera isalso crucial and can significantly deteriorate both identification and verificationperformance [32]. The ISO/IEC SC37 working draft [137] has strict recommen-dations about acquisition control.

In 3D recognition, the ergonomics of systems are relatively demanding, ascooperative users are needed for their location in the acquisition area, and be-cause the acquisition time lasts several seconds. The subjects should have aneutral expression and take off their glasses and hats, if these latter hide somepart of the forehead. The hairstyle and the beard can also influence the resultsof the verification/identification process, especially if the subjects have changedit between the enrollment and the recognition process. As the sensor use insome cases near-infrared wavelengths, the ambient lightning will not disturbthe acquisition process.

5.4.2 Face acquisition for identity documents

Annex A of the ICAO report [128] has a set of guideline regarding photographsfor biometric identity documents. Specifically, it mentions that

62

Page 68: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

• the face should cover about 70-80% of the photograph,

• the user must look directly at the camera,

• the skin tones should be “natural” (no specific details are given),

• the photograph should have “appropriate brightness and contrast” (nospecific details are given),

• no hair should come across the eyes, which should be open,

• the user should be placed in front of a plain, light-coloured background(no specific details are given),

• the photographs should be taken with uniform lighting (no colour temper-ature or other details are provided), so that no shadow appears,

• the user should not wear glasses that cover parts of his/her eyes,

• no head coverings are permitted except for religious reasons, and in thiscase the face should be visible from bottom of chin to top of forehead,

• the user’s expression’s must be neutral.

These broad guidelines can be complemented by the already mentioned, morespecific guidelines of the ISO/IEC SC37 working draft [137].

3D facial recognition is completed without any difficulties by human beings.The optical human system, a stereo-based ”sensor”performs robust facial recog-nition. Despite that, the technologies used in automatic 3D approaches arenot yet mature for a large-scale introduction in identity documents. Indeed,even if these methods are interesting for decreasing the problems of pose andillumination variations (as observed in 2D), the algorithms are not sufficientlyrobust when handling facial expression variations. The compatibility between3D systems in such an application is also important. Indeed, if there are differ-ences in resolution’s quality of the sensors used, the systems will also have lowperformance.

5.5 Computational resources

A typical eigenfaces-based template requires between 1000 and 3500 bytes ofmemory for storage [203]. Depending on compression, a raw face image file canrange from 100 kB to 1 kB. According to [32], compression seems not to impactidentification performance up to compression ratios of 30 to 1. Compressionratios of 7:1 (70 kB to about 11 kB) have also been investigated for colour JPEGswith less than 2% relative reduction identification rate over the uncompressedimage [127].

In testing and after feature extraction, a baseline eigenfaces-based system,which just uses dot products of face presentation weights with respect to aneigenface model’s weights, will need approximately 2n floating-point operationsper dot product, which means using for example 100-eigenfaces models willresults in 200 floating-point operations per model compared. As a guide, sup-port vector machine classifiers with 2000-3000 support vectors for 32x32 images

63

Page 69: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

require in the order of 10 Mflops to classify one image [164]. More complexmethods will require substantially more computational power, as for instancecomputing a Gabor wavelet transform involves computing a fast Fourier trans-form.

Dedicated hardware can be used to speed up computations, for example [92]reported real-time performance in a test with a 5-users database.

The storage of a 3D face template on a chip memory is possible, as the tem-plate’s size is about 3000 bytes. Furthermore, an implementation of the wholesystem onto an identity document, using a sensor on card or match on cardapproach, is technically impossible, as it is necessary to use currently imposingmaterial during the acquisition step.

5.6 Open source systems

Several open source face recognition systems exist, for example Intel’s OpenCVlibrary [42] contains face detection and recognition algorithms. A completeevaluation system is provided by the Colorado State University [36], compris-ing implementations of four baseline face recognition algorithms: eigenfaces,combination PCA and LDA based on the University of Maryland algorithm,Bayesian Intrapersonal/Extrapersonal Image Difference Classifier based on theMIT algorithm, elastic bunch graph matching based on the USC algorithm.

For 3D facial recognition, several approaches are presented in the literature,but to the best of our knowledge, none are available as an open source system.

5.7 Databases

5.7.1 2D facial databases

Many databases are publicly available for 2D face recognition.

AT&T Laboratories Cambridge’s Database of Faces AT&T Laborato-ries Cambridge’s Database of Faces 5[246] contains 10 images for each of its 40subjects, with various lighting conditions and facial expressions. The imagesare grayscale, 92x112 pixels.

BANCA BANCA [15] contains video and audio data for 208 users, capturedwith 2 different cameras (one webcam and one high-quality digital camera) and2 different microphones in 12 sessions. It offers three lighting and acousticalconditions. In addition, 10 frontal images of 30 users are provided for worldmodeling. The images are 24-bits colour, 720x576 pixels.

FERET FERET [216] comprises scanned images from analogue 35 mm film.The images were taken in 15 sessions over 3 years, comprising a total of 1199users and 14’126 images. Images have neutral and non-neutral expressions.

5Formerly known as the Olivetti database, this database is available online athttp://www.cl.cam.ac.uk/Research/DTG/attarchive/facedatabase.html.

64

Page 70: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

PIE PIE [254] (CMU Pose, Illumination and Expression database) contains68 users and a total of 41368 images. Each user is simultaneously photographedby 13 high-quality cameras located at different positions, thus providing 13different viewpoints. The person is photographed under 43 different illuminationconditions, and provides 4 different facial expressions. The images are colour,640x486 pixels. This database is not particularly suited for identity documentsas conditions are likely to be less controlled than they would be in an identityverification scenario. The background, for instance, is non-uniform.

UMIST UMIST [106] contains 20 users and a total of 564 images 6. Theusers are captured under a variety of poses. The images are 256-levels grayscale,220x220 pixels. This database could be useful for experiments relating to iden-tity documents, as the background is plain, and different poses are present.However, it has a small number of users.

XM2VTS XM2VTS [191] contains 295 users and was recorded in four ses-sions. Different subsets of this database are available. 8 frontal images peruser (2 per session) are available, in total 2360. Furthermore, single-session im-age sets with different illumination conditions are available, adding 4 images peruser, adding 1180 images. Lastly, an additional profile views dataset is available,adding 2360 images. All images are colour, 720x576 pixels. The background isuniform blue. Because of its controlled conditions and its relatively large userpopulation, this database can be used for experiments with identity documents.

Yale faces database Yale faces database 7 contains 15 users and total of 165images. 11 images per user are taken, in a variety of expressions and lightingconditions. The images are grayscale, 320x243 pixels. The lighting conditionsare well controlled. While the images are of good quality and various expres-sions could be useful in testing face recognition for identity documents, the userpopulation contained in the database is too small.

AR face database AR face database 8 [180] contains 126 users (70 men and56 women). 26 images per user are taken over two sessions 2 weeks apart.Each user has a variety of expressions, illumination conditions, and partial faceocclusions (scarf). The Images are 24-bits colour, 768x576 pixels. In addition,30 sequences of 25 images each are provided.

5.7.2 3D facial database

Contrary to 2D face, only few databases are available for 3D facial recognition[107]. The Max Planck Institute for Biological Cybernetics 9 has created a 3Dfacial database, acquired with a laser scanner, and containing 200 subjects. The

6This database is available at http://images.ee.umist.ac.uk/danny/database.html.7http://cvc.yale.edu/projects/yalefaces/yalefaces.html.8http://rvl1.ecn.purdue.edu/~aleix/aleix_face_DB.html.9Max Planck Institute (MPI) for Biological Cybernetics homepage:

http://www.kyb.tuebingen.mpg.de/.

65

Page 71: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

whole 3D data are available only for 7 subjects10. The XM2VTS database [191]has acquired 3D models of 293 subjects, and all these data are available 11.

5.8 International competitions

Three databases are currently the most commonly used for 2D face recognitioncompetitions. They are FERET, XM2VTS, and BANCA.

5.8.1 Competitions on FERET

The FERET database [216] also defines a protocol for identification and verifi-cation. Three evaluations took place using this protocol, the last in 1996 and1997 [215]. The evaluation protocol stipulates that only a development set ismade available, while the real testing set remains under control of the evaluationorganisers. Several testing scenarii are used, which take into account two of themain factors of recognition degradation in biometrics: environmental conditions(lighting differences) and inter-session time. It also investigates performance infully-automated conditions (only the face image is given) and partially auto-matic conditions (eye coordinates are given). For the March 1997 evaluation,the best-performing partially automatic system had a correct identification atrank 1 of 96% for test images taken the same day, which dramatically droppedto about 59% for test images taken on different days. About 82% correct iden-tification at rank 1 was obtained for test images taken on the same day butwith different lighting. Images taken over 1 year apart result in a drop to 52%at rank 1. For fully automatic systems, the best result obtained is about 94%for test images taken on the same day (a 2% absolute difference with partiallyautomatic systems). For images taken on different days, the figure drops toabout 58%.

The Face Recognition Vendor Tests (FRVT) aim at independent governmentevaluation of commercial systems, though academic institutions are free to par-ticipate. Two evaluations took place, in 2000 and 2002 12. A further one isplanned for 2006 13, with the goal to measure progress since FRVT 2002. Ad-ditional facial images (high resolution and multiple-samples still facial images)will be used for the purpose of the this evaluation. The 2000 FRVT tested with adatabase of 1’196 users (FERET) [32], while the 2002 FRVT tested with 37’437users [213]. The results of the FRVT 2002 will be briefly summarised here, asthey are important and can be used to set expectations for the identity docu-ments application. It is especially interesting to note the marked differences inperformance between the vendors in the verification scenario. The best systemhad about 90% correct verification at 1% false accept (about 5% EER), whilethe worst system had about 25% EER. Also, verification performance markedlydegrades in time, with systems loosing between 5% and 8% verification rate peryear since enrollment.

10MPI database available online at http://faces.kyb.tuebingen.mpg.de/.11XM2VTS database available online at http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/.12FRVT 2002 homepage: http://www.frvt.org/FRVT2002.13FRVT 2006 homepage: http://www.frvt.org/frvt2006.

66

Page 72: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

5.8.2 Competitions on XM2VTS

As part of the audio and video-based personal authentication conference (AVBPA)2003, a face verification competition was held on the XM2VTS database usingthe Lausanne protocol [191]. The best performing partially automatic systemshad error rates between 0.97% FA at 0.5% FR and 0.25% FA at 0.5% FR depend-ing on the train/evaluation set partition. The best performing fully automaticsystems had error rates between 1.36% FA at 2.5% FR and 1.35% FA at 0.75%FR depending on the train/evaluation set partition.

As part of ICB 2006, a face verification contest has been helded on theXM2VTS database [192]. The best performing system achieved 0.26% FA at1.57% FR, with automatic registration of the images. The worst perfromingsystem achieved 5.13% FA at 3.25% FR. These algorithms were also testedin order to evaluate their sensitivity to severe changes in subject illumination.In these later conditions, the best performing system with manual registrationachieved 0.77% FA at 1.25% FR, while the worst performing system achieved6.20% FA at 77.37% FR.

5.8.3 Competitions on BANCA

During the international conference on biometric authentication in 2004 (ICBA2004), the BANCA database and evaluation protocols were used [190]. Fourdifferent institutions (academic and commercial) participated. Only data fromthe“controlled conditions”subset was used. In the partially-automated scenario,the best system obtained an HTER of 2.9% when equal costs are assumed forFA and FR. For the fully-automatic case, the best system obtained about 5.4%HTER.

The BANCA database was used as part of the face verification contest atthe international conference on pattern recognition in 2004 (ICPR 2004), with52 users [189]. 10 different institutions participated. Data from all 3 conditionssubsets were used. For partially automatic verification, the best system had anHTER of about 2.2%, with significant differences between algorithms. The bestfully automatic system obtained an HTER of about 3.1%, also at equal costsbetween FA and FR. Using sequestered (never seen before) test data led to asharp decrease in performance, with the best system dropping to about 13.5%.

5.8.4 3D face verification competitions

So far, no international competition was completed for 3D facial recognitionsystems, aside the Face Recognition Vendor Test (FRVT) 2002 which has eval-uated 3D morphable models technologies, methods increasing the performanceof face recognition systems. But at the time of writing, some new competitionsare in progress.

Face Recognition Vendor Test 2005 The FRVT 2006, supervised by theNIST, is planning to evaluate among others performance on 3D facial scans,what was not the case in the previous evaluations (FERET, FRVT 2000 andFRVT 2002).

67

Page 73: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Face Recognition Grand Challenge Additionally, since 2004, the FRGC 14

has been promoting face recognition technology in three main areas: high res-olution images, three-dimensional face recognition and new preprocessing tech-niques [212]. The FRGC will assess the merits of all these three approaches si-multaneously. 50’000 samples, for training and validation purposes, will be usedin the FRGC. These include single still, multiple stills, outdoor/uncontrolled,3D single view, 3D full face, and acquisition from several cameras. The goal ofFRGC is to develop still and 3D algorithms in order to improve performance anorder of magnitude better than FRVT 2002 (from 20% FRR in FRVT 2002 to2% FRR at FAR=0.1%).

14FRGC homepage: http://www.frvt.org/FRGC/.

68

Page 74: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 6

Modality: Fingerprint

6.1 Introduction

Even if some archaeological vestiges allow to posit that human being knewthe individuality of fingerprints 4000 years ago, the real start of the study foridentification purposes was initiated by Faulds, Herschel and Galton in the late18th century. The individuality of this modality was first formulated, almostsimultaneously, by Faulds [93] and Herschel [114] for the purpose of recidivists’and recovered marks’ identification. In the late 18th, Galton published uponthe permanence of fingerprint all over life time [101]. He also introduced theuse of minutiae features in the matching process.

Matching automation appeared since 1960 with Automatic Fingerprint Iden-tification Systems (AFIS), for avoiding fastidious manual searches in large data-bases. Since then, the market did not stop developing and evolving, and alarge number of criminal, non-criminal and civilian identification applicationsare available today. These two main applications, criminal and civilian sec-tors have some differences and some similarities [294]. The size of the databaseis larger for criminal than for civilian applications, and the image quality ismore homogeneous for civilian applications as captured by the same sensor,but they follow nevertheless the same basic stages: acquisition, representation,pre-processing, feature extraction and matching.

A fingerprint can be described in three levels of features. The first levelconcerns the general flow of the ridges. In a fingerprint, a core and up to twodeltas can be generally observed (Figure 6.1). They are also considered as level1 features.

When the number and the position of these focal points (delta(s), core,. . . )change, the general ridge flow shape can differ. The general shape can thus beclassified (among other methods) according to the number and positions of thedeltas and the position of the core. The different shapes can be classified as:left and right loop, whorl, arch and tented arch (Figure 6.2).

The next two levels are more localized features. The second level relatesto the minutiae features observed on a fingerprint. According to Galton, theindividuality of this modality is related to the particular arrangement of ridgeterminations and ridge bifurcations. Figure 6.3 presents examples of possibleminutiae characteristics, which are all composed by ridge terminations and ridge

69

Page 75: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Figure 6.1: Representation of fingerprint core and delta.

Figure 6.2: Representation of general shapes of fingerprints.

bifurcations. Other characteristics, such as wrinkles, creases and warts alsoincluded in this second level are presented in Figure 6.4.

The third and last level concerns the sweat pores observed on the ridgesand the ridge sides. Figure 6.5 presents examples of sweat pores presence infingerprint representation.

6.2 Overview of algorithmic approaches

6.2.1 Human matching process

Before presenting the different algorithmic approaches used in automatic sys-tems, we recall the ordinary process used by human experts in an off-line match-ing process. The Locard’s tripartite rule is a good pragmatic statement [167],however the practice is more diverse (for more information on about, see [56]):

- If more than 12 concurring points are present and the fingermark is sharp,the certainty of identity is beyond debate. (The imperative requirementfor the absence of significant differences is implicit).

- If 8 to 12 concurring points are involved, the case is borderline, and thecertainty of identity will depend on: the sharpness of the fingermark, therarity of its type, the presence of center of the figure (core) and the triangle(delta) in the exploitable part of the mark, the presence of pores, theperfect and obvious identity regarding the width of the papillary ridges andvalleys, the direction of the lines, and the angular value of the bifurcations.In these instances, certainty can only be established following discussionof the case by at least two competent and experienced specialists.

70

Page 76: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Figure 6.3: Representation of minutiae observed on fingerprints.

Figure 6.4: Representation of wrinkles, creases and warts observed on finger-prints.

- If a limited number of characteristic points are present, the fingermark can-not provide certainty for an identification, but only a presumption pro-portional to the number of points available and their clarity.

6.2.2 Automatic matching process

In an on-line automatic matching process, three approaches are possible [173]:the correlation-based, the minutiae-based and the ridge features-based match-ing.

Correlation-based matching The images are superposed and a correlationcalculation is completed for different positions, obtained by translation and ro-tation.

Minutiae-based matching This approach, well reviewed in [294], is com-monly used in most fingerprint identification systems. Methods such as localarea contrast enhancement and contextual filtering are used for the enhance-ment procedure, a crucial step during the pre-processing stage [8]. Contextual

71

Page 77: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Figure 6.5: Representation of sweat pores observed on a fingerprint.

filtering or frequency analysis methods are used to enhance the fingerprint im-age, as the ridges have well-defined frequency and orientation in local area. Thetools used in this matter are Fourier transforms, Gabor filters and wavelets. Forthe minutiae features extraction, this approach often use binary and skeletonimages. The main stages of the whole feature extraction consist in orientationfield estimation by using information about local average directions of the ridges(gradient or ridge-valley algorithm), ridge detection by using either a threshold-ing algorithm given a gray scale image, the ridge-valley algorithm, or gray levelhistogram decomposition, and thinning, by using algorithms based on mathe-matical morphology [294]. After feature extraction, locations and orientationsof minutiae are stored as a set of points. A matching score is completed afteralignment of these sets of the two fingerprint images. Figure 6.6 describes thefeature extraction process of this approach.

Figure 6.6: Feature extraction process [173].

Ridge features-based matching Features which are more reliably extractedfrom the ridge pattern in comparison to minutiae are used, such as local ori-entation and frequency, ridge shape, texture information. For example, thealgorithm presented in [144] used this approach. The main steps are as follows:normalization and background segmentation, localization of the singular point,determination of features’ extraction sectors, Gabor filterbank features extrac-tion and statistical features, combination of the extractions to obtain a unique

72

Page 78: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

feature vector from it (called FingerCode) and euclidean distance calculation.Figure 6.7 describes the process of this particular approach. This method allowsobtaining a 80x8 fixed-size feature vector (FingerCode).

Figure 6.7: System diagram of the FingerCode approach [144].

Note that most AFIS systems just act as a filter whose candidates are thencompared using the above holistic process, carried out by a fingerprint examinertrained to competency.

The templates obtained in the minutiae-based matching approach are usuallyof 250 to 700 bytes [203]. However, ridge features-based matching approachproduces templates usually of about 900 to 1200 bytes.

According to the results of the two fingerprint verification competitions in2000 [170] and 2002 [171], the minutiae-based methods performed better thanthe correlation-based methods. The reason for the superiority of minutiae-basedalgorithms is the stability and large signal capacity of the minutiae representa-tion [222]. The use of minutiae information in large-scale systems will persistdue to its high discrimination power, but future systems may integrate also non-minutiae features, particularly for poor quality fingerprint images [294]. Forverification purposes, a ridge features-based algorithm has better performancethan a minutiae-based algorithm, only if the false acceptance rate required bythe specific application is not too low, or has slightly lower performance thanstate-of-the-art minutiae-based algorithms [144]. For forensic purposes, most ofthe marks recovered from crime scenes are partial and of poor quality. Minutiae-

73

Page 79: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

based approaches are preferred in such situations, as the general shape is noteasily recognizable automatically or amenable to manual coding using a skilledoperator.

6.3 Performance

Automatic fingerprint recognition approaches are competitive systems [34, 173].But some limitations can influence their performance [8].

At the data acquisition level, the sensor, if optical, has to be cleaned aftereach use, for avoiding background noise, resulting of the deposition of fingerprintsecretions. This noise can distort the original image and introduce artifacts dur-ing the features extraction. Pressure and contact angle differences, displacementand rotation of the finger on the sensor, non-linear distortion and skin conditioncan also influence the performance of such systems in the following algorithmicstage.

During the extraction process, errors in the form of missed or wrongly detectedminutiae can also appear, which can influence the performance of such a system.

Finally, the performance of such systems can also be influenced by the pro-portion of the population which do not have the modality or have a fingerprintof not sufficient quality to be enrolled (about 4% of the population). In thecase of a large-scale use, such as it is for identity documents’ purposes, thispercentage corresponds to almost 300’000 persons who would be in that case inSwitzerland, what is relatively important and must not be neglected. However,according to a National Institute of Standards and Technology (NIST) researchreport, this percentage was recently proved to be overestimated [115].

The NIST conducted several technology evaluations in order to estimate theperformance of fingerprint recognition solutions.

For the joint “303A Report” to U.S. Congress from NIST, the Departmentof Justice and the Department of State [201] 1, the recommendations proposedin 2003 were based on previous NIST’s studies presented in the Appendix A ofthis report [200] 2. In its Image Quality Study, the NIST had presented themain factors that influence the flat-to-rolled matching process: the number offingers used in the comparison process, the correspondence areas between thefingerprints compared, and the quality of the fingerprints. On a small dataset of fingerprints, a correct acceptance rate of 99% was obtained for verifica-tion purposes with the thumbs, at a false acceptance rate of 1%. On a largerbackground data set of fingerprints (620’000 subjects), the correct identificationrates at rank one were 95%, 90% and 86% when a probe set of 500, 10’000 and100’000 subjects respectively were used for the identification process.

The NIST conducted in 2004 a technology evaluation in order to estimatethe performance of the system and sensor used in the US-VISIT program [285].

1Section 11.2 presents more information about the recommendations of the ”303A Report”.2The Appendix A of this joint report is available online at

http://www.itl.nist.gov/iaui/894.03/fing/fing.html.

74

Page 80: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Verification and identification performances have been estimated in this evalu-ation. For the verification process, the experiments were conducted on a set of6’000 finger pairs (right and left index) against 6’000 gallery images. The cor-rect acceptance rates obtained at a false acceptance rate of 0.01% were 97.5%and 95.3% for the right and respectively left index. In combining these twofingers by adding the matching scores and by applying a specific threshold, thecorrect acceptance rate was 99.5% at a false acceptance rate of 0.1%. For theidentification process, the experiments were conducted on a set of 60’000 fin-ger pairs and on a background gallery of 6’000’000 finger pairs. The correctacceptance rate obtained at a false acceptance rate of 0.31% was 95.9%. Theresults showed also the influence of the background size on the performance. Ifthe background size increased, the correct acceptance rate was constant, whilethe false acceptance rate increased linearly. The image quality was presentingin this report as the ”most critical single factor impacting the performance offingerprint recognition systems” and its impact is ”greater than the impact ofthe difference in algorithms”.

The NIST conducted in 2004 studies for evaluating the fingerprint matchingperformance by using their Verification Test Bed (VTB) [287] 3. The evalua-tions were separated in small-scale studies on data sets of 20, 216, 1021 and2700 subjects (10 seconds video sequence per finger for the first set and 2 finger-print cards per subject for the other sets) and large-scale studies on data sets of52’000 (and 46’000 subjects for the background), 225’000 (and 225’000 subjectsfor the background), 274’000 (and 6’000’000 subjects for the background) and600’000 subjects (1 fingerprint card per subject for the two first sets, and 2 setsof left and right index per subject for the two last sets). The small-scale studieswere conducted for evaluating the influence, in verification purposes, of the useof plain or rolled fingerprints and of the choice of the fingers on the perfor-mance. For rolled-to-rolled fingerprint comparisons, the correct acceptance ratewas 96% at a false acceptance rate of 1%, while the correct acceptance rate ofplain-to-rolled comparisons was 90% at the same false acceptance rate. Whenindex finger and thumb are combined, the correct acceptance rate was 99% ata false acceptance rate of 1%, while the correct acceptance rate when the twoindex fingers were combined was 98% at the same false acceptance rate. Theauthors explained this difference by the fact that a larger area were availablefor the thumb und thus more minutiae were detected and used for the recog-nition process. With the large-scale studies, similar results were obtained forverification purposes, with inked and live-scan fingerprints. At a false accep-tance rate of 1%, the correct acceptance rate was about 91%. With a data setof 620’000 subjects, the identification rate for right index fingers was 76% fora rank-1 thresholding, while this rate was 85% for the combination of the twoindex fingers.

The NIST conducted in 2004 studies for evaluating the matching perfor-mance of plain-to-rolled fingerprints using their Algorithmic Test Bed (ATB)in identification purposes [289]. With data sets of about 1’000, 48’000, 60’000,274’000 and 300’000 subjects, the accuracy of plain-to-rolled was similar torolled-to-rolled recognition, with a correct acceptance rate of about 98%. Theuse of 10 fingers in the plain-to-rolled comparison process will not provide abetter accuracy than with fewer fingers.

3The VTB used the NIST Fingerprint Image Software (NFIS), described in Section 6.6.

75

Page 81: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

The German Federal Office of Security in the Information Technology (Bun-desamt fur Sicherheit in der Informationstechnik) has evaluated the influence ofminutiae-based matching algorithms, sensor technologies and ageing of the dataavailable as model [48]. The evaluations, called BioFinger1 4, were conductedon a set of 30 persons (all the fingers, except the little fingers, were used). Foreach of the 11 sensors tested (6 optical, 3 capacitive, 1 thermal and 1 piezo-electric sensors), the fingers were acquired in 3 different sessions (3 samples persession and per finger). Each sensor was combinated with each of the sevenminutiae-based matching algorithms tested for the evaluation of the recogni-tion’s performance. Among all the possible combinations, the half had an equalerror rate lower than 5%, a third had an equal error rate lower than 3% andonly 8% had an equal error rate lower than 1%. For a false acceptance rate of0.1%, the half had a correct acceptance rate higher than 90%, while only 23%of all these combination had a correct acceptance rate higher than 97%. Thebest results were obtained with optical sensors, while the differences betweenalgoritms were less relevant. Regarding the ageing and its influence on the recog-nition process, the autors estimated that the false rejection rate will duplicateif the age difference between the enrolled template and the template acquiredduring the transaction reaches 10 years. Another evaluation (BioFinger2) willbe conducted in order to estimate the performance improvement when multiplefingerprints are used in the verification process.

“State of the art fingerprint matching algorithms are nowhere near the theo-retical upper bound on their performance” [294] and thus even more efforts willbe necessary to improve their performance. Several challenging evolutions haveto be done in this matter:

Inconsistent and irreproducible contact The problems in inconsistentcontact entail that such systems should be invariant to translations or rotations,and that they should be able to match, even if the overlap area is small.

Elastic distortions Minutiae-based algorithms should also be invariant tothe nonlinear deformations introduced by the switch-over 3D-2D. If a model ofthese distortions could be created, which was not fully explored until now, theperformance of such systems can increase in an important way.

Incomplete ridge structure The pre-processing stage should be able towork robustly with incomplete fingerprint images, especially when core or delta(s)are not visible.

6.4 Sensors

To obtain a digital image from a fingerprint, several methods are conceivable[173]. For AFIS-systems, inked-fingerprints are usually used for acquire digitalimages off-line. For on-line methods, two types of sensors are available using

4The BioFinger1 evaluation is available online at http://www.bsi.bund.de/literat/index.htm.

76

Page 82: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

touched and sweep methods (see Table 6.1 for a quick overview of existingtouched-based sensor approaches).

Optical Solid-state UltrasoundFTIR Capacitive UltrasoundOptical fibers ThermalElectro optical PiezoelectricDirect reading

Table 6.1: Touched-based sensor approaches for fingerprint technology.

Touched methods The fingerprint is simply put in contact with a flat sensor.Touched methods use optical, solid-state and ultrasound sensor based. Suchmethods are easy to use and need no training. But some issues are associated:hygienic problems; unfavorable influence of dirty fingers on the image quality;remanent fingerprint residue on some sensors; critical location of the fingerprinton the sensor’s area to obtain sufficient information; influence of the size of thesensed area on the cost.

Sweep methods The fingerprint is simply moved vertically on a window sen-sor, which is as wide as the finger. Then the slices are combined to obtain thefingerprint in its entirety. The sensor’s cost is lower than previously for thetouched methods, but some inconveniences have to be revealed: the period ofadaptation for users is higher than other methods; sufficient number of sliceshave to be taken, what requires a powerful microprocessor; the reconstructionprocess takes time.

6.4.1 Optical sensor

For touched methods, several sensor types are available. The first sensor typepresented in this chapter is the optical sensor.

Frustrated Total Internal Reflection (FTIR) The ridges of the finger arein contact with one side of a prism. From second side, light is produced andfocused through a lens onto an image sensor on the third side. This glass prismcan also be substituted by a sheet prism, small prisms adjacent to each other,using the same methodology.

Optical fibers The prism and the lens of the latter method is substitutedby a fiber-optic platen. The ridges of the finger are directly in contact withthe upper side of the platen. On the opposite side, a CCD/CMOS receives theresidual light. Such methods are very expensive, as there is a large sensor’s area,but the global size of the sensor is reduced, with regard to the FTIR sensors.

Electro optical This sensor is composed by two layers. A first polymer layerallows a light emission if a potential is applied, and a second photodiode arraylayer, converts the light emitted into a digital image. Such methods create apoor quality image, compared to the FTIR methods.

77

Page 83: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Direct reading The fingerprint is directly focused by a high-quality camera,without the need of a contact with a sensor. However well-focused and highcontrasted images are difficult to obtain.

6.4.2 Solid-state sensor

The second sensor type presented in this chapter is the solid-state sensor, alsocalled silicon sensor. The sensor consists of an array of pixels, in which eachpixel is a tiny sensor and without any optical and CCD compound. Such systemsneed smaller size, and the cost are thus reduced.

Capacitive The sensor is a 2D array of micro-capacitor plate, protected by athin coating. The other plate of the micro-capacitor is the ridges of the finger.The electrical charges are different in function of the distance between the ridgesand valleys and the plate.

Thermal The sensor is made of pyro-electric material, maintained at a hightemperature by electrical heating, producing current based on temperature dif-ferentials. The ridges and the valleys produce a different temperature differ-ential, generating the creation of the image. This approach is usually used insweep methods.

Electric field The sensor consists of a drive ring, generating a sinusoidal sig-nal, and a matrix of active antennas, receiving a small amplitude signal trans-mitted by the ring and modulated by the finger skin structure. The finger hasto be simultaneously in contact with both elements. For obtaining the image ofthe fingerprint, the sensor matrix is amplified, integrated and digitized.

Piezoelectric The sensor consists in a pressure sensitive area, composed bya non conductive di-electric material. When mechanical stress is applied to thesensor, an electrical signal, proportional to the pressure, is produced, generatingthe image of the fingerprint, as the pressure is different for ridges and valleys.The sensor is not sensitive enough to detect small differences in pressure. Suchapproaches produce poor quality images, compared to the other sensors.

6.4.3 Ultrasound sensor

The third and last sensor type presented in this chapter is the ultrasound sensor.This sensor consists of an acoustic signal sent through the platen, and the echosignal is captured. It works like an echography and produces good qualityimages. The acquisition needs a few seconds, the sensor is quite expensive andis thus not mature enough for fingerprint recognition systems.

6.4.4 Ergonomics and acquisition environment

In automatic recognition, the ergonomics of the systems are relatively pleasant.Indeed, as it is a question of putting in contact, during about 1 second, one orseveral fingers with one or more sensors, the system is arranged in a way thatthe subject can do it in a not binding position. The finger has to be placed flat

78

Page 84: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

on the sensor and has to be localized in the center of the sensor. Some hygienicproblems may appear. In Asian countries for example, feedback from usersranked hygienic issue at the forefront of the concerns. Some problems couldalso appear for people with amputated fingers. Finally, template ageing existssomehow for fingerprints. Indeed, during the life time, the quality of finger-print pattern can decrease, especially for manual workers, and thus the featuresextracted during the verification/identification process will not correspond anymore to the features extracted during prior enrollment.

The environment where the system is placed will also affect the quality offeatures extracted. Depending on the temperature or the humidity, the fingerwill be more or less dry, thus affecting the features acquired. The location(position, inclination,...) of the system for the verification/identification processcan also influence the results, especially if it differs from the enrollment process.

6.4.5 Fingerprint acquisition for identity documents

In identity documents’ applications, two possibilities are available: the acquisi-tion device is external or internal to the document. In function of the purpose,some sensors are preferable to others. Indeed, big sensors, as FTIR optical sen-sor, can not be integrated in an identity document’s sheet. This kind of useneeds thin sensor like optical fibers or capacitive sensor. With such devices,the biometrics can be directly acquired on the identity document. Furthermore,liveness detection (the ability of a sensor to detect if the information presentedto the sensor is ”alive”) should be integrated in all fingerprint sensors approachesused in the field of identity documents. Detection methods, based on perspi-ration, distortion, spectral technology, ultrasonic, pulse and electric resistance,may be used to prevent any abuse in this field.

6.5 Computational resources

A fingerprint template can be stored in several ways, as described in Section2.7.2. The template generally consists of a matrix containing series of x, y coor-dinates and the orientation of the minutiae positions, approximately less than700 bytes in most of the cases. The chip memory is then sufficient to store thefingerprint template. The possibility also exists to compute the entire processon the card, from the acquisition stage to the decision stage. Actually match-on-card technology using fingerprint recognition is already available from severalbiometric vendors (see Section 2.7). The storage and the identity verificationare completed directly on the smart card. As a sensor is also integrated in thecard, there would be no longer the necessity for a separate fingerprint acquisitiondevice, reducing thus the cost of an access control system. As all the processingsteps are performed on the card itself, in a closed environment, this increasesthus the security and the privacy of this biometric recognition system. Indeed,the owner of the card has a complete control on his biometric information storedon it.

6.6 Open source systems

79

Page 85: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

NIST Fingerprint Image Software 2 (NFIS2) This software was devel-oped by the National Institute of Standards and Technology (NIST) 5. Here arethe main components of this software:

1. Fingerprint segmentation algorithm: the NFSEG can segment the four-finger plain impression found on a fingerprint card into individual imagesor can be used to remove only white space from fingerprint images;

2. Fingerprint Pattern Classification: the PCASYS can categorize the gen-eral shape of the fingerprint into six classes (arch, left or right loop, scar,tented arch, and whorl).

3. Minutiae detector: the MINDTCT can locate the ridge endings and bifur-cations on a fingerprint image and assess the minutiae quality, based onlocal image conditions.

4. Fingerprint Image Quality Algorithm: the NFIQ can assess the imagequality.

5. Fingerprint Matching Algorithm: the BOZORTH3 can proceed in verifi-cation and identification modes.

The NFIS2 Software contains also reference implementation of the ANSI/NIST-ITL 1-2000 standard and a large collection of general-purpose image utilities,such as JPEG, WSQ encoders and decoders.

FingerCode An open source software, based on the work of [144] (see Section6.2), is available online 6. This open source is implemented in MATLAB R©.

Fingerprint Verification System An easy to use library that allows pro-grammers to integrate fingerprint technology into their software is availableonline 7.

6.7 Databases

Some fingerprint databases are publicly available [34]. First, the National In-stitute of Standards and Technology (NIST) has a large number of fingerprintdatabases, with rolled fingerprint (NIST-9, NIST-10 and NIST-29), livescanvideo fingerprint (NIST-24), latent crime scene fingerprint (NIST-27), finger-print of different resolutions (NIST-30), and some other databases (NIST-4 andNIST-14). The University of Bologna (Italy) organized Fingerprint Verifica-tion Competition (FVC) in 2000 [170], 2002 [171] and 2004 [172] using threedifferent live-scan devices, for each competition, to acquire the images of thedatabases. These databases contain also for each competition new syntheticgenerated fingerprints sets 8. These twelve databases are all available in [173].

5NFIS2 open source system available online at http://fingerprint.nist.gov/NFIS/.6FingerCode source code implemented in MATLAB R© available at

http://utenti.lycos.it/matlab/speed.htm.7Fingerprint Verification System source code available at http://fvs.sourceforge.net/.8More information about the Synthetic Fingerprint Generator (SFinGe) program, devel-

oped by the Biometric Systems Lab (University of Bologna), can be found in [173] and onlineat http://biolabs.csr.unibo.it.

80

Page 86: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

6.8 International competitions

The National Institute of Standards and Technology (NIST) is at the origin ofsome international competitions, so is the University of Bologna. It is necessaryto clarify that, because every evaluation uses different databases, the results arenot comparable between competitions. Indeed, every result is related to thedatabase on which the evaluation was completed, as the performances can varydramatically based on the characteristics, or type of the data [170, 286].

6.8.1 Fingerprint Vendor Technology Evaluation 2003

This evaluation is the FpVTE 2003 [286] 9, performed by the NIST. It wasconducted to evaluate the accuracy of the state-of-the-art fingerprint matching,identification and verification systems. 34 systems from 18 different compa-nies, were evaluated. The tests were conducted on three separated datasets:large-scale, medium-scale and small-scale tests. The large-scale test used 64’000fingerprint sets, containing 1 to 10 fingerprint images each, and these sets werepartitioned in 31 subgroups (varying the number of fingers contaning in thesets). The correct acceptance rates, when multiple fingers were used, wereabout 100% at a false acceptance rate of 0.01%. The three best matchers hadon all the subtests of this large-scale evaluation, correct acceptance rates higherthan 95% at a false acceptance rate of 0.01%. The medium-scale test used aset of 10’000 fingerprint images, containing only right index fingers. For thetwo best algorithms, the correct acceptance rates were higher than 99.3% at afalse acceptance rate of 0.031%. Finally, the small-scale test used a set of 1’000fingerprint images, containing also only right index fingers. The three best al-gorithms had correct acceptance rates of about 100% at a false acceptance rateof 0.1%. In conclusion, the most accurate system at a false acceptance rate of0.01% had correct acceptance rates higher than 98.6%, 99.6% and 99.9% forevery single-finger subtests, two-finger subtests, and 4, 8 or 10-finger subtestsrespectively. The variable which had the largest effect on system’s accuracy,exept the number of fingers used, was the fingerprint quality.

6.8.2 Studies of One-to-One Fingerprint Matching withVendor SDK Matchers

This evaluation is the Studies of One-to-One Fingerprint Matching with VendorSDK Matchers [279] 10, performed by the NIST in 2003. It was conducted toevaluate the accuracy of one-to-one matching used in the US-VISIT program.Additionally, 8 fingerprint matching vendor systems were also evaluated. Thesesystems were tested on 12 different single finger data sets of varying difficulties.A random sample of 5800 subjects were selected from some live-scans databasesand inked impressions databases. In this study, only fingers from same typeare compared between them (right middle fingers between them and so on).Each SDK evaluated had performed 614’638’238 comparisons. According tothe results of this evaluation, the two most accurate systems had a correct

9The FpVTE 2003 Summary of Results and Analysis Report are available online athttp://www.itl.nist.gov/iaui/894.03/fing/fing.html.

10These one-to-one studies of vendor SDK matchers are available online athttp://www.itl.nist.gov/iaui/894.03/fing/fing.html.

81

Page 87: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

identification rate of more than 98%, at a false acceptance rate of 0.01%, andthe worst more than 94%, at a similar false acceptance rate. This evaluationrevealed also that in general, better performance was obtained with thumbs,rather with index. Furthermore, with right fingers, the performances were alsobetter than with left fingers. However, thumbs and index fingers had similarperformance when high quality images were used for the matching process.

6.8.3 Fingerprint Verification Competition

This evaluation is the Fingerprint Verification Competition (FVC) performed bythe University of Bologna in 2000 11 [170]. The four databases created for thisevaluation used the state-of-the-art sensor technologies for three of them and thesynthetic fingerprint generation program SFinGe for one of them. Each databasecontained 880 impressions from 110 different fingers. The first two were acquiredin two different sessions from 25 people. 4 fingers per person and per session wereacquired. The third database were acquired from 19 people with age variations.In four sessions, 6 fingers per person were acquired twice. The impressions ofthe 10 last fingers were used as a training set for the participants. 11 algorithmshad been submitted for testing. According to the results of this evaluation, thetwo most accurate systems had an average equal error rate lower than 2.28%,with an average equal error rate (for all participants and all databases) around14%. This evaluation revealed also that the synthetically generated databasewas adequate to evaluation purposes, such as the FVC2000.

6.8.4 Second Fingerprint Verification Competition

This evaluation is the second Fingerprint Verification Competition (FVC) per-formed by the University of Bologna in 2002 12 [171]. Four new databases werecreated, with three state-of-the-art sensors technologies and one syntheticallygenerated database (generated by the SFinGe program). The evaluation wasconducted on three groups, one for each non virtual fingerprint database, of 30people each. The 4 fingers per person were acquired in three different sessions,making vary some acquisition’s conditions, such as distortion, rotation, dry andmoist. 4 impressions per finger and per session were acquired. For each data-base, only a subset of 110 fingers, with 8 impressions per finger were taken intoaccount in this evaluation. The impressions of the 10 last fingers were used as atraining set for the participants. 33 algorithms had been submitted for testing.According to the results of this evaluation, the six most accurate systems hadan average equal error rate lower than 1.00%, and false non-match rates lowerthan 1.46% and 1.87%, at a false match rates of 0.01% and 0.001% respectively,with an average equal error rate for all participants and all databases of about7%.

6.8.5 Third Fingerprint Verification Competition

This evaluation is the third Fingerprint Verification Competition (FVC) per-formed by the University of Bologna in 2004 13 [172]. Four new databases were

11FVC 2000 homepage at http://bias.csr.unibo.it/fvc2000/.12FVC 2002 homepage at http://bias.csr.unibo.it/fvc2002/.13FVC 2004 homepage at http://bias.csr.unibo.it/fvc2004/.

82

Page 88: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

created, with three state of the art sensors technologies and one syntheticallygenerated database (generated by the SFinGe program). The evaluation wasconducted on three groups, one for each non virtual fingerprint database, of 30people each. The 4 fingers per person were acquired in three different sessions,making vary some acquisition’s conditions, such as distortion, rotation, dry andmoist. 4 impressions per finger and per session were acquired. For each data-base, only a subset of 110 fingers, with 8 impressions per finger were taken intoaccount in this evaluation. 67 algorithms, classified in open and light categories,had been submitted for testing. According to the results of this evaluation, thefive most accurate systems had an average equal error rate lower than 2.90%,and false non-match rates lower than 4.57% and 7.44% at false match rates of0.01% and 0.001% respectively.

6.8.6 Fourth Fingerprint Verification Competition

A fourth fingerprint verification evaluation will be performed by the Universityof Bologna, the Michigan-State University, the San Jose State University andthe Universidad Autonoma de Madrid in 2006. The results will be available onthe FVC 2006 homepage on January 2007 14.

14FVC 2006 homepage at http://bias.csr.unibo.it/fvc2006/.

83

Page 89: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 7

Modality: Iris

7.1 Introduction

In 1965, Adler, cited in [282], claimed that the human iris, which has a verycomplex layered structure unique to an individual, is an extremely valuablesource of biometric information. The general structure of the iris is geneticallydetermined, but the particular characteristics are “critically dependent on cir-cumstances (e.g. the initial conditions in the embryonic precursor to the iris)”and stable with age: iris recognition is thus considered as a promising biomet-ric approach [281]. At the time of the use of the anthropometrical system ofBertillon [24], such an individuality was not yet presupposed, even if it wasnecessary to introduce the colour of the iris.

Some people believe that the iris patterns are reflecting, amongst otherthings, the state of health of each of the organs in the human body, callingthis science iridology. But several scientific tests declared the iridology a med-ical fraud [22] and discredited its claims [281].

The number of features in the human iris is large [284]. It contains manycollagenous fibers, contraction furrows, coronas, crypts, colour, serpentine vas-culature, striations, freckles, rifts and pits. The iris is stable, as it is an internalorgan and thus protected by the eyelid, cornea and aqueous humour. Thismodality does not vary with age starting from the first year after birth untildeath. No foreign material usually contaminates the iris.

The uniqueness of the iris is due to the chaotic morphogenesis of that organ[78]: the statistical probability that two irises would be exactly the same isestimated at 1 in 10e72 [284]. Two different irises are extremely unlike to beequal, even in the case of genetically identical twins [80].

7.2 Overview of algorithmic approaches

Several iris recognition systems have been developed, which exploit the com-plexity and stability over time of iris patterns and claim to be highly accurate[169, 181, 281]. The most well-known algorithm, on which the principle state-of-the-art iris recognition systems are based, is the algorithms developed by Prof.Daugman 1 (Computer Laboratory, Cambridge University UK). This approach

1Personal homepage of Prof. Daugman at http://www.cl.cam.ac.uk/users/jgd1000/.

84

Page 90: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

comprises the following steps [79]:

Position localization of the iris After located the position of the eye onthe image, before the image capture, the first step is to locate the region of theimage corresponding to the iris. Indeed, it is necessary to localize precisely theinner and outer boundaries of the iris, and to detect and exclude eyelids if theyintrude. This preprocessing step is completed by the application of pseudo-polarcoordinates, taking into account that the inner and outer circular boundariesmay not be completely concentric.

Normalization The portion of the image corresponding to the iris is trans-lated to a normalized form, with a radius from 0 to 1, so that possible dilationof the pupil does not affect the system.

Features Extraction The features extraction process is completed by the useof 2D Gabor wavelets to perform a multiscale analysis of the iris. The regionsof the image are analyzed at different scales by frequency-selective filters. Thusthe information about local phase, coded with two bits corresponding to thesigns of the real and imaginary parts, is obtained. The result is a 256-byte code,which represents a specific iris and is called IrisCode.

Figure 7.1: Localization of the iris patterns and the correspondent IrisCode [79].

Matching step Similarity scores are obtained by computing a Hamming Dis-tance, with exclusive -or operations, to detect the fraction of the bits of the twoIrisCode that disagree. The computation speed is very high, as it uses Booleanlogic approach.

Here is a short description of three other iris recognition approaches. Thefirst approach, described in [281, 282, 283], localizes the iris using a histogrambased model fitting method. For representation and matching, it registers a cap-tured image to a stored model, filters with isotropic 2D bandpass decomposition(Pyramid Laplacian), followed by a correlation matching based on Fischer’s Lin-ear Discriminant. The second approach, described in [266], uses a combinationof gradient decomposed Hough transform and integro-differential operators forlocalize the iris, a 2D Hilbert transform to extract the features, and a Hamming

85

Page 91: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

distance calculation for the matching process. The last approach, described in[181], uses circular and linear Hough transforms for localizing the regions of in-terest and occluding eyelids, a convolution of the normalized regions of interestwith 1D Log-Gabor filters and phase quantising for encoding the features, anda Hamming distance calculation for the matching process.

The template obtained by iris recognition systems is approximately 500 bytes[203].

7.3 Performance

The performance of iris recognition systems is impressive. Even if these ap-proaches have high accuracies, some difficulties appear with iris recognitiontechnology, and thus influence the performance of such systems [281]:

While some iris features are yet in lace at birth, some features are only ma-ture around the second year of birth. Furthermore, the pigmentation patteringcontinues until adolescence and a depigmentation appears with advanced age.Additionally, some drug treatments and intensive exposure to contaminants canalter the iris pigmentation. Even if these problems do not really affect the per-formance of state-of-the-art systems at the moment, it will certainly do in thefuture, in a large scale application of such a technology due to template ageing.

During the acquisition process, some difficulties may appear. The iris is atarget which has a small size and which is in movement. The iris is also locatedbehind a curved, wet and reflecting surface. It can be obscured by lashes, lensesand reflecting eyeglasses. The iris can also be partially occluded by eyelids.According to the properties of the eye, the illumination light influences theform and the full-presence of the iris.

During the preprocessing steps, the shape of the iris can influence theIrisCode. Indeed, pupils are often not very round.

The performance of iris recognition systems remain however impressive.Some laboratory-based experiments allow to evaluate the overall performanceof state-of-the-art iris technology solution [281]. No false match was observed inall these experiments, only few false non-match errors were observed. Indeed,in one case, for a false match rate of 0%, the false non-match rate was about2%.

In [266], the system developed obtains a false rejection rate of 8.2%, witha false acceptance rate of 0% and the verification time lasts about 600ms. In[181], the system developed obtains a false acceptance rate of 0.005%, with afalse rejection rate of 0.238%. A US Army Research Lab deployment test [155]reports about 6% false reject and 0.00001% false accept on a sample populationof 258 users.

The International Biometric Group has conducted from July 2004 to April2005 the“Independent Testing of Iris Recognition Technology”[131], where threeiris recognition acquisition devices, using the software and the SDK providedby Iridian, were tested with 1224 subjects. The samples were acquired in twoseparate sessions (only 458 subjects participated in the second session). For

86

Page 92: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

the two enrollment transactions, two to four samples per eye and per devicefor each subject were acquired, while three samples per eye and per device foreach subject were acquired for the three different recognition transactions. Thematching process was completed off-line, after the acquisition process. The bestresult obtained was a FNMR of 0.583% at a FMR of 0.00129% in the transactionlevel and a FNMR of 0.759% at a FMR of 0.00013% in the enrollment level.

7.4 Sensors

The acquisition device of such systems, used to capture the details of the irispatterns, is well described in [281]. The image should resolve a minimum of 100pixels of along the iris radius, from the pupil boundary to the white boundary.Some approaches capture the image from a distance of 15 to 46 cm with aresolution of 100 to 200 pixels for the diameter of the iris, and some othercapture the iris from 20 cm with a resolution of about 256 pixels. A newacquisition device captures the iris pattern from a distance of 3 meters, witha similar resolution as traditional systems, while the subject is on the move[149], but no real performance evaluation was conducted with this prototypedevice. Originally, visible range illumination was used during the acquisitionstage. Recently, the near infrared (NIR) illumination, in the 700nm-900nmregion was proposed to cope with de variability of ambient illumination. Amonochrome CCD camera is used to acquire high-resolution images. Due to theuse of NIR illumination, even darkly pigmented irises reveal rich and complexfeatures. The capture device has also to be able to localize first the eyes inthe face, and has thus to be a wide-angle camera. It is why, most of the irisrecognition systems usually use two different cameras, for this two differentpurposes.

The“Independent Testing of Iris Recognition Technology”, conducted by theInternational Biometric Group, has evaluated three iris recognition acquisitiondevices [131]: LG IrisAccess 3000 Enrollment Optical unit, the OKI IRISPASS-WG and the Panasonic BM-ET300. The lower failure to enroll rate obtainedwith one of the devices was 1.61% for both eyes, and 8.50% for a single eye,while the worst failure to enroll rate obtained with one of the devices was 7.05%for both eyes, and 9.63% for a single eye. The failure to acquire rate when nosamples were acquired on the transaction level was between 0.32% and 0.69%,while the failure to acquire rate when at least one sample of the right eye orat least one sample of the left eye were not acquired on the transaction levelwas between 0.91% and 0.1.77%. The enrollment duration is about 35 secondsfor two of the devices and about 50 seconds for the third device, while theacquisition duration of a successful transaction’s sample was between 1.92 and5.05 seconds.

7.4.1 Ergonomics and acquisition environment

The recognition process with iris pattern needs a good positioning of the eyesin front of the sensor. Some systems inform if the subject is too close or toofar from the sensor. The subjects have also to take off the glasses and contactlenses. Some problems could also appear with blind people, or with individualswith some eye diseases. As some sensors use near-infrared wavelengths, the

87

Page 93: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

ambient lightning will disturb the acquisition process only if it is not too dark.Indeed, darkness can dilate the pupil and hide most of the iris pattern.

7.4.2 Iris acquisition for identity documents

The iris-based recognition technology uses sensor approaches which involve animposing technology and thus is not appropriate for sensor on card-based archi-tecture solutions. But the high computational speed, the non-complexity of thealgorithm approaches and the low memory needed for storing templates allowthe use of such a technology for match on card and data on card architecturesolutions.

7.5 Computational resources

The storage of an iris template, like the IrisCode, is possible, as the template’ssize is very low (about 500 bytes). Furthermore, as the matching algorithm usedin such systems is very fast, it seems to be possible to complete the matchingprocess on the card itself, but not the complete recognition process as the sensor-on-card approach with fingerprint. Recently, PreciseBiometrics 2 has proposed amultimodal match-on-card solution incorporating iris, face and fingerprint [23].

7.6 Open source systems

An open source iris recognition system, implemented in MATLAB R© and de-rived from [181], is available online 3.

7.7 Databases

Some databases were created for commercial use, and thus the data on the sta-tistical properties and singularity of iris patterns, presented in [79], is basedon 9.1 million comparisons. Unfortunately, these databases are not available.But recently, Four iris databases were collected and are available for researchpurposes. The National Laboratory of Pattern Recognition, Institute of Au-tomation from the Chinese Academy of Sciences made available to all interestedresearchers, iris databases : the CASIA Iris Image Databases (ver 1.0 andver 2.0) 4. The version 1.0 includes 756 iris images from 108 eyes, captured intwo different sessions, three samples collected in the first and four in the secondsession. The version 2.0 includes 2400 iris images from 120 eyes, captured by twodifferent devices, each device collecting 20 samples per eye. The Department ofInformatics from the University of Beira Interior (Portugal) made available forbiometric purposes, an iris database containing images with noise: the UBIRISdatabase 5. This database is composed of 1877 iris images, captured in twodifferent sessions, five samples collected on a set of 241 subjects for the firstsession and five samples collected on a set of 231 subjects for the second session

2Precise Biometrics homepage at http://www.precisebiometrics.com/.3Source code available at http://www.csse.uwa.edu.au/~pk/studentprojects/libor/sourcecode.html.4CASIA Iris Image Databases available at http://nlpr-web.ia.ac.cn/english/irds/irisdatabase.htm.5UBIRIS database available online at http://iris.di.ubi.pt/.

88

Page 94: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[226]. The images contain different kinds of noise, simulate a minimal collab-oration from de subjects. The Department of Automation & Computer-AidedEngr., Computer Vision Laboratory, from the Chinese University of Hong Kongis making available for biometric purposes, an iris database: the CUHK 6.This database is composed of 252 iris images, captured in a single session on 36subjects, 7 samples collected for one eye. The Department of Computer Sciencefrom the Palacky Univeristy in Olomouc (Czech republic) is making available forbiometric purposes, an iris database: the UPOL iris database 7. This data-base is composed of 384 iris images, captured in a single session on 64 subjects,3 samples collected for each eye.

7.8 International competitions

The first competition of iris recognition, borrowing the main ideas from theFVC2004, has been organized by the Institute of Automation of the ChineseAcademy of Sciences (CASIA), using two databases including 13200 iris images(20 samples of the eyes of 330 volunteer, under two different illumination set-tings). As the other international biometric competition, each participant hadto submit executable algorithms. The results were firstly presented at the FifthChinese National Conference on Biometrics Recognition, in December 2004 inChina, but no proceedings paper was published on this topic. To the best ofour knowledge, only four iris recognition systems have participated to this com-petition. The first two best iris recognition systems have obtained an averageequal error rate for both databases of 4.25% and of 12.48%, while the worst twosystems have obtained an average equal error rate of 19.88% and of 21.07% 8.

To avoid this lack of international evaluation and to assess the performanceof state-of-the-art iris recognition solutions, the NIST will conduct the Iris Chal-lenge Evaluation 9. The first phase of this evaluation consists in a distributionof an iris recognition challenge problem from August 2005 until June 2006. Thesecond phase consists in a large-scale iris technology evaluation, measured onsequestered data and will take place in the first quarter of 2006.

From January until August 2006, the NIJ-TSA Iris Recognition Study 2006(IRIS06) focused on“standards-based performance and user cooperation studiesof commercial iris recognition products” 10. The aim of thsi study are:

- evaluation of the performance and interoperability of iris recognition prod-ucts using international and US iris image data interchange format stan-dards;

- investigation of hte influence of test subject presentation parameters, suchas pose and eye-gaze angles, and of the ability of iris recognition technologyto acquire and recognize non ideal iris images.

6CUHK iris database available online at http://www2.acae.cuhk.edu.hk/%7Ecvl/main_database.htm.7UPOL iris database available online at http://phoenix.inf.upol.cz/iris/.8These average results were sended to us courteously by Mr. Qiu Xianchao, from the

National Laboratory of Pattern Recognition - Institute of Automation, of the China Academyof Sciences.

9ICE homepage at http://iris.nist.gov/ICE/.10IRIS06 homepage: http://www.authenti-corp.com/iris06.

89

Page 95: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 8

Modality: On-line signature

8.1 Introduction

3000 4000 5000 6000 7000 8000 9000 100002000

3000

4000

5000

6000

7000

8000

9000

10000

x coordinate

(a) x-y plane representation.

0 50 100 150 200 250

4000

6000

8000

X

0 50 100 150 200 250

4000

6000

8000

Y

0 50 100 150 200 250

500

1000

Pre

ssur

e

0 50 100 150 200 25011001200130014001500

Azi

mut

h

0 50 100 150 200 250500

600

700

Ele

vatio

n

sample number

(b) x, y, pressure, azimuth and elevationsignals.

Figure 8.1: Example of a signature realisation from the SVC 2004 corpus.

On-line handwritten signatures are those for which the pen trajectory anddynamics are recorded during the signing. Recorded information typically con-tains pressure, pen azimuth and pen elevation in addition to the trajectory onthe pen tablet. The raw signature data is typically preprocessed by translation,rotation and/or scaling [100] to afford some degree of invariance to changes witheach realisation. Some algorithms also require segmentation, for instance into”components” that are perceptually significant according to a motor model [43].

Features can be extracted from the pre-processed signature, which can thenbe represented according to two broad paradigms [218]. In the function-orientedparadigm, signals extracted from signature data (such as pressure or velocities)are considered as functions of time, the values of which directly constitute thefeature vectors. In the parametric paradigm, local or global parameters arecomputed from the measured signals and used as features. Local parametersrepresent properties of sampling points, pen strokes, or components and can beextracted from local segments of the signature; examples of local parametersare the starting angle at initial pen-down, a segment’s local radius of curvature,and so on. Global parameters concern some property of the complete observed

90

Page 96: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

signature; for instance the total signing time, pen-up to pen-down ratio, bound-ing box aspect ratio and so on. A list of commonly-used global features and analgorithm to perform feature selection can be found in [151, 154]

Most works on on-line signature verification propose to use exactly the sameset of features for every user, but some studies have reported gains in perfor-mance from using user-dependent feature sets, arguing that this accounts forsigner intra-variability [165].

8.2 Overview of algorithmic approaches

Over the past 30 years, numerous algorithms and models have been developedto verify on-line signatures. While many algorithms rely on a temporal rep-resentation of the signature, some authors (notably Nalwa [199]) suggest thaton-line signatures should be parameterized spatially. Currently, the lowest errorrates are achieved by hidden Markov models using mixture of Gaussians outputdistribution, and Gaussian mixture models.

8.2.1 Dynamic Time Warping

The most widely studied on-line signature verification method is elastic match-ing (string matching) using dynamic time warping (DTW), also called dynamicprogramming (DP). Originally used in on-line signature verification by Satoand Kogure in 1982 [247], DTW has been gradually refined over the years. Twomain approaches are seen in published literature: in the first the data pointsare used directly for matching after preprocessing (typically including subsam-pling), while in the second the signature is segmented according to perceivedimportance of boundary points.

Sakamoto et al. [245] have used position, pressure, and pen inclination toachieve 3% EER using three signature realisations templates per user with a 8-users corpus comprising a total of 1066 authentic signatures and 1921 forgeries.Jain et al. [140] have used a mixture of global features such as the numberof strokes and local features, both spatial (e.g. x and y coordinate differenceswith respect to the previous point) and temporal (e.g. relative speed betweenpoints). They achieve about 2.2% EER using between three and five signaturerealisations templates per user with a 102-users corpus comprising a total of1232 authentic signatures and 60 forgeries, thus probably underestimating theFAR.

Yanikoglu and Kholmatov [296], the winners of the signature verificationcompetition 2004 (see Section 8.8), have used a Dynamic Time Warping toalign signatures based on two local features (∆x and ∆y), after which theycompute three distances with respect to that user’s training set, perform PCAto decorrelate the three distances and classify on this last measure. They report1.65% FRR and 1.28% FAR on a 94-users database, using 8 signatures per userfor the user models and holding out 2 and 7 signatures for testing, thus testingwith 182 authentic signatures. 313 skilled forgeries are used for testing.

Recently, DTW has been used as one of the classifier in a multi-classifierscheme [198]. It has also been used as the main classifier in a multi-stageverification system which was tested on 121 users with 726 authentic signaturesand 89 forgeries, obtaining about 0.23% FAR at 3.63% FRR [229].

91

Page 97: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

8.2.2 Hidden Markov models

Inspired by the successful application of Hidden Markov models (HMMs) to on-line character recognition [162], HMMs have now become the best-performingmodels for on-line signature verification. The most commonly used similaritymeasure for HMMs is the log-likelihood ratio of the test signature given the usermodel to the test signature given the background model.

Yang et al. [295] have used quantised angle sequences as features, tryingseveral HMM topologies and number of states. The best results, 3.8% EER,are obtained with a 6-states, left-to-right with skips topology, using 8 trainingsignature realisations per model with a 31-users corpus comprising a total of496 authentic signatures. The results are given for random forgeries.

Kashi et al. [151] have used a mixed-model approach, where global featuressuch as average horizontal speed are combined with a variable duration discreteoutput HMM using inclination angles (with respect to the horizontal axis) andthe difference between adjacent inclination angles as feature vectors. The re-ported error rate for a 20-states, left-to-right with no skips topology is 2.5%EER, using 6 training signature realisations per model with a 59-users corpuscomprising a total of 542 authentic signatures and 325 forgeries.

Yoon et al. [299] transform the data into polar space with speed information,and further use vector quantisation to generate the feature vectors. A 5-states,left-to-right with skips HMM is used for verification, resulting in 2.2% EER using15 training signature realisations per model, with a 100-users corpus comprising2000 signatures. The results are given for random forgeries.

Richiardi and Drygajlo [236] have used a GMM (1-state HMM) with 64diagonal-covariance mixture components to model local features and their timederivatives, and have obtained 1.7% EER on a 50-user subcorpus of the MCYTdatabase, which provides 25 authentic signatures and 25 skilled forgeries foreach user, training with 5 signatures per user and thus testing with a total 1000authentic and 1250 forgeries.

Fierrez-Aguilar et al. [95] have used a 2-states, 32-mixtures per state, left-to-right HMM to model local features, and used score normalisation to obtain5.79% EER on the SVC2004 development set (40 users, 20 authentic and 20forged signatures per user, training with 5 signatures).

These results show that signature verification algorithms based on HMMshave the potential to perform as well or better than those based on DTW or avariant thereof.

8.2.3 Neural networks

Neural networks have been explored for on-line signature verification but theperformance reported in published literature is inferior to other methods suchas DTW, HMMs or GMMs. Chang et al. [58] have used a Bayesian neural net-work trained with incremental learning vector quantisation. The EER achievedon chinese signatures is about 2.3%, using 4 signatures per user model . The80-user corpus comprises a total of 800 authentic signatures and 200 skilledforgeries. Wu et al. [293] have used linear predictive cepstral coefficient derivedfrom the x, y trajectory of the pen to train single-output multi-layer perceptrons(MLPs). Each ”word” (chinese character) of a user’s signature is modeled inde-pendently by an MLP. The EER achieved on chinese signatures is 4.3%, using

92

Page 98: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

an average of 12 authentic signature realisations and 12 forgeries to train eachuser’s MLPs. The 27-users corpus comprises a total of 810 authentic signaturesand 638 forgeries. It is not clear how this system would be applied to romancharacter-based signatures, where the relationship between the real letter andthe signature-style letter is more ambiguous.

8.2.4 Euclidean distance

Euclidean distances or other distance measures have been used for on-line sig-nature verification, generally achieving performance inferior to DTW, HMMs orGMMs. Rhee et al [235] use a model-based segmentation step prior to comput-ing an Euclidean distance to a reference signature for each user. This results inan EER of 3.4%, using 10 signature realisations to build a reference signaturewith a 50-users corpus comprising a total of 1000 authentic signatures and 1000very skilled forgeries.

Kashi et al. [152] have also used Euclidean distance with global and localfeatures.

8.2.5 Regional correlation

The regional correlation approach has many proponents [210]. Nalwa [199] usesa function-based approach where the signature is parameterised using functionsof arc-length, then cross-correlating these functions with each user’s functionprototype in her signature model. This achieves 3.6% EER on average over 3different databases, amounting to a total of 204 users, 2676 authentic signaturesand 1150 forgeries. Each user model was built using six signature realisations.While the corpus size is larger than what is used in most research papers, somepruning occurred which caused some inconsistent genuine signatures to be re-jected.

Lau, Yuen and Tang [163] have used a correlation-based approach and achievedabout 1.7 % EER on a database of 100 persons, where each person contributes5 authentic signatures and is forged 3 times. Each user is modelled using 2 sig-natures, thus resulting in testing with 300 authentic and 300 forged signatures.

8.3 Performance

The current state of the art EER for medium-to-large databases (above 50 users)lies between 0.5% and 5%. A problem in comparing performances is that littlework has been published on standard databases (with the notable exception ofthe SVC effort), thus some databases may be easier than others, for examplebecause forgers are not as talented, or have more or less information about theirtargets. Environmental noise is absent from signature data, and performancedrops will occur mostly due to ergonomical issues (tablet position, writing space)and inter-session time.

Health-related issues can affect handwriting, for example most patients thatare not responsive to treatment with neuroleptics will exhibit symptoms of mi-crographia, whereby the writing size is significantly reduced [50].

Some percentage of the population will also exhibit dyspraxia (difficulty inplanning and carrying out complex movements), meaning more variability can

93

Page 99: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

be expected in signature realisations. These factures contribute to decreasingsignature modality performance and are likely underestimated by the bias inpopulation sampling present in most academic databases.

8.4 Sensors

Sensors for on-line signature acquisition typically divide the acquisition part intwo components: the pen itself and the surface on which the pen writes. Theelectronics can be contained entirely within the writing surface (as in the caseof PDAs), entirely within the pen (as in some research prototypes), or dividedbetween the pen and the writing surface (as in the case of pen tablets commonlyused by digital artists).

8.4.1 Electronics in the writing surface

Most modern PDAs (3COM/PalmOne’s Palm, Sony’s Clie, HP/Compaq’s Ipaq,and numerous others as well as a recent portable game machine by Nintendo(Nintendo DS)) propose pen-input as a modality, generally coupled with more-or-less natural handwriting recognition. Some laptop computers (Toshiba Satel-lite 5200 series) also propose that the touchpad double as signature input hard-ware. These sensors typically report only (x, y) coordinates, and a binary pres-sure value. Pen orientation (azimuth and elevation) are not reported. Withcurrent processor speeds routinely in the MHz range for these devices, embed-ded signature verification becomes an attractive prospect.

“Dumb pens” offer the advantage that pen replacement in case of loss isvery simple, and could be key to a more widespread acceptance of signatureverification technologies in fields such as banking.

However, the lack of pressure information (and for some algorithms, of penorientation information) is a serious drawback for on-line signature verification,which performs better if that signal is provided.

8.4.2 Electronics in the pen

While this approach is not the most commonly seen in recent research, thebenefits of having a single device to acquire signature (portability, no need forspecial surface, possibly lower costs in volume production) have pushed severalgroups to investigate the feasibility of a standalone pen that does not need aspecific surface to acquire dynamic handwriting signals.

In 1983, Plamondon et al. have proposed a self-contained pen, fitted withtwo pairs of orthogonal accelerometers and a pressure switch [217], based on amathematical model of handwriting.

The SmartPen developped by Reynaerts and others [231] acquires x, y andz forces as well as elevation and azimuth.

More recently, Hook et al. have proposed the Biometrical Smart Pen1 [120],which exists in several variants. The mechanical pen acquires forces in the x,y, and z (pressure) direction, as well as elevation and azimuth. Furthermore, itis fitted with a fingerprint sensor. Another version is a microphone-based pen,

1More details at http://www.bisp-regensburg.de.

94

Page 100: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

which can be used to perform handwriting analysis from the writing sounds. Thedata transfer from the pen to a computer can be wired or wireless (bluetooth).

8.4.3 Electronics both in the pen and the surface

General-purpose sensors suitable for on-line signature acquisition are called pentablets or graphic tablets. A research prototype has been used for signatureverification in 1977 by Herbst and Liu [113], providing x, y positions and bi-nary pressure information. Nowadays, many manufacturers supply tablets, forinstance Wacom, Aiptek, AceCad, Genius, GTCO CalComp, and others.

In the Wacom implementation (Graphire, Intuos, Volito, and other tablets),the surface of the pen tablet sends electro-magnetic signals which power the pen,thus avoiding the use of batteries in the pen (other vendors, such as AceCador Aiptek, require batteries to be inserted in the pen). A capacitive pressuregauge in the pen tip as well as two additional sensors provide pressure andazimuth/elevation data. The tablet also determines the x, y position of the pen.Wacom tablets have a resolution of 2540 or 5080 lines per inch, 1024 pressurelevels, and can sample at up to 200 Hz.

The AceCad AceCat range tablets have 2540 lines per inch resolution, 1024levels of pressure, and sample the pen data at 125 Hz.

The HyperPen series of Aiptek offers 3048 lines per inch resolution, and 512pressure levels.

The prices for general-purpose pen tablets suitable for on-line signature ver-ification starts at about CHF 90.

Dedicated signature pads also exist, some of them showing immediate feed-back through a LCD placed directly on the writing surface. One of the mostwidely used is Interlink/Integrisign’s ePad2, offering a resolution of 300 linesper inch, 128 levels of pressure and a sampling frequency of 100 Hz. It is shownon Figure 8.2. An interesting model in the ePad series is the ePad-ID whichcombines a fingerprint sensor and a pen tablet. None of the ePad series canacquire azimuth and elevation data.

Figure 8.2: Dedicated signature sensor (Interlink’s ePad).

MotionTouch provides a range of dedicated signature-capture sensors (withor without LCD feedback), having 500 or 1000 lines per inch resolution, 512

2http://www.integrisign.com/products/epad.html.

95

Page 101: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

levels of pressure, and 100 to 200 Hz sampling. The Legapad model was adoptedby the UK bank Nationwide Building Society.

8.4.4 Ergonomics and acquisition environment

The angle of the signature should be kept constant with respect to the acquisi-tion surface, or rotation normalisation should be performed as a pre-processingtask, which unfortunately can result in slightly decreased verification perfor-mance.

The size of the signing area should be kept constant, to avoid resampling ofthe signature which can also be detrimental to verification rates.

There seems to be no significant difference between signatures acquired withthe user sitting and those acquired with the user standing [87].

Visual feedback is important for signature tasks. Thus, the user should beable to see the signature appear under the tip of the pen as she signs. Tothis end, a sheet of paper can be placed between the tip of the inking pen andthe tablet surface, to provide more friction and a natural feel to the writing.Furthermore, this enables researchers to also investigate off-line signature at alater stage should they wish to do so and leaves a (legally binding) written proofof the transaction, which could be useful in applications such as e-voting.

8.4.5 Online signature acquisition for identity documents

To summarise, a general-purpose pen tablet device should be used as they arewidely available and the price is fairly low; an A6 tablet is sufficient for acqui-sition. Feedback should be provided, either in the form of an immediate LCDresponse or a simple sheet of paper. The writing area should be constrainedby a box, which will fix both maximum size and orientation of the signature.Provision should be made for left-handed users to fix the tablet at an appropri-ate angle. Similarly, vertical-style signatures (for example chinese or japanese)should be accommodated for by providing a second set of box sizes on a sheetof paper.

8.5 Computational resources

One signature datafile comprising x, y, pressure, azimuth and elevation datais typically between 5 kB and 10 kB. More efficient file formats can be used,and compression ratios of about 3:1 can be obtained by using a general-purposecompression such as Lempel-Ziv. Depending on the classifier type and modelparameters, parametric models can typically be stored in 1 kB-2 kB.

Feature extraction is fairly simple in most parameterisations, and can in-volves regression computations (for velocity, acceleration, and other derivativecomputations). The feature vector will typically comprise between 1 and 15dimensions. Once the features are extracted, modeling for instance with mix-tures of Gaussians will involve several iterations of the EM algorithm. Scoring apattern will involve evaluating likelihoods on each point of n-dimensional dataand summing the results.

Thus, because of the small size of the features extracted from a signatureand the possibility to achieve low error rates with simple classifiers, it seems

96

Page 102: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

match-on-card would be feasible for the signature modality.

8.6 Open source systems

Although no open source on-line signature verification reference systems areavailable to the best of our knowledge, numerous statistical and other modelingtoolkits are freely available and can be used to implement published algorithms.

8.7 Databases

Not many databases are publicly available to on-line signature verification re-searchers. Most research groups develop their own database.

MCYT MCYT [207] contains signature and fingerprint data for 330 users. A100-users subset of this database is available to the members of the EuropeanBioSecure network of excellence. Each user provides 25 authentic signaturesamples (x, y, pressure, azimuth and elevation), and is forged 5 times by 5different users. The forgers are given time to practice on their target and areshown a static image of the target’s signature.

SVC2004 SVC2004 [298] is divided in two parts: an evaluation set of 40 userswhich is freely available, and a sequestered set used in the competition, whichis not distributed. Each user contributes 20 signatures, and is forged 20 times.The data is acquired in two sessions at least a week apart. The forgeries are per-formed by at least 4 different forgers, which are allowed to practice by watchinga dynamic replay of the signing sequence. The data contains (x, y, pressure,azimuth, elevation, pen down status, time stamp) signals. A noteworthy infor-mation is that this for privacy reasons, users were advised not to contribute theirreal signatures so this database contains alias signatures. This means the intra-user variability is probably overimportant. Also, this database contains bothchinese-style (ideograms) and latin-style (left-to right latin alphabet) signatures.

Philips Laboratories Philips Laboratories [86] contains 51 users, and eachuser provides 30 signatures. There are also 3000 amateur forgeries (practicedbased on static image and over-the-shoulder), and 240 professional forgeries(contributed by forensic document examiners). This database is not generallyavailable.

8.8 International competitions

The only international competition so far is the Signature Verification Compe-tition 2004 [298]. It defined two tasks, task 1 with only (x, y, pen down status)information and task 2 with additional pen pressure and orientation signals.Task 1 is suitable for PDA-type devices which report only coordinates and bi-nary pressure (see section 8.4), while task 2 is suitable for higher-end devices.It is interesting to note that contrary to most previously published results, theresults for task 1 were marginally worse than the results for task 2.

97

Page 103: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

The testing protocol for task 2 is suitable for testing inclusion of on-linesignatures in biometric identity documents, but the database itself, acquisitionmethodology, and forgery methodology are not appropriate to this application.

98

Page 104: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 9

Modality: Speech

9.1 Introduction

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 104

−0.4

−0.2

0

0.2

0.4

0.6

sample number

norm

alis

ed s

ampl

e am

plitu

de

Figure 9.1: Speech waveform for the TIMIT sentence “she had your dark suitin greasy washwater all year”.

Automatic speaker recognition was pioneered in 1970 by Doddington [85],and subsequently became a very active research area. Today, speaker recognitionsystems and algorithms can be subdivided into two broad classes:

Text-dependent systems rely on the user pronouncing certain fixed utter-ances, which can be a combination of digits, a password, or any other phrase.Thus, the user will prove her knowledge of the passphrase in addition to provid-ing her biometrics. Text-prompted systems are a special kind of text-dependentsystems which ask the user to pronounce a certain utterance which may not beknown in advance, to deter recording and replaying of the user’s passphrase.

Text-independent systems allow the user to pronounce any utterance oftheir choosing.

99

Page 105: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

9.2 Overview of algorithmic approaches

An article that compares the hidden Markov model, dynamic time warping andvector quantisation approaches is [300]. Another comparing vector quantisationand dynamic time warping is [13].

9.2.1 Dynamic time warping

Taking into account the dynamics of speech parameters has been proposed inthe eighties and seen many subsequent refinements. A useful technique in thiscontext is Dynamic Time Warping (DTW), which allows for compensation ofthe variability of speaking rate inherent to human speakers. Dynamic TimeWarping has relatively low computational requirements, and is mostly used fortext-dependent verification. Nowadays, DTW is less frequently used as a stand-alone speaker recognition algorithm [209], but rather as a way to supplement thedecision process with auxiliary information. Recently, DTW has been used tomodel pitch contours as auxiliary information, providing improved recognitionrates [4], and as part of a multi-model speaker recognition system [91].

9.2.2 Vector quantisation

Vector quantisation (VQ) for speaker recognition has been proposed and testedfor a digit-based system over a 100-users database in 1985 [258], and has seen lit-tle use recently [193]. This approach is not commonly used anymore for speakerverification because it is consistently outperformed by statistical methods, whichdo take into account feature overlap and correlations by incorporating covari-ance information. However, VQ can outperform statistical methods when littledata is available. [183]

9.2.3 Statistical methods: HMM

Probabilistic methods rely on a parametric modeling of the speech signal. Themodeling can be time-dependent (hidden Markov models) or not (Gaussian mix-ture model (GMM)). The value of model parameters have to be learned fromtraining data, which is a critical point in probabilistic methods: sufficient train-ing data has to be obtained. HMMs are very commonly used for text-dependentsystems, where scores are typically obtained by finding the best path throughthe states. Ergodic (fully connected) HMMs have also been used for speakerrecognition [183].

Poritz proposed using HMMs (5 states) to model speakers in 1982 [220],and performed identification on 10 speakers which resulted in no error. In 1991Rosenberg et al. [239] used speaker-dependent whole word (digits) models, andtested on a 10-users population.

9.2.4 Statistical methods: GMM

Schwartz, Roucos and Berouti first proposed probabilistic modeling for speakerrecognition in 1982 [252]. In 1995, Reynolds [232] proposed using a mixture ofGaussian models (termed MoG or more commonly GMM) for modeling speech

100

Page 106: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

features belonging to a particular user. This approach has proved very suc-cessful and GMMs are now the dominant model for speaker recognition, oftenin combination with higher-level information provided for instance by DTW.A further refinement on the GMM method comes in the form of the universalbackground model (UBM) [234]: a large amount of data from many speakers isbundled together and a high-order GMM (typically 512 to 2048 mixture compo-nents [28]) is trained on that data. Then, a limited amount of speaker-specificdata is used to adapt the UBM to each speaker. Essentially, the idea is to usea well-trained model (the UBM) as a good basis for initialisation of the usermodels. The vast majority of speaker recognition systems today are based onGMMs.

9.2.5 Neural networks

Neural networks have sometimes been used for text-independent speaker recog-nition, trained by providing both client and impostor data. Oglesby and Masonfirst proposed a multi-layer perceptron (MLP) neural network with LPC-cepstralcoefficients in 1988 and 1989 [204, 205], then expanded their work to a radialbasis function network in 1991 [206] with better results than both VQ andMLP approaches. In [98], a radial basis function neural network is used forspeaker identification on the TIMIT and NTIMIT databases. More recently, anauto-associative neural network has been tested on part of the NIST 2002 SREdatabase [111].

9.2.6 Support vector machines

Support vector machines (SVM), an approach that has successfully been appliedto many pattern recognition problems, has also been used in speaker recognition.

Schmidt and Gish proposed in 1996 to use support vector machines to per-form speaker identification [249]. They tested their approach on a 26-users sub-set of the switchboard corpus and reported better results than with GaussianMixture models. In 2001, Gu and Thomas [109] reported improvements overGMMs by using SVMs for a 250-speakers phone-quality database. More re-cently, Wan and Renals [277] have also reported better results for SVMs thanfor GMMs.

9.3 Performance

As for the case of face verification, error rates of speaker recognition systems arevery dependent on the application, and good results in an evaluation campaignor in vendor specifications do not mean that these will be obtained in the field.

While speaker recognition research has been going on for some time andspeech processing is a mature field, many problems remain unsolved. We willlist below some of the problems inherent to this modality, which are in additionto the inter-session variability shared with other biometric modalities.

Channel (convolutional) noise will distort speech signals as soon as theyleave the speaker’s mouth. All microphones have their specific transfer func-tions, most of the time non-linear, and reverberation in a room will also alter

101

Page 107: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

speech. It is known that speaker recognition performance degrades significantlywhen the enrollment and deployment conditions are not matched. Noise ro-bustness techniques used in speech processing for speech recognition (such ascepstral mean normalisation) can often be applied to speaker recognition. Com-pensation techniques derived from forensic speaker recognition [39] can also beapplied to the biometric case.

Environmental (additive) noise is added to the speech signal by otheraudio sources surrounding the speaker, for example car noise, interfering speech,background music, etc. In general, at low signal-to-noise ratios the error rates ofspeaker recognition systems drop significantly. Again, experience in other fieldsof speech processing can be drawn upon and applied to speaker recognition. Arecent approach [237] is to ignore classifier decisions when they are deemed toounreliable and to acquire a new presentation in this situation.

Emotional state and health factors come into play because speech is apartly behavioural, partly physiological modality. The emotional state of usersis known to alter the characteristics of the vocal tract. Many diseases, includingthe common cold, can change the voice of a person so much as to make it un-recognisable even by human listeners. Alcohol ingestion can also lead to signifi-cant changes in speech for some users, notably in fundamental frequency [117].

Since speaker recognition performance depends so much on acquisition andtesting conditions, it is nearly pointless to provide approximative error rateranges and the latter are not directly comparable to other modalities. Interna-tional competitions enable more accurate comparisons.

9.4 Sensors

Numerous transducers exist to transform the acoustic pressure wave producedby the speech aparatus into electrical waves. The most common types are:

Moving-coil (dynamic) Moving-coil microphones have a light diaphragmattached to a conductive coil which is placed in the gap of a magnet. When thesound pressure waves hit the diaphragm, it moves accordingly and displaces thecoil, which in turn induces current. Many dynamic microphones have a peak inthe frequency response around 5 kHz, and a fall-off from around 8 kHz due tothe mass of the coil-diaphragm assembly, though some higher-quality dynamicmicrophones have a flatter frequency response [244].

Ribbon Ribbon microphones have a strip of metal placed between the twopoles of a magnet. When sound pressure waves hit the ribbon, it moves in themagnetic field and a current is induced through it. This type of microphone hasa flatter frequency response than moving-coils, with a roll-off below about 40Hz and above about 14 kHz.

102

Page 108: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Capacitor (condenser) The capacitor microphone has a fixed backplate,which is charged (permanently in the case of electret microphones), and a mov-ing diaphragm constituting the other plate. When sound pressure waves hitthe diaphragm, the capacitance of the assembly varies, meaning that the volt-age across the capacitor varies in proportion. The main advantage of capacitormicrophones is that the mass of the diaphragm is very small, thus allowing itsgood frequency response throughout, with a small peak in resonance between12 kHz and 20 kHz.

Bauer [21] wrote a review of many types of microphone (including carbonmicrophones!) and their historical development.

Three important characteristics of a microphone are the directional response,frequency response, and sensitivity. The directional response quantifies the out-put level of the microphone at different incidence angles of the sound pressurewave with respect to the capsule’s front. An omnidirectional response meanssounds are picked up from all directions with equal sensitivity. It is important toknow that the directional response of a microphone changes with the frequency,for instance omnidirectional microphones tend to become more directional withhigher frequencies.

The frequency response quantifies the gain of the microphone over a rangeof frequencies. Ideally, the response would be perfectly flat meaning that allfrequencies would be transduced into current in the same way. Practically,microphones will deviate from that and typically quote the frequency responseover a range of frequencies with a maximum deviation from the response at1kHz, for example “40Hz-14kHz (±3dB)”.

The sensitivity of a microphone indicates how much voltage will ensue froma given sound pressure level (with a reference to 1 V at 94 dB SPL).

9.4.1 Ergonomics and acquisition environment

To avoid picking up environmental noise both in enrollment and in deployment,a directional microphone should be used (cardioid or hyper-cardioid), and ideallya sound-proof booth would be used (which is unfortunately unpractical).

The position and distance of the mouth with respect to the microphoneshould be kept constant.

9.4.2 Voice acquisition for identity documents

About 30 seconds to 1 minute of speech should be collected from the user. Thisshould ideally be phonetically-rich sentences in the native language of the user.

9.5 Computational resources

Raw speech data takes an amount of space depending on the sampling rate,quantisation levels, and number of channels (mono most of the time). Thus,a 16-bits, 16 kHz sampled speech signal will take about 31 kB per second ofspeech. Speech parameters (for example MFCCs) will compress the signal by afactor depending on the output frame rate and the number of features selected.

103

Page 109: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

For example, a 39-coefficients parameterisation with a 10 ms frame rate resultsin about a 2:1 compression for the above example.

The memory needed for template storage depends on the template type.For GMM-based models, the size of the model will depend on the number ofGaussian components and the type of covariance matrices, as well as parametertying. A typical size for a small (32-64 components) model would be between 5kB and 10 kB without Lempel-Ziv compression.

Depending on the implementation of the matching algorithm (an idea is notto load the speech features for the whole utterance at once, but “stream” themto the chip), match-on-card could be feasible for the speech modality.

9.6 Open source systems

The LIA RAL package 1, by the Universite d’Avignon in France, is an opensource speaker verification software implemented in C++ and based on theAlize toolkit [37]. It can read several feature types (HTK, SPRO, raw), runsreasonably fast, and has been used in NIST evaluations. Thus, it can be a goodbasis with which to compare other systems.

9.7 Databases

Many databases are available freely or at low cost for speaker recognition tasks [53,104, 187]. However, many are geared towards telephone-quality speech and thusof not practical interest to identity documents, where it is anticipated acquisi-tion will only occur through microphones and not telephone channels. Thus, wedo not present PolyCost, SIVA, HTIMIT, LLHDB, Switchboard, OGI SpeakerRecognition Corpus, YOHO, etc. Below, we focus only on those that may berelevant to our application.

AHUMADA AHUMADA [208] contains speech for 104 male users and 80females users, captured with 4 different microphones and more than 10 differenttelephone handsets, sampled on DAT tape. The data consists of isolated digits,strings of digits, phonetically balanced sentences, read text at various speakingrates, and spontaneous speech. The data was recorded in 6 different sessionsdays or weeks apart. A subset of this data (electret microphone) can be usedfor initial testing of identity documents, but more users are needed.

BANCA BANCA [15] contains speech data for 208 users, captured with 2 dif-ferent microphones (one high-quality and one low-quality) in 12 sessions (threeacoustical conditions). The data, about 40 seconds per session, consists of iso-lated digits and spontaneous speech, and is sampled at 32 kHz, quantised at 16bits per sample, and recorded in mono. This database offers a sufficient amountof users for perform initial testing for identity documents, but the acoustic con-ditions of the recording may not match the identity documents application.

1Available at http://www.lia.univ-avignon.fr/heberges/ALIZE/LIA_RAL/index.html.

104

Page 110: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

EUROM1 EUROM1 (the multilingual European speech database) [57] con-tains speech data in 7 languages (Danish, Dutch, English, French, German,Norwegian, and Swedish), with 60 users per language. Most users were onlyrecorded once, but some were recorded on different days; this is not consistentfrom country to country The data contains numbers and read speech, with em-phasis on phonetic balancing. The data is sampled at 20 kHz and quantised at16 bits, and recordings take place in an anechoic room. Laryngograph data isalso available for some subset of the data. While the total population is large,language effects may prevent this database from being used for speaker verifi-cation evaluation; furthermore, the inter-session time is not strictly controlledas this database was not originally meant for speaker verification tasks.

King-92 King Speaker Verification (King-92) [116] contains speech data from51 male users, recorded over 10 30-to-60 seconds sessions acquired weeks ormonths apart. The data is acquired with both telephone handsets (the recordingquality varies depending on the location of the recording due to equipmentdifferences) and a wideband microphone in a sound booth. The data is sampledat 8 kHz (originally 10 kHz in 1987 but resampled) and 16-bits quantised. Thisis not suitable for our identity documents purposes because the gender balancehas to be representative of that found in the Swiss population, and the numberof users is too limited.

STC STC Russian Speech Database [260] contains speech data from 89 users(54 males and 35 females), recorded over 15 or less 25-seconds sessions acquiredwithin 1 to 3 months. The data is acquired using a high-quality, omnidirectionalmicrophone in an office setting. The data contains 5 read sentences per session,is sampled at 11 kHz and quantised to 16 bits by a low-quality PC sound card.This can also be used as an initial development set for our application, thoughlanguage effects may be a problem as the data is in Russian.

TIMIT TIMIT Acoustic-Phonetic Continuous Speech Corpus [103] containsspeech data for 630 users (438 males and 192 females), recorded over a single30-to-40 seconds session. The data is acquired in a sound booth, sampled at16 kHz and quantised to 16 bits. The data contains phonetically-balanced readsentences in american English. The main problem with this data is that it hasbeen recorded in one session and thus inter-session effects can not be evalu-ated. Furthermore, the amount of data per user is fairly limited. Campbelland Reynolds [53] advise against use of this database for speaker verificationevaluation.

TSID TSID Tactical Speaker Identification Speech Corpus contains 40 users(39 males, 1 female) recorded in a single session. The data is acquired in open air,using military radio handsets and an wideband electret microphone. The datacontains phonetically-balanced read sentences, digit strings, and spontaneousspeech. This database is not suitable for our application because it is mono-session and gender-imbalanced. Furthermore, the open air environment doesnot correspond to the anticipated deployment environment.

105

Page 111: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Verivox Verivox [150] contains 50 male users recorded in a single 30-minutessession. The data is acquired in a sound booth, using a high-quality microphone.The data consists of digit sequences in Swedish. The data is sampled at 22 kHz,then downsampled to 8 kHz and quantised using 8 bits A-law companding.While a large amount of data is available per speaker, the gender-imbalance,Swedish language and narrowband sampling mean this database is not suitablefor our purposes.

XM2VTS XM2VTS [191] contains 295 users and was recorded in 4 sessionsabout a month apart. It contains about 24 seconds of speech per user, readmaterial (2 digits sequences and one sentence), for a total of total 7080 files.The files are sampled at 32 KHz and 16-bits quantised. While the amount ofdata per user is not very large and could lead to under-trained models, thisdatabase represents a good start for verification tests in our application.

9.8 International competitions

The main competition in speaker recognition is the USA’s National Institute ofStandards (NIST) speaker recognition evaluations series.

The NIST speaker recognition evaluations (SREs) [84] have been going onsince 1996, where the test started with 20 males and 20 females. The 2003evaluation focused on the Switchboard corpus (cellular or PSTN parts), andrecognition of a single speaker with about 2 minutes of training, a single speakerwith about an hour of training data, and a single speaker when the training datahas 2 speakers. NIST SRE’s are strongly focused on telephone speech, and thusnot appropriate for our purpose, though some evaluation methodology can beborrowed.

106

Page 112: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 10

Multimodality

10.1 Multimodality and Identity Documents

As presented in Sections 2.3.2 and 2.10, the use of multiple biometrics is usefulfor increasing the security of biometric systems against spoof attacks and forminimizing the limitations of single biometric, as non-universality and intra-and inter-class variation. In a large scale introduction of biometric informationin identity documents, the use of at least two modalities is recommended, inorder to allow a more robust and more secure verification process between theinformation on the travel document and the supposed owner, as proposed inthe NIST’s joint report to the U.S. Congress and the European Commission’sProposals [201, 88] 1. This multimodal approach is also recommended in orderto avoid penalizing people who do not possess the required biometric.

The use of multiple biometrics, instead of a single modality, in identity doc-uments will not be only beneficial, but will also create some constraints. First,the information size to be stored on the identity document will be increased,whether it is the raw data (as recommended by the ICAO) or the biometrictemplates. A combination into a single template of the biometric features ormodels from different modalities of a single person should be also a way to pro-tect people’s privacy. An example of a template combination of multiple units ofa single modality (fingerprint) is presented in [297] 2. For multiple modalities,a single template from partial biometric features or models will also providean additional security measure for privacy protection, as the information avail-able for each modality taken separately are not enough for an identification /verification process and thus can not be used for other purposes. Then, theuse of multiple biometrics in identity documents will increase the number ofsensor devices and recognition systems required at borders control and in en-rollment centers, in comparison to the use of a single modality. This also meansthe training of additional people, in order to be able to use in a suitable waythese systems. The increase of qualified staff and technical components will thusincrease the overall costs of such an introduction.

1Chapter 11 presents more information about the NIST’s recommendations and the Euro-pean Commission’s proposals.

2Section 10.2.3 presents more details on this approach.

107

Page 113: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

10.2 Multimodal biometric systems and data-bases

The Section 2.10 presents several scenarios and strategies for combining bio-metrics. Some examples will be presented here, according to these categories,with indication, if mentioned, about the modalities, the acquisition protocol,the fusion levels and the fusion approaches used. This review is partially basedon [146, 169, 241, 242].

10.2.1 Single biometric, multiple sensors

In [267], a multimodal Embedded Hidden Markoy Model-based (EHMM) ap-proach was proposed for 2D and 3D face recognition, with a fusion method, aweighted-sum rule after score normalisation, at the match level. The experi-ments were conducted on 3000 images of 50 subjects, acquired in 5 differentsessions. For each session, 12 views per subject were captured, varying lightingand facial expressions. The images consist of grayscale images and the corre-sponding depth map. The fusion of the data is obtained by normalising thescores in order to map them to a same domain and by giving a relative weight,chosen experimentally for each kind of data. By using the combined approach,the authors obtained an improvement of the EER of about 2-5% after fusion,comparing to the monomodal approaches.

In [59], a multimodal PCA-based approach was proposed for 2D and 3D facerecognition, with a fusion method, a weighted-sum rule after score normalisa-tion, at the match level. The experiments were conducted on the images of278 subjects, acquired in two different sessions. The range camera capturedsimultaneously colour images and range data images, thanks the projection ofstriped light. The fusion of the data is obtained by normalising the scores inorder to map them to a same domain and by giving a confidence accordinglyto the distances which separated the top rank and the two follows ranks. Byusing a PCA-based approach for the two kind of data, the authors obtaineda rank-one recognition rate of 92,8% after fusion, while these rates were onlyabout 83,1% for 2D and 83,7% for 3D before the combination. The data usedin this study is a part of the Human ID databases and is available for researchpurposes 3.

In [60], a multimodal approach was proposed for a PCA-based 2D face recog-nition (for visible-light and infrared light), with fusion methods at the decisionand the score level. The experiments were conducted on the images of 240subjects. For each session, four views per subject were captured, varying light-ing and facial expressions. Two cameras were used to acquire the images, along-wavelenght IR camera and a visible-light digital camera. The three fu-sion approaches used were an unweighted rank based strategy, a logarithmicallyrank transformation strategy and a score based strategy. All these combinationsmethods outperforms the monomodal approaches, and the score based strategyoutperforms the other fusion methods.

3Computer Vision Research Laboratory of the Univesity of Notre Dame homepage athttp://www.nd.edu/~cvrl/.

108

Page 114: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

10.2.2 Multiple biometrics

In [142], a multimodal approach was proposed for face, fingerprint and hand-geometry, with fusion methods at the score level. The experiments were con-ducted on the traits of 100 subjects. For 50 subjects, five facial and five fin-gerprint images were acquired by a CCD camera and a fingerprint sensor re-spectively. For 50 other subjects (even if some of them were also present in theprevious set), five hand-geometry images were obtained by a commercial cam-era. All the traits of this first set of subjects were randomly paired. Anotherdatabase of 50 subjects was created for the three traits, captured by a videofor face images, by another commercial sensor for fingerprint, and by the samecamera for hand-geometry. It results in a database of 100 subjects, with 1000genuine and 24’500 impostor score vectors, each vector containing the scores ofthese three modalities. The matching approaches for these modalities are asfollows: minutiae-based matcher for fingerprint, which has as output similarityscores, PCA-based algorithm for face recognition, which has as output an Euclid-ean distance, and a 14-dimensional features vector for hand-geometry, whichhas for output an Euclidean distance. Seven score normalisation techniques(simple distance-t-similarity transformation with no change in scale, min-maxnormalisation, z-score normalisation, median-MAD normalisation, double sig-moid normalisation, tanh normalisation and Parzen normalisation) and threefusion techniques on the normalized scores (simple-sum-rule, max-rule and min-rule) were tested in this study. Excepted for one normalisation technique (themedian-MAD), all fusion approaches outperform the monomodal approaches.For example, the fingerprint system, which is the best monomodal system in thisstudy, obtained a genuine acceptance rate of 83.6% at a FAR of 0.1%, while themultimodal approach obtained a genuine acceptance rate of 98.6% at a FAR of0.1% when the z-score normalisation and the sum-rule were used. At low FARs,the tanh and min-max normalisation techniques outperforms the other tech-niques, while at higher FARs, the z-score normalisation performs better thanthe other techniques. By using user-specific weights in the min-max and tanhnormalisation, a significant performance improvement can also be obtained.

In [257], a multimodal approach was proposed for face and fingerprint, withfusion methods at the score level. The experiments were conducted on thetraits of 972 subjects. For face recognition, the FERET image database wasused in order to obtain 2 face images of 972 subjects, while for fingerprintrecognition, a proprietary database was used. This later database contains twofingerprint images, captured by a live-scan, of 972 subjects. All the traits ofthese subjects were randomly paired. Three fingerprint recognition commercialsystems and one face recognition commercial system were used in this study.Seven score normalisation techniques (min-max, z-score, tanh, adaptive, two-quadrics, logistic and quadric-line-quadric) and five fusion techniques on thenormalized scores (simple-sum, min-score, max-score, matcher weighting anduser weighting) were tested in this study. The EER of the best fingerprintsystem and of the face recognition system was respectively 2.16% and 3.76%,while the max-score fusion approach on quadric-line-quadric normalized scoresobtained an EER of 0.63%. Excepted for the min-score fusion approach, all thenormalization-fusion combinations outperform any monomodal systems testedin this study.

109

Page 115: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

The BIOMET database [102] includes five different modalities, face, voice,fingerprint, hand and signature, and was acquired in three different sessions:130 people in the first, 106 in the second and 91 in the last session. An audio-video camera has acquired frontal and sideways shots, using a list of sentencesto pronounce. Furthermore, an infrared camera (1, 5 and 10 images per personin the respective sessions) and a 3D acquisition system based on structuredlight (only 5 acquisitions of 91 people were done) were also used in order to beinvariant of the background lightening. For acquiring 2 dimensional images ofthe hand (with a resolution of 130 to 200 dpi in the fist two session and 300dpi in the last one), a scanner was used (1 image per person in the first twosessions and 3 in the last one). The on-line signature modality was acquired on agraphical tablet with a grip pen during the first session and an ink pen (signingon a sheet of paper, on the tablet) during the two last sessions, capturing fiveparameters, the x, y coordinates, pressure, azimuth and altitude of the pen.Genuine signatures and impostor signatures were acquired during the sessions(15 genuine signatures and 17 impostor signatures per person were acquired,realized by five different impostors. Two different sensors, an optical and acapacitive, were used to capture the middle and index fingers of the right hand(1 image per person in the first session using only the optical sensor, 2 images perperson in the second one and 3 images with the optical sensor and 4 images withthe capacitive sensor per person in the last one). Validation of the collected dataand evaluation of the difficulty of the database were also accomplished (for eachmodality separately), in performing the same algorithms used in the project onother biometric databases. More information about this database is availableonline 4, but the biometric data collected is not yet available.

The BANCA database [15] includes two different modalities, face and voiceand was acquired in four European languages. Two different quality acquisi-tion levels (high and low) and three different recording conditions (controlled,degraded and adverse) were used to capture the modalities of 208 people. Acheap analogue web cam (for the degraded condition) and a high quality digitalcamera (for the adverse and controlled conditions) were used for acquiring theface modality. Two microphones, a poor and a good quality one, were usedsimultaneously for recoding the voice modality. For each of the 12 sessions(4 sessions per condition) per person, every people have recorded 1 true clientaccess and 1 informed impostor attack. This database is available online forresearch purposes 5.

MCYT database [207] includes two different modalities, fingerprint and sig-nature, of 330 individuals in 4 different places (35, 75, 75 and 145 in the respec-tive places). Two different sensors, an optical and a capacitive, were used tocapture for each sensor 12 samples of all the fingers of each person under threedifferent control levels, which are low, medium and high (3 samples in the firsttwo control levels, and 6 samples in the last control level). The on-line signa-ture modality was acquired on a pen tablet, capturing five parameters, the x, ycoordinates, pressure, azimuth and altitude of the pen. Genuine signatures andskilled forgeries, produced by the five subsequent users of the concerning person,

4http://www.int-evry.fr/biometrics/english/index.php?item=1&menu=projects.5http://www.ee.surrey.ac.uk/Research/VSSP/banca.

110

Page 116: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

were acquired (25 genuine signatures and 25 skilled forgeries per person). Thisdatabase is available for research purposes from the Biometrics Research Lab ofthe University of Madrid 6.

In [94], a multimodal approach was proposed including a face verificationsystem based on a global appearance representation scheme, a minutiae-basedfingerprint verification system and an on-line signature verification system basedon HMM modeling of temporal functions, with fusion methods, sum-rule andsupport vector machine (SVM) user-independent and user-dependent, at thescore level. The experiments were conducted on 50 subjects of the MCYTdatabase for fingerprint and signature and on 50 subjects of the XM2VTS facedatabase. All the traits of these subjects were randomly paired. The EERsof the face, the on-line signature and the fingerprint verification systems were10%, 4% and 3%, respectively, while the sum-rule, the SVM user-independentand the SVM user-dependent fusion approaches obtained EERs of 0.5%, 0.3%,and 0.05% respectively.

In [161], a multimodal approach was proposed for palmprint and hand geome-try, with fusion methods at the features level by combinating the feature vectorsby concatenation, and the matching score level by using max-rule. The exper-iments were conducted on hand images of 100 subjects, 10 images for eachsubject, captured by a digital camera (500 images for training and 500 imagesfor testing). The two modalities were derived from the same image. Only thefusion approach at the matching score level outperforms the monomodal sys-tems. For a FRR of 1.41%, the multimodal approach obtained a FAR of 0%,while the palmprint-based verification system, the best monomodal approach inthis study, obtained at a FRR of 2.04% a FAR of 4.49%.

In [241], multimodal approach was proposed for face, fingerprint and hand-geometry, with three fusion methods at the matching score level, sum-rule, de-cision trees and linear discriminant function, after a score normalisation (thescores are mapped to the range [0-100]). The experiments were conducted on aset of 50 subjects, 5 face images and 5 fingerprint images of the same finger werecaptured. Hand geometry images were captured on a second set of 50 subjects.Each trait were randomly paired to obtain a virtual multimodal database. Themultimodal approach with the sum-rule fusion method outperforms the otherfusion strategies, as well as the monomodal systems. At a FAR of 0.03%, thecombination approach obtained a FRR of 1.78%, while the best monomodalsystem, fingerprint approach, obtained at a FAR of 0.01%, a FRR of 25%.

In [278], a multimodal approach was proposed for a PCA-based face verifica-tion system and a key local variation-based iris verification system, with fusionmethods at the matching score level by using unweighted and weighted sum-rules, Fischer Discriminant Analysis and Neural Networks. The experimentswere conducted on two different databases, the first of very high quality andthe second of lower quality. 5 face images for each of 90 subjects of the firstdatabase were collected partially from public available face databases, such as

6Biometrics Research Lab homepage at http://atvs.ii.uam.es.

111

Page 117: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

ORL, MIT and Yale, while for the second database, 10 face images for each40 subjects were collected from the ORL database. The authors collected alsofor the first database 5 iris images per subject of another set of 90 subjects,while 10 iris images were acquired per subject from another set of 40 subjectsfor the second database. For each database, the biometric traits were randomlypaired to obtain a virtual bimodal database. With the first database, only theweighted sum-rule and the neural network fusion methods outperform the bestmonomodal verification system, the iris-based approach, while all the fusionstrategies outperform the monomodal approaches with the second database.Furthermore, with a neural network training for every subject, this fusion ap-proach obtained the highest verification accuracy with the two databases. Withthe second database, the enrollment failure, due to the poor quality of the irisimages, can also be decreased by fusion approaches.

A review of bimodal approaches for acoustic speech and visual speech can befound in [61]. The authors have presented the principal components and the gainin accuracy and robustness of such approaches, compared to single modalities.

BioID is a commercial multimodal approach for a model-based face classifier,a VQ-based voice classifier and an optical-flow-based lip movement classifier [99]for verifying persons. Lip motion and face images were extracted from a videosequence and the voice from an audio signal. Accordingly to the security level,experiments on 150 persons demonstrated a decrease below 1% of the FAR.

In [118], a bimodal approach was proposed for a PCA-based face and aminutiae-based fingerprint identification system with a fusion method at thedecision level. The experiments were conducted on 1500 fingerprint imagesfrom a set of 150 subjects, captured with an optical sensor, and on 1132 imagesfrom a set of 86 subjects. To obtain a virtual bimodal database, 86 subjectsof the fingerprint subset were randomly paired with the 86 subjects of the facesubset. At a FAR of 0.01%, the monomodal systems obtained a FRR of 61.2%and 10.6% for face and fingerprint respectively. For the same FRR, the fusionapproach obtained a FRR of 6.6%.

In [27], a bimodal approach for face and speech recognition was proposed, witha fusion method at the decision level by using Bayesian statistics. This fusionmethod takes into account the estimated biases of individual expert opinions, inorder to obtain a single decision from the bimodal system. This single decisionapproach reached a success rate of 99.5%.

In [46], a bimodal approach for a pixel level-based face recognition and a VQ-based text-independent speaker recognition system was proposed with a fusionmethod at the matching score level by using a weighted geometric average aftera normalisation step. The bimodal approach obtained a correct identificationrate of 98%, while the monomodal approaches obtained a correct identificationrate of 88% and 91% for the voice and the face based systems respectively.

112

Page 118: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

10.2.3 Single biometric, multiple matchers, units and/orrepresentations

In [297], a monomodal approach was proposed for minutiae-based fingerprintrecognition, with a combination method at the features level. The experimentswere conducted on two fingerprints of 100 subjects, each finger acquired twotimes. After the extraction process, the minutiae points of both fingers wereoverlayed, regarding their center of masses. For the matching process, twofingerprints of a same person are compared to the combined templates. Thecorresponding points between the first fingerprint and the template are removedand the remaining points are compared to the second fingerprint. The matchis accepted if the ratio ot the matched points between this second fingerprintand the remaining minutiae is above a certain threshold. The FAR was about1.8% and the EER was about 1.9%. The authors have also demonstrated thata single fingerprint was not sufficient to regain the combined template in thetemplate database, as the identification rate at top-5 ranking was about 39%.

In [168], a monomodal approach was proposed for a PCA-based, an ICA-based and LDA-based face recognition systems with a fusion method at thematching score level by using a sum-rule and a RBF Network-based strategies.The experiments were conducted on 2060 face images, collected from a set of206 subjects derived partially from public available face databases, such as ORL,Yale, AR. The recognition accuracy of the combined approaches was significantlyhigher, 90.0% and 90.2% for the sum-rule and the RBF-based respectively, thanthose of the single classifiers, 79.1%, 88.1% an 81.0% for the PCA-based, ICA-based and LDA-based respectively.

In [221], the authors proposed the combination at the decision level using theconfidence level of each classifier, of 4 different fingerprint matching algorithms,3 minutiae-based (Hough transform, string distance and 2D dynamic based) and1 ridge features-based (filterbank-based). The experiments were conducted on167 subjects, for which 2 impressions of 4 different fingers were acquired twotimes (after an interval of six weeks) by an optical sensor. The experimentsshowed that the combination of all the matchers improved the performancemore than 3%, without increasing the time to match. When such an approachis used, the authors also proposed to apply a classifier selection scheme, theclass separation statistic, before the decision combination, in order to rejectmatchers which give no performance improvement after combination. Someother experiments showed also that the combination of multiple representationsor multiple units of a same modality also improve the performance of such asystem.

In [143], the authors proposed the combination, at the score level with lo-gistic transform, of 3 different fingerprint minutiae-based matching algorithms(Hough transform, string distance and 2D dynamic based). The experimentswere conducted on 167 subjects, for which 2 impressions of 4 different fingerswere acquired two times (after an interval of six weeks) by an optical sensor.The experiments performed the combination of all three different pairwise al-gorithms. With one matcher, the performance improvement is not significant,

113

Page 119: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

while when the two other algorithms are combined, a significant performanceimprovement was noticed.

114

Page 120: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 11

Integration to identitydocuments

This section first describes technical requirements of the International Civil Avi-ation Organization (ICAO) and the National Institute of Standards and Tech-nology (NIST), following by some Regulations and Proposals of the EuropeanCouncil, as these recommendations will be followed by the Swiss governmentin its choice of biometric solutions for Swiss identity documents. Finally, themain projects of biometric identity documents in some European countries willbe presented, as Switzerland, France, Belgium, Italy, Spain, the Netherlands,Great-Britain, Germany and the United States of America [67].

11.1 ICAO technical specifications

The recommendations about the deployment of biometrics in machine readabletravel documents of the ICAO were presented previously in Section 3.5, whilethe most important technical components of such a deployment will be presentedbelow [125]:

- Inclusion of a contactless IC ISO compliant chip, readable up to 10 cm. Thischip should possess an ISO compliant operating system, a high speed ofdata retrieval, a high storage capacity.

- Encryption and protection of the data according to the ICAO’s PKI technicalreport [157] 1.

- Indication about the positioning of the storage medium (on the data page, inthe centre of booklet, between front-end paper and front cover or betweenrear-end paper and rear cover) and the standardized organization of thedata stored on the chip, according to the ICAO’s PKI technical report[157] and LDS technical report [126] 2.

1Section 2.8.1 presents more information about the PKI technical report.2Sections 2.8 and 2.7.2 present more information about the ICAO’s PKI and LDS technical

reports.

115

Page 121: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

- Modification of the durability of the contactless IC chip (from 10 to 5 years)[129] 3.

11.2 NIST recommendations

In a joint report to U.S. Congress from National Institute of Standards andTechnology (NIST), the Department of Justice and the Department of State,some recommendations were proposed after evaluations of the accuracy of bothface and fingerprint modalities for the U.S. border checking [201]. This report,usually referred to as the ”303A Report”, has the following recommendations:

- For the verification process, a bi-modal approach, with two fingerprints andface, is needed for achieving the required accuracy for the border control.Each biometric image, of 10kB or less, should be stored on a chip of 32kBcapacity 4. These two modalities are the only biometric modalities whichhave studied by NIST for accuracy certification.

- For the identification process, the use of ten plain image impressions to per-form the background check is recommended.

In the Appendix A of this report [200], the NIST specifies some additionalrequirements:

- For certifying the accuracy of any biometric, large-scale databases are needed.The evaluations can only be conducted on databases of at least 100’000subjects, in order to properly determine the performances of biometricrecognition systems.

- The enrollment images, as well as the images acquired during the verification/ identification process should have similar quality standards, and thusinternational standards, as ANSI, WSQ and JPEG standards, have to beused for a large interoperability between systems.

- The storage medium should be adequally protected with a Public Key In-frastructure (PKI), in order to authenticate that the travel document wasprovided from a legitimate source, to ensure that the data has not beenaltered or modified since the issuance, and to protect the data’s privacy.

11.3 European Community - European Council,legal specifications

Since 2003, six proposals and decisions, concerning the use of biometric infor-mation in travel documents at European level, were submitted or adopted tothe European Commission and Council [147] 5.

3Section 2.7.2 presents more information about the Annex I of the ICAO’s MRTD report.4In a later report, the NIST recommended rather the use of face recognition “only for cases

where fingerprints of adequate quality cannot be obtained” [285].5The report of the Joint Research Center is available online at

ftp://ftp.jrc.es/pub/EURdoc/eur21585en.pdf.

116

Page 122: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

In September 2003, the Proposal for a Council Regulation amending Regula-tions (EC) 1683/95 laying down a uniform format for visas and (EC) 1030/02laying down a uniform format for residence permits for third-country nationals[88] 6, proposed to move forward the implementation date for photograph (2005instead of 2007) and to integrate biometric information in the visa and residencepermits. Furthermore, this proposal specified that a robust security level canonly be achieved by the use of a minimum of two modalites. The modalitieschosen for this purpose are: a facial image designed as the primary biometricidentifier and a fingerprint image designed as the secondary biometric identifier,which is more able to work with large databases. These biometric identifiers willonly be used for verification purposes, and in no case searches will be performedin the Visa Information System database.

In June 2004, the Council Decision No 2004/512/EC establishing the VisaInformation System (VIS) 7, decided to establish a common identification sys-tem for visa data, in order to exchange these data between Member States [74].A centralised architecture, the Central Visa Information System (CS-VIS) andan interface in each Member State, the National Interface (NI-VIS), should bedeveloped by the Commission, respectively by the Member States.

In December 2004, the Council Regulation No 2252/2004 on standards forsecurity features and biometrics in passports and travel documents issued byMember States 8 specifies the minimum security standards required for bio-metric Identity Documents, accordingly to the ICAO requirements, in order tohave a harmonisation at an European level [75]. The use of biometrics shallestablish a more reliable link than previously between the genuine holder andthe document. The medium storage of the identity document shall contain afacial image 9 and fingerprints 10. These data shall be secured and the mediumstorage ”shall have sufficient capacity and capability to guarantee the integrity,the authenticity and the confidentiality of the data”. This regulation requiresthat additional technical specifications, on security features for enhancing anti-forgery, counterfeiting and anti-falsification standards, on the storage mediumand on requirements for the biometric format, shall be established. The personaldata stored on the identity documents can be available by the concerned person,who has a rectification or erasure right. This regulation specifies also that thebiometric information stored shall only be used for verifying the authenticity ofthe document and the identity of the holder.

In December 2004, the Proposal for a Regulation of the European parliamentand of the Council concerning the Visa Information System (VIS) and the ex-change of data between Member States on short stay-visas [89] 11 has for mainobjective to define the purpose, the functionalities and responsabilities for theimplementation of the VIS.

In February 2005, the Council Regulation No C(2005)409 on standards for

6Available at http://europa.eu.int/eur-lex/lex/LexUriServ/site/en/com/2003/com2003_0558en01.pdf.7The Council Decision No 2004/512/EC is available online at

http://europa.eu.int/eur-lex/pri/en/oj/dat/2004/l_213/l_21320040615en00050007.pdf.8Council Regulation No 2252/2004 available online at

http://europa.eu.int/eur-lex/lex/LexUriServ/site/en/oj/2004/l_385/l_38520041229en00010006.pdf.9At the latest 18 months after the adoption of the requirements of this regulation (end of

August 2006 at the latest).10At the latest 36 months after the adoption by the European Union of additional require-

ments concerning fingerprints.11 http://europa.eu.int/eur-lex/lex/LexUriServ/LexUriServ.do?uri=CELEX:52004PC0835:EN:HTML.

117

Page 123: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

security features and biometrics in passports and travel documents issued byMember States 12 presented technical specifications which are not yet describedin the Council Regulation No 2252/2004 [68]. This regulation is based on inter-national standards such as the ISO’s and ICAO’s requirements and presents thespecifications for the biometric information used in travel documents, the stor-age medium, the logical data structure on the chip, the security specifications ofthe data stored on the chip, the conformity assessment of chip and applicationsand the RF compatibility with other travel documents.

Recently, the European Commission has officially asked the United StatesCongress to accept another extension of the deadline for introducing biomet-ric information in all European travel documents until August 2006, insteadof October 2005, as only six European Countries will satisfy the US require-ments at the initial deadline 13. This request was finally accepted by the UnitedStates 14 and thus the EU members have until September 2006 to fulfill the USrequirements for biometric passports. Indeed, the USA require that the biomet-ric passports of each state should be submitted by the beginning of September2006, in order to be validated by the end of October 2006. In addition, the USArequire that lost and stolen passports have to be reported by the issuance stateto the US Department of Homeland Security (DHS) and to the Interpol, thatintercepted lost and stolen passports have to be reported to the DHS’s Fraudu-lent Document Analysis Unit, and that the trends and analysis of these lost andstolen passports, as well as the other security features of the travel documents,have to be shared with the DHS.

The actual Presidency of the European Union, the United Kingdom, hasinvited the Council, the European Commission and the Member States to “draftcommon security standards for national identity cards taking into account theachievements in relation to the EU passport and the ICAO framework” [224].Some further discussions should be started about the security of enrollment andissuing processes, digital signatures and anti-fraud measures.

Finally, in November 2005, the Proposal for a Council Decision concerningaccess for consultation of the Visa Information System (VIS) by the authori-ties of Member States responsible for internal security and by Europol for thepurposes of the prevention, detection and investigation of terrorist offenses andof other serious criminal offenses has for main objectives “to allow for and tolay down the conditions under which Member States authorities responsible forinternal security and the European Police Office (Europol) may access the VisaInformation System (VIS)” for the purpose of “prevention, detection and inves-tigation of terrorist offences” and other type of crime and offenses for whichEuropol are competent [90] 15. The European Data Protection Supervisor hasput forward an opinion about this proposal [123] 16, and proposes some improve-ments which ca be made for example about the VIS accessibility conditions, thelevel of data protection and the supervision that should be ensured.

12Commission decision No C(2005)409 available online in French athttp://europa.eu.int/comm/justice_home/doc_centre/freetravel/documents/doc/c_2005_409_fr.pdf.

13Press release available online at http://europa.eu.int/idabc/en/document/4068/194.14Press release available online at http://europa.eu.int/idabc/en/document/4380.15Proposal available online at http://europa.eu.int/eur-lex/lex/LexUriServ/site/en/com/2005/com2005_0600en01.pdf.16Opinion available online at http://www.edps.eu.int/12_en_opinions.htm.

118

Page 124: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

11.4 Biometric identity documents projects

As mentioned in [153], 11 Visa Waiver Program countries (out of 27) haveintroduced a biometric passport before the end of 2005, while 8 VWP countrieshave extended their deadline introduction to the end of 2006. For the remainingVWP countries, the introduction date of their biometric identity document isnot decided yet.

11.4.1 Switzerland

The introduction of biometric information in Swiss identity documents willfollow the ICAO Doc 9303 [124] and the European regulations (Section 11.3presents more information about the Council Regulations and Proposals). Cur-rently, the Swiss prescription and law on identity documents [69, 70] will bemodified 17 in order to allow the introduction of biometric information in Swissidentity documents, at first as a pilot project of a duration of 5 years, and thenin a permanent way. During this pilot project, a limited number of biomet-ric passports will be issued on a voluntary base, in parallel with the currentmachine-readable passport (2003 model) from September 2006. The currentSwiss passport contains the following informations: the name, the first name,the date of birth, place of origin, nationality, the height, the signature, the pho-tography, the information concerning the prefecture which has issued the card,the issuance and expiration date and the card number. The new biometric pass-port will contain on a chip, besides all the information already present on thecurrent passport, the facial image in a digital form. The biometric informationwill be stored on the chip and in the information system relative to the Swissidentitiy document called ISA, which is the database containing already thepersonal information present on the current passport since 2003. This databaseallows the correct management of Swiss identity documents, avoids the issuanceof several identity document for the same person, prevents any abuses and veri-fies the person’s identities and the authenticity of the documents. The biometricdata on the chip will be authenticated and protected with a electronical signa-ture, while the data contained in the ISA will be secured with protection meansproper to the information system. As specified in the Swiss prescription andlaw on identity documents [69, 70], restrictives measures are applied to the ac-cess to the data contained in the ISA. Indeed, these data are only accessibleto authorized people, and can not be used for any police investigation. Thisnew identity document will be valid during 5 years and will cost 250 CHF. Forthe pilot project, enrollment centers will be put in place in eight Swiss Cantons(AG, BE, BS, GR, SG, TI, ZH and VD) and in eight Swiss representationsoutward (Paris, Francfort, London, Mexiko, Toronto, Sao Paulo, Hong Kongand Sydney). The budget assigned to this five year pilot project is estimated ofabout 14 millions CHF.

The Swiss Federal Department of Justice and Police is also investigating inthe purpose of a digital identity card. The electronic signature and certificatepresent on the chip will be used for the access to public and private authentica-tion processes and for signing electronically authentic documents. The current

17The draft versions of the Swiss prescription and law on identity documents are availableat http://www.schweizerpass.admin.ch.

119

Page 125: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Swiss identity card has a format of a credit card and contains the same infor-mation as the Swiss passport 18.

11.4.2 France

The INES program 19 has for main objectives to enhance the security of theFrench issuance process, to improve the management of identity documents, toissue highly secure documents, and to allow citizens to authenticate on the In-ternet. The new Identity Card called CNIE 20 will have a format of a creditcard and will include a contactless chip. The informations printed on the cardare as follows: the name, the first name, the date and place of birth, the gen-der, the address, the signature, the information concerning the prefecture whichhas issued the card, and the card number. The information contained on thecontactless chip will be subdivided in five distinctive blocks. The Identity block,which will only be accessible to authorized people and will be secured withcryptographic approaches, will include the biometric information, as the pho-tography and two fingerprints, and all the information present on the card. TheCard authentication block will allow an automatic and anonymous authentica-tion of the card itself. The Certified identification block, secured with a PINcode, will allow the access to public and private authentication processes. TheElectronic signature block will allow, by mean of a PIN code, to sign electroni-cally authentic documents. The Personal ”portfolio” block will allow people tostore complementary personal information. Each person in possession of sucha card will receive a document with all the information included in the card,and will have a rectification right. The access to the biometric information byauthorized people will be contactless and its traceability guaranteed, while theother functionalities will be used with contact. This new identity documentwill be valid during 5 years. Before its introduction, the French governmenthas mandated the Internet Rights Forum 21 for an online debate, as well aspublic debates in several regions, about this new identity card. The summaryand the conclusion about this latter was recently submitted to the French HomeSecretary and made publicly available [97] 22. This debate has produced inter-esting results. While three-fourths of the population support the introductionof a new identity card for security reasons, some fears and reluctances appeared.The main recommendations of this organisation are as follows:

- A rigorous study has to be conducted in order to evaluate the real extent ofthe identity fraud in France.

- The new identity card should not be introduced at the same time as the newbiometric passport.

- The citizens should have a free and permanent online access to their ad-ministrative folders and to the state of progress of their administrativeprocedures.

18Complete description of the Swiss identity card available on the Swiss Federal Office ofPolice’s homepage at http://www.schweizerpass.admin.ch.

19Complete description of the French INES program available athttp://www.foruminternet.org/telechargement/forum/pres-prog-ines-20050201.pdf.

20CNIE : Carte Nationale d’Identite Electronique.21Internet Rights Forum’s homepage at http://www.foruminternet.org.22This report is available on the Internet Rights Forum’s homepage.

120

Page 126: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

- A coherence between the unique and centralised identification used with theINES project and the plurality of the identifiers used currently in theelectronic administration.

- A global and permanent control of the system should be conducted by theCNIL.

- The use of contactless chip should only be admitted when studies demonstratethat no covert access to the data will be possible.

- The CNIE should only be introduced when the computerization of the registryoffice is achieved.

- An explanation campaign should be conducted in order to inform people aboutthe use of the CNIE in administrative e-procedures.

- The CNIE should be costless for the first issuance, but payable in the case ofloss and renewal.

- The mandatory nature of the CNIE should be evaluated with care, as it willbe a considerable modification.

- The issuance in municipalities should first be debated with regional politicians.

- Some harmonisation works about norms, standards and interoperability shouldbe conducted with other countries.

After the publication of these recommendations, the French Governmentwould have decided to move back to 2008 the introduction of the CNIE, inorder to focus their efforts on the development of the new biometric passportsand to eliminate the mandatory nature of the CNIE in the INES project.

The French biometric passport, fulfilling the ICAO Doc 9303 [124] and theEuropean regulations, will be issued in autumn, 2006. The chip will contain,besides all the information already present on the current passport, at first adigital photography and then fingerprints. The CNIL has given its approvalabout the introduction of such identity documents 23.

11.4.3 Belgium

The eID 24, the new electronic identity card, is introduced since 2003. Untilthe 31th of December 2009, all non-electronic identity cards will be replaced.This new card has a format of a credit card and includes a chip, readable bycontact, but containing no biometric information. The information printed onthe card are as follows: the photography, the name, the first name, the gender,the signature, the date and place of birth, the nationality, the national identifi-cation number, and the issuance and expiration date. All these information arecontained in the chip (except the photography), as well as the address of thesubject. The electronic signature and certificate present on the chip can be usedfor the access to public and private authentication processes. As a centralizeddatabase of the national population already exists, the same framework is used

23CNIL’s approval available at http://www.cnil.fr/index.php?id=1773.24Complete description of the new Belgian electronic identity card program available at

http://eid.belgium.be/fr/navigation/12000/index.html.

121

Page 127: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

with this new identity card. Every people has an access and modification rightof all the information included in the card and also an access to the people whohas consulted it, and for which reason. The consent of the concerned personwill be necessary for accessing the information present on the card. This newidentity document is valid during 5 years.

The new Belgian biometric passport 25 will follow the ICAO Doc 9303 [124]and the European regulations (see also Section 4.1 for more information aboutthe Council Regulations and Proposals). This new passport, with a contactlessreadable chip, is issued since November 2004. The information contained onpage 2 of the document are also contained on the chip: the name, the firstname, the nationality, the gender, the date and place of birth, the issuance andexpiration date and the signature and photography. The fingerprints will beintroduced later. This new identity document is valid during 5 years.

11.4.4 Italy

The new Italian identity smart card 26 use two kinds of technology, a chip witha capacity of 34kB, and a laser readable band. The information stored on thechip will only be used for the access to public authentication and identificationprocesses, by means of symmetric and asymmetric keys, for electronic signaturepurposes, and for online voting. The laser band will be used as an identitycard. The information printed on the card are as follows: the photography,the name, the first name, the gender, the height, the nationality, the date andplace of birth, the identification number, the address and the validity period ofthe card. While the laser band will contain the personal information, the chipwill contain a digitized signature and a fingerprint. All these informations willonly be stored on the card, and not in a centralized database. The access tothe information contained on this card will only be possible with consent of theconcerned person. This new identity document will be valid during 5 years.

11.4.5 Spain

The new Spanish identity card contains the same data as the old one. Theinformations printed on the card are as follows: the name, the first name, acolour photography, the signature, the identification number, the issuance dateand the validity period. The readable part contains the gender, the residenceplace, the date and place of birth, the name of his/her parents and the infor-mation concerning the administrative service which has issued the card. Thechip will include, in addition to all these informations, an electronic attestationof authentication, a certificate for electronic signature, biometric information (afingerprint and the photography), the digitized signature. An asymmetric cryp-tographic approach will be used to enhance the security of the identity card, apublic key known by the police and the private key stored on the chip, activatedby a PIN code. This new identity document will have an estimated life time of10 years.

25Complete description of the new Belgian biometric passport available athttp://www.diplomatie.be/fr/travel/passports.asp.

26Complete description of the Italian Identity Card available athttp://www.cartaidentita.it.

122

Page 128: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

11.4.6 Netherlands

A biometric passport has being tested on a six-month period in the Netherlandssince August 2004, until the end of February 2005. [17, 19, 20, 18] 27. This pilottest, named “2b or not 2b” and conducted by the Dutch Ministry of the Interiorand Kingdom Relations, evaluated the impact of the new generation of Dutchpassports and identity cards, in the request process, the biometric acquisitionprocess and in its day-to-day use. 14’700 volunteers, from 6 different municipal-ities have taken part in this pilot evaluation. The chip contains amongst otherthings a photography and two fingerprints. During the issuance process of thispilot project, additional information about physical characteristics which caninfluence the biometric recognition process, such as beard, glasses, dark skin,hobby or jobs that can damage the fingerprints, are also collected. This pilotproject acquired these two modalities and will store them in the biometric pass-port and in a “test document”, which was in possession of the authorities. Thislatter was only used for the performance evaluation of the biometric technologiesand the quality evaluation of the biometric information, in a anonymous way,and was destroyed at the end of the evaluation. When the biometric passport isdelivered to the bearer, a verification procedure (identical to a border checkingcontrol) is completed in order to determine if the biometric data recorded on thechip corresponds to the data acquired during this process. The first results ofthis large-scale evaluation will be presented later. This new identity documentwill have a validity period of 5 years.

11.4.7 Great-Britain

Since the beginning of 2005, UK has adopted the Identity Cards Bill 28, whichhas for main objective to introduce a national identity card [271]. The Billis composed of eight components. The National Identification Register, a cen-tralised register, which will contain information from all UK residents agedover 16. The National Identity Registration Number will give a unique numberfor every individual. The biometric information, such as face, fingerprint andiris, which have to be collected for every individual. The Identity Card willbe generated from this register and contain a chip, protecte by cryptographicmethods. The informations printed on the card are the photography, the name,the address, the gender, the date of birth and the identification number. Theinformation stored on the chip are the name, the first name, the date and placeof birth, the photography of the face (and the shoulders), the fingerprints, theirises, the digitalized signature, residence places, the nationality, the nationalidentity register number, and all other information that the concerned personwill store on it. Legal obligations will require this identity card for obtainingsome public services and administrative convergence will allow the centraliza-tion of all registration numbers used by a single person. Cross notification willallow all agencies to reach the data of a person, without its consent, throughthe Secretary of State. New crimes and penalties will be established, in orderto comply with the requirements of the program. The National Identity Scheme

27The promotion document of the “2b or not 2b” project, as well as the Biometrie In-Zichtjournal, are available online at www.bprbzk.nl.

28The UK Identity Cards Bill and the explanatory notes to the Bill are available online athttp://www.publications.parliament.uk/pa/cm200405/cmbills/008/2005008.htm.

123

Page 129: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Commissioner will have for objective the supervision of this program. As previ-ously mentioned, the access to the information contained on this card will onlybe possible with consent of the concerned person. However, these informationwill be available for some public authorities, under some conditions and rules.

The choice of the UK Government is questioned by a recent report of theLondon School of Economics & Political Science [265]29. For the Advisory Groupand the Research Group (38 persons in all), authors of this report, the IdentityCards Bill proposals are“too complex, technically unsafe, overly prescriptive andlack a foundation of public trust and confidence”and“do not represent the mostappropriate, secure, cost effective or practical identity system for the UnitedKingdom”. Indeed, other methods, more appropriate for the purpose, should beused to achieve the main objectives of the Bill. The technology which is plannedto be used at large scale is “untested and unreliable”. Furthermore, the costs ofsuch an identity system has been underestimated by the UK government, and thelegal framework proposed will create conflicts with national and internationallaws. Add another oversight body in the UK will increase the complexity andinefficiency of the the current oversight process. The LSE report concludes alsothat the Government has “failed to correctly interpret international standards,generating unnecessary costs, using untested technologies and going well beyondthe measures adopted in any other country that seeks to meet internationalobligations”.

An enrollment trial was performed by the UK Passport Service from April toDecember 2004, with more than 10’000 persons [270]. The main goals of this trialwas to “test the processes and record customer experience and attitude duringthe recording and verification of facial, iris and fingerprint biometrics”. Theenrollment procedure had not taken place in laboratory conditions, but in sixfixed and one mobile enrollment centers, placed in the United Kingdom. A boothwas specially designed for the enrollment procedure of this trial. During theenrollment, the subjects were placed on a standard office chair within the booth,with an operator outside the booth which has a visual contact. The trial hadthe following stages: registration, photograph participant (head and shoulders),record facial biometric, record iris biometric, record fingerprint biometric, recordelectronic signature, print card, post-enrollment questionnaire, verification andpost-verification questionnaire. The subjects were separated in three groups:the quota sample with 2’000 subjects (representative to the UK population),the opportunistic sample with 7’266 subjects (recruited from the area aroundthe enrollment centers, without any other criteria) and the disabled participantsample with 750 subjects. The enrollment times for all three biometrics wasabout 8 minutes for the quota sample and about 10 minutes for the disabledparticipants. The verification times for the quota sample were 39 seconds, 58seconds and 1 minute 13 seconds for facial, iris and respectively fingerprintverification. The verification times for the disabled participants were 1 minute3 seconds, 1 minute 18 seconds and 1 minute 20 seconds for facial, iris andrespectively fingerprint verification. For all three biometrics, the enrollmentsuccess rates were higher for the quota sample than the disabled participants:nearly 100% and 98% for face, 90% and 61% for iris and nearly 100% and 96% forfingerprint. The verification success rates were also higher for the quota sample

29The Interim Report of the LSE is available online athttp://www.lse.ac.uk/collections/pressAndInformationOffice/PDF/IDreport.pdf.

124

Page 130: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

than the disabled participants: nearly 69% and 48% for face, 96% and 91% foriris and nearly 81% and 80% for fingerprint. Whilst the participant experienceof positioning for the iris modality was considered as more or less difficult, theparticipants have yet chosen this modality as their preferred. Furthermore, themajority of participants concidered that the use of biometric data in identitydocuments will increase the passport security, prevent the identity fraud andillegal immigration, and will not be an infringement of their liberties. Whilsta majority of the participants are not very or not at all concerned before theenrollment trial about having their biometrics recorded, the number of thosewho were very concerned about it before decreased after enrollment. The mainrecommendations of this report are as follows:

- The camera should be maneuverable enough in the enrollment booth.

- Any headwear may be removed by applicants, or arranged so that the face orforehead is not obscured.

- Considerations should be given when a biometric modality is not available ona temporary basis.

- Trained operators should be specially present when disabled persons are en-rolled.

- Further investigations should be performed with persons who had enrollmentdifficulties, without being disabled.

- The enrollment of the fingerprint modality should be successful, even if somefingers do not satisfy some quality measurements.

- If the first verification procedure fails, some further attempts should be al-lowed.

- The use of fingerprint sensors with a large acquisition area should be preferred,in order to acquire sufficient information.

- The biometric devices used for UK passports should be chosen on the basisof a test ring.

- Some education initiatives have to be conducted, on the basis of the resultsfrom the customer experience and attitude questions obtained during thetrial.

Since the beginning of this year, Great-Britain is delivering the BiometricBritish Passport which will contain the holder’s facial image on a chip 30. Ac-cording to the UK Passport Service, the biometric information will be usedto link a person to a passport in order to help the detection of counterfeit ormanipulated documents and to confirm the identity of the individual.

30Complete description of the Biometric British Passport athttp://www.passport.gov.uk/general_biometrics.asp.

125

Page 131: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

11.4.8 Germany

All new German biometric passport (the ePass) will include from November2005 a contactless readable RF-chip and a digital photography stored on it. TheePass 31 issued from the beginning of 2007 will also include two fingerprints.The access to the information contained on the card will only be possible aftera previous optical lecture of the machine readable zone and a cryptographicalprotocol between the reader and the identity document. The biometric informa-tion will only be stored on the chip. Indeed, no centralised biometric databasewill be created for the new passport. This identity document will be used forauthentication purposes and will allow signing electronically authentic docu-ments. The new German biometric identity document will follow the ICAO andEuropean Union requirements.

Furthermore, some studies have already been conducted by the ”Bundesamtfuer Sicherheit in der Informationstechnik” for evaluating facial recognition tech-nologies, such as BioFace 1 & 2 and BioP I (see Section 5.3) and fingerprintrecognition technologies such as BioFinger 1 (see Section 6.3) for Identity Doc-ument purposes. Another study, based on BioP, has been conducted on a largerpopulation set and with additional biometric modalities (fingerprint and iris) inorder to evaluate more in details the influence and the feasibility of the intro-duction of biometric information in travel documents [49] 32. This evaluationwas conducted during four month on 2000 subjects. The biometric systems usedwere: two fingerprint verification systems, one face recognition verification sys-tem and one iris recognition verification system. Biometric images (as specifiedin the ICAO specifications) and biometric templates were used separately inorder to compare these two approaches. The main results of this evaluation canbe summarized as follows. First, the performances of each system were depen-dent on the frequency of use of the subjects, particularly for the iris recognitionsystem (false rejection rates for the iris system have reached in this evaluationabout 20% with low frequency users). Then, the use of biometric template forthe reference data performs better than the biometric images. Finally, this eval-uation proposed a ranking of the three modalities, accordingly to the resultsobtained in these tests: fingerprint, face and iris.

The BioPII results, especially those about the iris modality, was questionedby Prof. John Daugman after this modality ranking and the high FRR and FARfor iris recognition reported by this evaluation [81]. So, all the comparisons withthe iris data will be computed again with the algorithm of Prof. Daugman.

11.4.9 United States of America

The Biometric U.S. passports, which will follow the ICAO requirements, willinclude on the chip a digital image of the face and the biographic informationprinted on the data page, such as the name, the date and place of birth, thepassport number, the dates of issuance and expiration, the issuing authority, thedocument type, the passport application reference number and a unique chip

31Complete description of the new German electronic identity doc-ument ePass available on the http://www.bmi.bund.de/ server and athttp://www.bsi.bund.de/fachthem/epass/index.htm.

32The BioP II evaluation is available online at http://www.bsi.bund.de/literat/index.htm.

126

Page 132: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

identification number [82] 33. All the information contained on the chip willbe digitally signed (but not encrypted), in order to protect their integrity, aswell as that of the identity document. The contactless chip will have a memoryof 64 kB of EEPROM storage and will be written only once, without that amodification can me made. This identity document will be valid during 5 yearsfor persons under 16 years old and 10 years for other persons and will be firstissued in mid-2005.

11.4.10 International Labour Organisation (ILO)

The Seafarers’ Identity Documents Convention No 185 (2003) 34 was adoptedby the ILO for introducing security elements in their identity cards, which areconform in all respects to the ICAO and ISO specifications and requirements[132]. The information include on the card will not be stored on a chip, butprinted directly on the card. These information shall be restricted to the fullname, the gender, the date and place of birth, the nationality, physical char-acteristics, a digital photography, the signature, and two fingerprint templates.These latter, non-reversible minutia-based templates, will be visible on the doc-ument and printed as numbers in a bar code, which will contain up to 686 Bytesof data. Up to 52 minutiae features per finger will be extracted and stored inthe barcode. The verification procedure will be completed in a serial mode:if the first finger is unavailable, failed to acquired or does not match with thecorresponding template, the second finger is

33The proposed rule of the Department of State is available online athttp://www.access.gpo.gov/su_docs/fedreg/a050218c.html.

34Complete version of the ILO Convention 185 available online athttp://www.ilo.org/ilolex/cgi-lex/convde.pl?C185.

127

Page 133: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Chapter 12

Summary and future work

In this chapter, despite Wayman’s pronouncement [280]:

“It is impossible to state that a single biometrics is better than theothers for any applications, populations, technologies, and adminis-tration policies ”,

we summarise the state-of-the-art and make recommendations concerningthe introduction of biometrics in Swiss identity documents.

12.1 Usage of biometrics

The Swiss public, parliaments and government will need to debate about theuse of biometric identity documents they would like allow/implement. Shoulda biometric identification system be put in place, incorporating a centraliseddatabase and a potentially a watchlist, or should it be a verification system toconfirm the identity of the identity holder, which is the primary objective ofidentity documents.

The locations of biometric identification or verification points should be dis-cussed.

Additionally, the enrollment and issuance processes for different identitydocuments may need to be revised to avoid problems with source documents.

12.2 Performance

The Swiss public, parliaments and government will need to debate about per-formance requirements for a biometric identity documents scheme. This choicewill dictate in large part the choice of a given modality and the technical re-quirements of the system.

We should point out that some modalities can be expected to achieve closeto (but not exactly) 0% false accept rate on large populations, with 2% to 8%false reject (iris), while other modalities can provide about 5% false reject with afalse accept rate of 0.01% on much smaller populations (fingerprint). However,if every Swiss citizen is enrolled in the system, the population size jumps to morethan 7 millions, and even small error rates on paper can become significant.

128

Page 134: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

In any case, what should be taken into account is that no biometric modalitywill give 0% error for both false accepts and false rejects. Furthermore, for allmodalities presented in this report some users will be non-enrollable, and thusa non-biometric option of identity verification should be provided (as in currentsecure passports).

Some modalities such as face and speech are very sensitive to changes inacquisition conditions and thus their use in a large-scale system for identitydocuments seems difficult at best, as noted in many medium-scale evaluationcampaigns.

12.3 Systems architecture and information se-curity

For this section and this report in general, we should emphasize that we assumea secure issuance process is in place, as is in principle the case today. Thatis, we do not deal with issuance issues such as fake source documents (birthcertificates and others) and we suppose clerical staff are skilled in establishingthe authenticity of the latter.

Also, it should be noted that some of our recommendations may be at oddswith the ICAO standard on some points (notably storage medium for biometricdata).

As much of the processing steps as possible should take place in a distrib-uted fashion, that is within one smartcard or PC host, without resorting to acentralised storage where information would need to travel over networks. Thisis mainly because a centralised database containing biometric templates of citi-zens can lead to compromise through technical or other means, in turn entailingprivate data loss and identity theft problems.

For several security reasons, the match on card approach is preferred to asimple template on card system: The identity document holder does not need totrust the computer and network to which the card is connected, or the humanoperator. Cracking or trojanization of the matching terminal is not an issue, asthe template never leaves the secure identity document.

Table 2.7.3 summarises the options available for locations of the processingsteps. Based on this table, the above, and typical performances, two modal-ities can be recommended: fingerprint or iris, together with a sensor-on-cardapproach for fingerprint. Also, we recommend that only templates should bestored on the smartcard, not raw data.

12.4 Privacy and public perception

As part of the feasibility study and before deployment, a full privacy impactassessment should be carried out, under the leadership of the federal data pro-tection commissioner.

Standard privacy guidelines (access rights, consent, ...) can be used as astarting point, and augmented with the guidelines exposed in Section 4.3. Animportant issue to address is whether biometrics in identity documents shouldserve for verification, or their scope be extended to incorporate identificationwith a central database. The proportionality rule can be applied in this case.

129

Page 135: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

Where a user is non-enrollable (for instance, it is estimated that about 4%of any population does not possess suitable fingerprints), he/she should notbecome subject to more intensive scrutiny (failure of the system comes throughno fault of their own).

12.5 Choice of modality

Here, we summarise the state of the art in the different modalities presented sofar, according to the following criteria: error rates, inter-session variability, uni-versality, noise sensitivity, architectural features such as feasibility of distributedtemplate storage, sensor cost, choice of vendors, pre-existing smartcard match-on-card implementations, susceptibility to covert acquisition. We roughly clas-sify each attribute into low (L), medium (M), or high (H) and present the resultsin Table 12.1 to the best of our knowledge.

It should be strongly emphasised that, at least in terms of biometric per-formance metrics such as error rates or inter-session variability, these resultsare comparing evaluations made using very different populations and protocols.Therefore, Table 12.1 and the following ranking are at best broad approxi-mations and will need to be confirmed through rigorous experiments using acontrolled population and corresponding protocols.

Criterion 2D face FP Iris Sig. Sp.

error rates M-H L very L L M-H

inter-session var. M L very L M M-H

universality H H H M-H M-H

risk of FTE L M-L L L L

noise sensitivity H M-L M-L L H

TTE L L L M M

distributed templates H H H H H

sensor cost M-L L H M L

choice of vendors H H very L M H

MOC implementation L H L none none

covert acquisition risk H M-H L M-L H

Table 12.1: Broad summary of modalities according to several criteria.

Taking into account the biometric identity document application, we proposethe following ranking based on the literature survey for this state-of-the-art. Allthe criteria presented in Table 12.1 were used to establish this ranking.

1. Fingerprints offer the advantage of low error rates, small inter-session vari-ability, low noise sensitivity (assuming residual fingerprints are removedevery few users), fast enrollment, a wide choice of vendors, low-cost sensors(less than CHF 100), numerous match-on card and/or sensor-on-card im-plementations, difficulty of covert acquisition and corresponding reductionin “identity theft” probability.

2. Irises offer very low false accept rates for large size populations with rea-sonable false rejection rates (which is the main advantage of this modal-ity), fast enrollment, small inter-session variability, and difficulty of covert

130

Page 136: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

acquisition. Unfortunately, currently the monopoly of iris recognition sys-tems is holded principally by one major vendor 1, but several commercialvendors for iris cameras can be found on the market. Furthermore, thesensors and recognition systems are very expensive (about USD 5000 forthe camera and more for the recognition system). Failure to Enroll ratemay also be an issue.

3. Signatures have been shown to have relatively low error rates on pop-ulations size in the low hundreds. It suffers mainly from inter-sessionvariability, but is immune to environmental noise. An advantage is that itis the only truly “revocable” modality: a signature can be changed if com-promised. Different vendors propose signature verification algorithms, andlow-cost sensors can be obtained in volume. Covert acquisition is a mod-erate risk as the user must explicitly sign for her signature to be acquired(but could be tricked into doing so by social engineering).

4. The 2D face modality suffers from environmental noise and inter-sessiontime effects, but in controlled/laboratory conditions error rates can bequite low. A large array of vendor solutions are on the market and thetechnology is maturing rapidly. Sensors vary largely in cost according toimage quality specification. It is very prone to covert acquisition. 3D facemodality was not taken into account because of the relative immaturityof the technology.

5. Speech is affected by noise and speaker verification performance dropssignificantly with channel or environmental noise. Another problem is thelong enrollment and verification time, as with current algorithms a userwould be required to speak for about 10-15 seconds for the verification tohappen. Inter-session variability and easy covert acquisition seem to makethis modality a poor choice for identity documents.

12.6 Multimodality for identity documents

For the identity documents application, multimodality may be an effective toolto reduce the FTE rate. The sequential use of multiple modalities guaranteesthat the non-enrollable population is reduced drastically. Furthermore, sequen-tial use of modalities permits fair treatment of persons that do not possess acertain biometric trait.

We also need to investigate multimodal fusion with partial templates, atthe score and decision levels, to provide better privacy protection to the usersenrolled, as partial templates by themselves (i.e. not in combination) wouldyield very low identification power.

12.7 Acquisition and evaluation protocols

An ill-managed introduction of biometrics in identity documents can decreasethe security and go against the initial end of such a use. Thus, given the very

1This monopoly situation is going to change, as the patents are expected to expire thisyear and next year.

131

Page 137: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

large target population size (millions of people)) only a wide-ranging test witha strict evaluation protocol on a large number of users will provide us with theconfidence needed to deploy a large-scale system.

We need to establish an acquisition and evaluation protocol, drawing fromboth international competitions, vendor tests, established practices, and ourexpertise in the field. This will propose application scenarios suitable for identitydocuments, such as border-crossing and airport registration settings.

We need to collect a large database of more than 100 users for all 5 modalitiespresented in this report (2D and 3D face, fingerprints, iris, on-line signaturespeech), so that vendor technologies can be evaluated using strictly the samescenarios and population for all modalities. No such large multi-modal databasecurrently exist.

132

Page 138: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

Bibliography

[1] Oxford English Dictionary. Oxford Edition, 2004.

[2] A4 Vision, Vision Access Technology White Paper. A4 Vision, 2003.

[3] B. Achermann, X. Jiang, and H. Bunke, “Face recognition using range im-ages,” in Proceedings of the International Conference on Virtual Systemsand Multimedia, 1997, pp. 126–136.

[4] A. Adami, R. Mihaescu, D. Reynolds, and J. Godfrey, “Modeling prosodicdynamics for speaker recognition,” in Proceedings IEEE InternationalConference on Acoustics, Speech, and Signal Processing (ICASSP’03),vol. 4, 2003, pp. 788–791.

[5] P. Agre, “Your face is not a bar code: arguments against automatic facerecognition in public places,” Whole Earth Magazine, no. 106, pp. 74–77,2001, also available at http://polaris.gseis.ucla.edu/pagre/bar-code.html.

[6] A. Albrecht, “Privacy best practices in deployment of biometric systems,”BIOVISION: Roadmap to Successful Deployments from the User and Sys-tem Integrator Perspsective, Final report IST-2001-38236, 28th of August2003.

[7] A. Albrecht, 2005, personal communication.

[8] R. Allen, P. Sankar, and S. Prabhakar, “Fingerprint identification technol-ogy,” in Biometric Systems: Technology, Design and Performance Evalu-ation, J. L. Wayman, A. K. Jain, D. Maltoni, and D. Maio, Eds. London:Springer-Verlag, 2005, ch. 2, pp. 21–61.

[9] A. Alterman, ““A piece of yourself”: Ethical issues in biometric identifica-tion,” Ethics and Information Technology, vol. 5, pp. 139–150, 2003.

[10] American National Standards Institute, Information technology - BioAPISpecification (Version 1.1) (formerly ANSI INCITS 358-2002). NewYork, USA: American National Standards Institute, 2002.

[11] American National Standards Institute, Biometric Information Manage-ment and Security for the Financial Services Industry: ANSI X9.84-2003.New York, USA: American National Standards Institute, 2003.

[12] A. Antonelli, R. Cappelli, D. Maio, and D. Maltoni, “A new approach tofake finger detection based on skin distortion,” in Proceedings of Interna-tional Conference on Biometrics 2006 (ICB), D. Zhang and A. Jain, Eds.,vol. LNCS 3832, 2005, pp. 221–228.

133

Page 139: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[13] A. Ariyaeeinia and P. Sivakumaran, “Comparison of VQ and DTW clas-sifiers for speaker verification,” in Proceedings European Conference onSecurity and Detection (ECOS’97), 1997, pp. 142–146.

[14] M. Audetat, A. Rebetez, and Y. Cornu, “Comment on grignote nos lib-ertes,” l’Hebdo, no. 8/2005, pp. 15–18, 2005.

[15] E. Bailly-Bailliere, S. Bengio, F. Bimbot, M. Hamouz, J. Kittler, J. Mar-iethoz, J. Matas, K. Messer, V. Popovici, F. Poree, B. Ruiz, and J.-P. Thi-ran, “The BANCA database and evaluation protocol,” in Proceedings of4th International Conference on Audio- and Video-Based Biometric Per-son Authentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS2688, 2003, pp. 625–638.

[16] D. Baldisserra, A. Franco, D. Maio, and D. Maltoni, “Fake fingerprintdetection by odor analysis,” in Proceedings of International Conferenceon Biometrics 2006(ICB), D. Zhang and A. Jain, Eds., vol. LNCS 3832,2005, pp. 265–272.

[17] Basisadministratie Persoonsgegevens en Reisdocumenten, 2b or not 2b:Doe mee aan de praktijkproef biometrie en ontvang 10 euro korting! Min-isterie van Binnenlandse Zaken en Koninkrijksrelaties, 2004.

[18] Basisadministratie Persoonsgegevens en Reisdocumenten, “Biometrie In-Zicht, April 2005 - Extra Editie,” Biometrie In-Zicht, vol. 1, no. 3, p. 1,2005.

[19] Basisadministratie Persoonsgegevens en Reisdocumenten, “Biometrie In-Zicht, Januari 2005,” Biometrie In-Zicht, vol. 1, no. 1, pp. 1–2, 2005.

[20] Basisadministratie Persoonsgegevens en Reisdocumenten, “Biometrie In-Zicht, Maart 2005,” Biometrie In-Zicht, vol. 1, no. 2, pp. 1–2, 2005.

[21] B. Bauer, “A century of microphones,” Journal of the AES (Audio Engi-neering Society), vol. 35, no. 4, pp. 246–258, April 1987.

[22] L. Berggren, “Iridology: A critical review,” Acta Ophthalmologica, vol. 63,no. 1, pp. 1–8, 1993.

[23] C. Bergman, “Multi-biometric match-on-card alliance formed,” BiometricTechnology Today, vol. 13, no. 5, p. 6, 2005.

[24] A. Bertillon, Identification Anthropometrique et Instructions Signale-tiques. Melun: Imprimerie administrative, 1893.

[25] C. Beumier and M. Acheroy, “3D facial surface acquisition by structuredlight,” in Proceedings of the International Workshop on Synthetic-NaturalHybrid Coding and Three Dimensional Imaging, 1999, pp. 103–106.

[26] C. Beumier and M. Acheroy, “Automatic 3D face authentication,” Imageand Vision Computing, vol. 18, no. 4, pp. 315–321, 2000.

134

Page 140: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[27] E. S. Bigun, J. Bigun, B. Duc, and S. Fischer, “Expert conciliation formultimodal person authentication systems using bayesian statistics,” inProceedings of the International Conference on Audio- and Video-BasedBiometric Person Authentication (AVBPA), vol. LNCS 1206, 1997, pp.291–300.

[28] F. Bimbot, G. Gravier, J.-F. Bonastre, C. Fredouille, S. Meignier, T. Mer-lin, I. Magrin-Chagnolleau, J. Ortega-Garcıa, D. Petrovska-Delacretaz,and D. Reynolds, “A tutorial on text-independent speaker verification,”Eurasip Journal on Applied Signal Processing, vol. 2004, no. 4, pp. 430–451, 2004.

[29] Biometric Technology Today, “Part 1: Biometrics and ePassports,” Bio-metric Technology Today, vol. 13, no. 6, pp. 10–11, 2005.

[30] Biometric Technology Today, “Biometric standards - an update,” Biomet-ric Technology Today, vol. 14, no. 1, pp. 10–11, 2006.

[31] Biometric Technology Today, “Biometric standards - an update,” Biomet-ric Technology Today, vol. 14, no. 1, pp. 7–9, 2006.

[32] D. Blackburn, M. Bone, and P. Phillips, “Facial recognition ven-dor test 2000, Evaluation report,” February 2001, available online athttp://www.frvt.org/.

[33] V. Blanz and T. Vetter, “A morphable model for the synthesis of 3Dfaces,” in Proceedings of the 26th International Conference on ComputerGraphics and Interactive Techniques, 1999, pp. 187–194.

[34] R. M. Bolle, J. H. Connell, S. Pankanti, N. K. Ratha, and A. W. Senior,Guide to Biometrics. New-York: Springer-Verlag, 2003.

[35] J. Bolling, “A window to your health,”Jacksonville Medicine (special issueon retinal diseases), vol. 51, no. 9, 2000.

[36] D. Bolme, R. Beveridge, M. Teixeira, and B. Draper, “The CSU face iden-tification evaluation system: Its purpose, features and structure,” in Pro-ceedings International Conference on Vision Systems 2003, Graz, Austria,April 2003, pp. 304–311.

[37] J.-F. Bonastre, F. Wils, and S. Meignier, “ALIZE, a free toolkit for speakerrecognition,” in Proceedings IEEE International Conference on Acoustics,Speech, and Signal Processing (ICASSP 2005), Philadelphia, USA, March2005, pp. 737–740.

[38] S. Bono, M. Green, A. Stubblefield, A. Juels, A. Rubin, and M. Szydlo,“Security analysis of a cryptographically-enabled RFID device,” 2005,draft version, available at http://rfidanalysis.org/.

[39] F. Botti, A. Alexander, and A. Drygajlo, “On compensation of mis-matched recording conditions in the bayesian approach for forensic au-tomatic speaker recognition,” Forensic Science International, vol. 146,no. S1, pp. S101–S106, December 2004.

135

Page 141: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[40] K. W. Bowyer, “Face recognition technology: Security versus privacy,”IEEE Technology and Society Magazine, vol. 23, no. 1, pp. 9–20, 2004.

[41] K. W. Bowyer, K. Chang, and P. J. Flynn, “A survey of 3D and multi-modal 3D and 2D face recognition,” Departement of Computer Scienceand Engineering of the University of Notre Dame, Tech. Rep., 2004.

[42] G. Bradski, “Programmer’s tool chest: The OpenCV library,” Dr. DobbsJournal, Nov. 2000.

[43] J.-J. Brault and R. Plamondon, “Segmenting handwritten signatures attheir perceptually important points,” IEEE Trans. Pattern Analysis andMachine Intelligence, vol. 15, no. 9, pp. 953–957, Sept. 1993.

[44] C. M. Brislawn, J. N. Bradley, R. J. Onyshczak, and T. Hopper, “The FBIcompression standard for digitized fingerprint images,” in Proceedings ofthe SPIE, vol. 2847, 1996, pp. 344–355.

[45] A. M. Bronstein, M. M. Bronstein, and R. Kimmel, “Expression-invariant3D face recognition,” in Proceedings of 4th International Conferenceon Audio- and Video-Based Biometric Person Authentication (AVBPA),J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 62–70.

[46] R. Brunelli and D. Falavigna, “Person identification using multiplecues,” IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), vol. 17, no. 10, pp. 955–966, 1995.

[47] Bundesamt fuer Sicherheit in der Informationstechnik, “BioFace compar-ative study of facial recognition systems - public final report,” Bundesamtfuer Sicherheit in der Informationstechnik, Godesberger Allee 185–189,53175 Bonn, Germany, Tech. Rep., June 2003.

[48] Bundesamt fuer Sicherheit in der Informationstechnik, “Evaluierung bio-metrischer Systeme Fingerabdrucktechnologien - BioFinger,” Bundesamtfuer Sicherheit in der Informationstechnik, Tech. Rep. Version 1.1, 2004.

[49] Bundesamt fuer Sicherheit in der Informationstechnik, “Untersuchungder leistungfahigkeiten von biometrischen verificationssystemen - biop ii,”Bundesamt fuer Sicherheit in der Informationstechnik, Tech. Rep. Version12.0, August 2005.

[50] M. Caligiuri, H.-L. Teulings, J. Filoteo, D. Song, and J. B. Lohr, “Quan-titative measurement of handwriting in the assessment of drug-inducedparkinsonism,” in Proc. 12th Biennial Conference of the InternationalGraphonomics Society, Salerno, Italy, June 2005, pp. 300–304.

[51] M. Caloyannides, “Does privacy really constrain security? or does it en-hance security in a paper-free world?” Keesing’s Journal of Documents &identity, no. 9, pp. 7–9, 2004.

[52] J. W. M. Campbell, “The role of biometrics in ID document issuance,”Keesing’s Journal of Documents & Identity, no. 4, pp. 6–8, 2004.

136

Page 142: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[53] J. Campbell and D. Reynolds, “Corpora for the evaluation of speakerrecognition systems,” in Proceedings IEEE International Conference onAcoustics, Speech, and Signal Processing (ICASSP’99), vol. 2, March 1999,pp. 829–832.

[54] A. Cavoukian, “Privacy and biometrics: An oxymoron or time to take a2nd look?” in Proceedings Computers, Freedom and Privacy 98, Austin,Texas, 1998, available online at http://www.ipc.on.ca/.

[55] Chairman of the Committee created by Article 6 of Regulation 1683/95laying down a uniform format for visas, “Note - technical feasibility ofthe integration of biometric identifiers into the uniform format...” Coun-cil of the European Union, Note 14534/04, November 2004, available athttp://www.statewatch.org/news/2004/dec/bio-visas.pdf.

[56] C. Champod, C. Lennard, P. Margot, and M. Stoilovic, Fingerprints andOther Ridge Impressions. Boca Raton: CRC Press, 2004.

[57] D. Chan, A. Fourcin, B. Gibbon, D.and Granstrom, M. Huckvale,G. Kokkinakis, K. Kvale, L. Lamel, B. Lindberg, A. Moreno, J. Mouropou-los, F. Senia, I. Trancoso, C. Veld, and J. Zeiliger, “EUROM - a spokenlanguage resource for the EU,” in Proceedings 4th European Conferenceon Speech Communication and Speech Technology (Eurospeech’95), vol. 1,Madrid, Spain, September 1995, pp. 867–870.

[58] H.-D. Chang, J.-F. Wang, and H.-M. Suen, “Dynamic handwritten chinesesignature verification,” in Proceedings second IEEE International Confer-ence on Document Analysis and Recognition, 1993, pp. 258–261.

[59] K. I. Chang, K. W. Bowyer, and P. J. Flynn, “Multi-modal 2D and 3Dbiometrics for face recognition,” in Proceedings of the IEEE InternationalWorkshop on Analysis and Modeling of Faces and Gestures (AMFG’03),2003, pp. 187–194.

[60] X. Chen, P. J. Flynn, and K. W. Bowyer, “Visible-light and infrared facerecognition,” in Proceedings of Workshop on Multimodal User Authentica-tion, 2003, pp. 48–55.

[61] C. C. Chibelushi, F. Deravi, and J. S. D. Mason,“A review of speech-basedbimodal recognition,” IEEE Transactions on Multimedia, vol. 4, no. 1, pp.23–37, 2002.

[62] C. S. Chua and R. Jarvis, “Point signatures: A new representation for 3Dobject recognition,” International Journal of Computer Vision, vol. 271,pp. 63–85, 1997.

[63] T. Clancy, N. Kiyavash, and D. Lin, “Secure smartcard-based fingerprintauthentication,” in Proceedings ACM Multimedia 2003 Workshop on Bio-metric Methods and Applications, Berkeley, USA, Nov. 2003.

[64] R. Clarke, “Human identification in information systems: Managementchallenges and public policy issues,” Information Technology and People,vol. 7, no. 4, pp. 6–37, 1994.

137

Page 143: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[65] R. Clarke, “Privacy impact assessments,” 2003, available online athttp://www.anu.edu.au/people/Roger.Clarke/DV/PIA.html.

[66] R. Clarke, “Identification and authentica-tion fundamentals,” 2004, available online athttp://www.anu.edu.au/people/Roger.Clarke/DV/IdAuthFundas.html.

[67] CNIL, “La carte d’identite electronique en Europe: Belgique, Italie,Espagne, Pays-Bas, Grande-Bretagne,” Commission Nationale del’Informatique et des Libertes, Tech. Rep., 2005.

[68] Commission des Communautes Europeennes, “Decision de la Commis-sion du 28/II/2005 etablissant les specifications techniques afferentes auxnormes pour les dispositifs de securite et les elements biometriques inte-gres dans les passeports et les documents de voyage delivres par les Etatsmembres,” Official Journal of the European Union, 2005.

[69] Confederation helvetique, “Loi federale sur les documents d’identite desressortissants suisses,” Recueil Systematique, 2001.

[70] Confederation helvetique, “Ordonnance federale sur les documentsd’identite des ressortissants suisses,” Recueil Systematique, 2002.

[71] T. Connie, A. Teoh, M. Goh, and D. Ngo, “Palmhashing: a novel approachfor cancelable biometrics,” Information Processing Letters, vol. 93, no. 1,pp. 1–5, 2005.

[72] Council of Europe, Convention for the Protection of Individuals with Re-gard to Automatic Processing of Personal Data. Council of Europe, 1981,no. 108, European Treaty Series.

[73] Council of Europe, Draft Progress Report on the Application of the Prin-ciples of Convention 108 to the Collection and Processing of BiometricData. Council of Europe, 2004.

[74] Council of the European Union, “Council Decision No 2004/512/EC es-tablishing the Visa Information System (VIS),” Official Journal of theEuropean Union, no. L 213, pp. 15–7, 2004.

[75] Council of the European Union, “Council regulation (EC) No 2252/2004of 13 December 2004 on standards for security features and biometricsin passports and travel documents issued by Member States,” OfficialJournal of the European Union, no. L 385, pp. 1–6, 2004.

[76] C. Crews, “Human bar code - monitoring biometric technologies in a freesociety,” Cato Institute, Policy Analysis 452, September 2002.

[77] J. Darch, B. Milner, X. Shao, S. Vaseghi, and Q. Yan, “Predicting for-mant frequencies from MFCC vectors,” in Proceedings IEEE InternationalConference on Acoustics, Speech, and Signal Processing (ICASSP 2005),Philadelphia, USA, March 2005, pp. 941–944.

[78] J. G. Daugman, “High confidence visual recognition of persons by testof statistical independence,” IEEE Transactions on Pattern Analysis andMachine Intelligence, vol. 15, no. 11, pp. 1148 – 1161, 1993.

138

Page 144: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[79] J. G. Daugman, “How iris recognition works,” IEEE Transactions on Cir-cuits and Systems for Video Technology, vol. 14, no. 1, pp. 21–30, 2004.

[80] J. G. Daugman and C. Downing, “Epigenetic randomness, complexity, andsingularity of human iris patterns,”Proceedings of the Royal Society Lond.B, vol. 268, pp. 1737–1740, 2001.

[81] J. Daugman, “Biop ii contreversy to be tackled,” Biometric TechnologyToday, vol. 13, no. 10, pp. 1–2, 2005.

[82] Department of State, United States of America, “Electronic passport, Pro-posed Rule,” Federal Register, vol. 70, no. 33, pp. 8305–8309, 2005.

[83] R. Derakhshani, S. A. Schuckers, L. A. Hornak, and L. O’Gorman, “De-termination of vitality from a non-invasive biomedical measurement foruse in fngerprint scanners,” Pattern Recognition, vol. 36, no. 2, p. 383 U396, 2005.

[84] G. R. Doddington, M. A. Przybocki, A. F. Martin, and D. A. Reynolds,“The nist speaker recognition evaluation - overview, methodology, systems,results, perspective,”Speech Communication, vol. 31, no. 2-3, pp. 225–254,June 2000.

[85] G. Doddington, “A method of speaker verification,” Ph.D thesis, Univer-sity of Wisconsin, Madison, USA, 1970.

[86] J. Dolfing, E. Aarts, and J. van Oosterhout, “On-line signature verificationwith Hidden Markov Models,” in Proceedings International Conference onPattern Recognition 1998, vol. 2, Aug. 1998, pp. 1309–1312.

[87] S. Elliott, “Differentiation of signature traits vis-a-vis mobile- and table-based digitizers,” ETRI Journal, vol. 26, no. 6, pp. 641–646, December2004.

[88] European Commission, Proposal for a Council Regulation amending Reg-ulations (EC) 1683/95 laying down a uniform format for visas and (EC)1030/02 laying down a uniform format for residence permits for third-country nationals. EUR-OP, 2003, no. COM(2003) 558 final.

[89] European Commission, Proposal for a Regulation of the European parlia-ment and of the Council concerning the Visa Information System (VIS)and the exchange of data between Member States on short stay-visas.EUR-OP, 2004, no. COM(2004) 835 final.

[90] European Commission, Proposal for a Council Decision concerning accessfor consultation of the Visa Information System (VIS) by the authoritiesof Member States responsible for internal security and by Europol for thepurposes of the prevention, detection and investigation of terrorist offencesand of other serious criminal offences. EUR-OP, 2005, no. COM(2005)600 final.

[91] K. Farrell, “Adaptation of data fusion-based speaker verification models,”in Proceedings IEEE International Symposium on Circuits and Systems(ISCAS’02), vol. 2, 2002, pp. 851–854.

139

Page 145: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[92] H. Fatemi, R. Kleihorst, H. Corporaal, and P. Jonker, “Real-time facerecognition on a smart camera,” in Proceedings of Acivs 2003 (AdvancedConcepts for Intelligent Vision Systems), Ghent, Belgium, Sept. 2003.

[93] H. Faulds, “On the skin-furrows on the hands,” Nature, vol. 22, p. 605,1880.

[94] J. Fierrez-Aguilar, J. Ortega-Garcia, D. Garcia-Romero, and J. Gonzalez-Rodriguez, “A comparative evaluation of fusion strategies for multimodalbiometric verification,” in Proceedings of 4th International Conferenceon Audio- and Video-Based Biometric Person Authentication (AVBPA),J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 830–837.

[95] J. Fierrez-Aguilar, J. Ortega-Garcia, and J. Gonzalez-Rodriguez, “Targetdependent score normalization techniques and their application to signa-ture verification,” in Proceedings Biometric Authentication: First Inter-national Conference (ICBA 2004), vol. LNCS 3072, Hong Kong, China,July 2004, pp. 498–504.

[96] J. Fontana, “A national identity card for canada?” Standing Committeeon Citizenship and Immigration, House Of Commons Canada, Tech. Rep.,2003.

[97] Forum des Droits sur l’Internet, “Projet de carte nationale d’identite elec-tronique,” Forum des Droits sur l’Internet,” Rapport, 16 Juin 2005.

[98] S. Fredrickson and L. Tarassenko, “Text-independent speaker recogni-tion using neural network techniques,” in Proceedings Fourth InternationalConference on Artificial Neural Networks, June 1995, pp. 13–18.

[99] R. W. Frischholz and U. Dieckmann, “BioID: A multimodal biometricidentification system,” IEEE Computer, vol. 33, no. 2, p. 1998.

[100] D. Fuentes, D. Mostefa, J. Kharroubi, S. Garcia-Salicetti, B. Dorizzi, andG. Chollet, “Identity verification by fusion of biometric data: on-line sig-natures and speech,” in Proceedings COST 275 Workshop on the Adventof Biometrics on the Internet, Nov. 2002, pp. 83–86.

[101] F. Galton, Finger Prints. London: Macmillian and Co., 1892.

[102] S. Garcia-Salicetti, C. Beumier, G. Chollet, B. Dorizzi, J. Leroux lesJardins, J. Lunter, Y. Ni, and D. Petrovska-Delacretaz, “BIOMET: Amultimodal person authentication database including face, voice, finger-print, hand and signature modalities,” in Proceedings of 4th InternationalConference on Audio- and Video-Based Biometric Person Authentication(AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 845–853.

[103] J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, N. Dahlgren, andV. Zue, “Timit acoustic-phonetic continuous speech corpus,” lDC catalognumber LDC93S1.

140

Page 146: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[104] J. Godfrey, D. Graff, and A. Martin, “Public databases for speaker recog-nition and verification,” in Proceedings ESCA Workshop on AutomaticSpeaker Recognition, Identification and Verification, Martigny, Switzer-land, April 1994, pp. 39–42.

[105] A. Goh and C. Ngo, “Computation of cryptographic keys from face bio-metrics,” Lecture Notes in Computer Science, vol. 2828, pp. 1–13, 2003.

[106] D. Graham and N. Allinson, Characterizing Virtual Eigensignatures forGeneral Purpose Face Recognition, ser. NATO ASI Series F, Computerand Systems Sciences, 1998, pp. 446–456.

[107] R. Gross, “Face databases,” in Handbook of Face Recognition, A. K. Jainand S. Z. Li, Eds. New-York: Springer-Verlag, 2005, ch. 13, pp. 301–327,to appear.

[108] Groupe de travail sur la protection des personnes a l’egard du traitementdes donnees a caractere personnel (G29), Avis No 7/2004 sur l’insertiond’elements biometriques dans les visas et titres de sejour en tenant comptede la creation du systeme d’information Visas (VIS). Union Europeenne,11 aout 2004.

[109] Y. Gu and T. Thomas,“A text-independent speaker verification system us-ing support vector machines classifier,” in Proceedings 7th European Con-ference on Speech Communication and Technology (EUROSPEECH 2001Scandinavia), Aalborg, Denmark, September 2001, pp. 1765–1768.

[110] C. Guerrier and L.-A. Cornelie, “Les aspects juridiques de la biometrie,”Institut national des telecommunications (INT), Tech. Rep., 2003.

[111] S. Guruprasad, N. Dhananjaya, and B. Yegnanarayana, “AANN modelsfor speaker recognition based on difference cepstrals,” in Proceedings In-ternational Joint Conference on Neural Networks, vol. 1, July 2003, pp.692–697.

[112] F. Hao, R. Anderson, and J. Daugman, “Combining cryptography withbiometrics effectively,” University of Cambridge Computer Laboratory,Tech. Rep., 2005.

[113] N. Herbst and L. C.N., “Automatic signature verification based on ac-celerometry,” IBM Journal of Research and Development, vol. 21, no. 3,pp. 245–253, 1977.

[114] W. Herschel, “Skin furrows on the hand,” Nature, vol. 23, p. 76, 1880.

[115] A. Hickling, C. Watson, and B. Ulery, “The myth of goats: How manypeople have fingerprints that are hard to match?” National Institute ofStandards and Technology (NIST), Tech. Rep. NISTIR 7271, 2005.

[116] A. Higgins and D. Vermilyea, “King speaker verification,” lDC catalognumber LDC95S22.

[117] H. Hollien, G. Dejong, C. Martin, R. Schwartz, and K. Liljegren, “Effectsof ethanol intoxication on speech suprasegmentals,” Acoustical Society ofAmerica Journal, vol. 110, no. 6, pp. 3198–3206, 2001.

141

Page 147: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[118] L. Hong and A. K. Jain, “Integrating faces and fingerprints for personalidentification,” IEEE Transactions on Pattern Analysis and Machine In-telligence, vol. 20, no. 12, pp. 1295–1307, 1998.

[119] L. Hong, A. K. Jain, and S. Pankanti, “Can multibiometrics improve per-formance?” Proceedings AutoID’99, pp. 59–64, 1999.

[120] C. Hook, J. Kempf, and G. Scharfenberg, “New pen device for biometrical3d pressure analysis of handwritten characters, words and signatures,”in Proceedings 2003 ACM SIGMM workshop on Biometrics methods andapplications, Berkeley, USA, November 2003, pp. 38–44.

[121] T. Hopper, “Compression of gray-scale fingerprint images,” in Proceedingsof the SPIE, vol. 2242, 1994, pp. 180–185.

[122] R.-L. Hsu and A. K. Jain, “Face modeling for recognition,” in ProceedingsInternational Conference on Image Processing (ICIP), 2001, pp. 693–696.

[123] P. HUSTINX, “Opinion of the european data protection supervisor onthe proposal for a council decision concerning access for consultation ofthe visa information system (vis) by the authorities of member statesresponsible for internal security and by europol for the purposes of theprevention, detection and investigation of terrorist offences and of otherserious criminal offences (com (2005) 600 final).” European Data Protec-tion Supervisor” Opinion, 20th of January 2006.

[124] ICAO, Document 9303 on Machine Readable Travel Document. Interna-tional Civil Aviation Organisation, 2003.

[125] ICAO, “Biometrics deployment of machine readable travel documents,technical report v. 2.0,” International Civil Aviation Organisation, Tech.Rep., 2004.

[126] ICAO, “Developpment of a logical data structure - LDS - for optional ca-pacity expansion technologies,” International Civil Aviation Organisation,Tech. Rep. Machine Readable Travel Documents, Revision - 1.7, 2004.

[127] ICAO, “Document 9303 on Machine Readable Travel Document, annexa - facial image size study 1,” International Civil Aviation Organisation,Tech. Rep., 2004.

[128] ICAO, “Document 9303 on Machine Readable Travel Document, annex a- photograph guidelines,” International Civil Aviation Organisation, Tech.Rep., 2004.

[129] ICAO, “Use of contactless integrated circuits in machine readable traveldocuments,” International Civil Aviation Organisation,” Biometrics De-ployment of Machine Readable Travel Documents, Annex I, 2004.

[130] International Biometric Group, “The bioprivacy initiative: A frameworkfor evaluating the privacy impact of biometric deployment and technolo-gies,” International Biometric Group (IBG), Tech. Rep. M1/03-0227, 2002.

142

Page 148: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[131] International Biometric Group, “Independent testing of iris recog-nition technology,” International Biometric Group, Final ReportNBCHC030114/0002, May 2005.

[132] International Labour Organization, Seafarers’ Identity Documents Con-vention: The standard for the biometric template required by the Conven-tion. International Labour Office, Geneva, Switzerland, 2004, no. 185,Revised.

[133] International Standards Organisation, ISO/IEC 7816. Geneva, Switzer-land: International Standards Organisation, 1998.

[134] International Standards Organisation, ISO/IEC JRC 1/SC 37 WorkingDraft, Biometric Data Interchange Formats - Part 2: Fingerprint Minu-tiae Data. Geneva, Switzerland: International Standards Organisation,2004.

[135] International Standards Organisation, ISO/IEC JRC 1/SC 37 WorkingDraft, Biometric Data Interchange Formats - Part 3: Fingerprint PatternSpectral Data. Geneva, Switzerland: International Standards Organisa-tion, 2004.

[136] International Standards Organisation, ISO/IEC JRC 1/SC 37 WorkingDraft, Biometric Data Interchange Formats - Part 4: Finger Image Data.Geneva, Switzerland: International Standards Organisation, 2004.

[137] International Standards Organisation, ISO/IEC JRC 1/SC 37 WorkingDraft, Biometric Data Interchange Formats - Part 5: Face iImage data.Geneva, Switzerland: International Standards Organisation, 2004.

[138] International Standards Organisation, ISO/IEC JRC 1/SC 37 WorkingDraft, Biometric Data Interchange Formats - Part 6: Iris Image Data.Geneva, Switzerland: International Standards Organisation, 2004.

[139] International Standards Organisation, ISO/IEC 19794-1:2005 (BioAPI2.0). Geneva, Switzerland: International Standards Organisation, 2005.

[140] A. Jain, F. Griess, and S. Connell, “On-line signature verification,”PatternRecognition, vol. 35, pp. 2963–2972, 2002.

[141] A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition:A review,” IEEE Transactions on Pattern Analysis and Machine Intelli-gence, vol. 22, no. 1, pp. 4–37, 2000.

[142] A. K. Jain, K. Nandakumar, and A. Ross, “Score normalization in multi-modal biometric systems,” Pattern Recognition, 2005, to appear.

[143] A. K. Jain, S. Prabhakar, and S. Chen, “Combining multiple matchers forhigh security fingerprint verification system,”Pattern Recognition Letters,vol. 20, no. 11-13, pp. 1371–1379, 1999.

[144] A. K. Jain, S. Prabhakar, L. Hong, and S. Pankanti, “Filterbank-based fin-gerprint matching,” IEEE Transaction on Image Processing, vol. 9, no. 5,pp. 846–859, 2000.

143

Page 149: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[145] A. K. Jain and A. Ross, “Learning user-specific parameters in a multi-biometric system,” in Proceedings International Conference on ImageProcessing (ICIP), vol. 1, 2002, pp. 57–60.

[146] A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometricrecognition,” IEEE Transactions on Circuits and Systems for Video Tech-nology, Special Issue on Image- and Video-Based Biometrics, vol. 14, no. 1,pp. 4–20, 2004.

[147] Joint Research Center, “Biometrics at the frontiers: Assessing the impacton society,” Joint Research Center, Institute for Prospective TechnologicalStudies, European Commission, Technical Report Series EUR 21585 EN,2005.

[148] A. Juels, D. Molnar, and D. Wagner, “Security and privacy issues in e-passports,” Cryptology ePrint Archive, Report 2005/095, 2005, availableonline at http://eprint.iacr.org/2005/095.

[149] L. Kanda, “Iris recognition on the move,” Biometric Technology Today,vol. 13, no. 10, p. 1, 2005.

[150] I. Karlsson, T. Banziger, T. Dankovicova, J.and Johnstone, J. Lindberg,H. Melin, F. Nolan, and K. Scherer, “Speaker verification with elicitedspeaking styles in the verivox project,” Speech Communication, vol. 31,no. 2-3, pp. 121–129, June 2000.

[151] R. Kashi, J. Hu, W. Nelson, and W. Turin, “A Hidden Markov Model ap-proach to online handwritten signature recognition,” International Journalon Document Analysis and Recognition, vol. 1, no. 2, pp. 102–109, 1998.

[152] R. S. Kashi, W. T. Turin, and W. L. N. Nelson, “On-line handwrittensignature verification using stroke direction coding,” Optical Engineering,vol. 35, no. 9, pp. 2526–2533, September 1996.

[153] Keesing’s Journal of Documents & Identity, “e-Passports 2005-2006 - TheState of the e-passport industry, institution = Keesing’s Journal of Doc-uments & Identity, type = Annual Report, month=, number=, year =2006,” Tech. Rep.

[154] H. Ketabdar, J. Richiardi, and A. Drygajlo, “Global feature selection foron-line signature verification,” in Proceedings International GraphonomicsSociety 2005 Conference, Salerno, Italy, June 2005.

[155] S. King, H. Harrelson, and G. Tran, “Testing iris and face recogni-tion in a personnel identification application,” 2002, available online athttp://www.aclu.org/Privacy/Privacy.cfm?ID=13556&c=130.

[156] T. Kinneging, “The ICAO PKI scheme: what’s in it for me?” Keesing’sJournal of documents and identity, no. 9, pp. 3–6, 2004.

[157] T. A. F. Kinneging, “PKI for machine readable travel documents offeringICC read-only access,” International Civil Aviation Organisation, Tech.Rep. Machine Readable Travel Documents, PKI Task Force, 2004.

144

Page 150: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[158] P. L. Kirk, “The ontogeny of criminalistics,” Journal of Criminal Law,Criminology and Police Science, vol. 54, pp. 235–238, 1963.

[159] M. Kosmerli, T. Fladsrud, E. Hjelmas, and E. Snekkenes, “Face recogni-tion issues in a border controll environment,” in Proceedings of Interna-tional Conference on Biometrics 2006(ICB), D. Zhang and A. Jain, Eds.,vol. LNCS 3832, 2005, pp. 33–39.

[160] K. Kryszczuk and A. Drygajlo, “Adressing vulnerabilities of likelihood-ratio-based face verification,” in Proceedings of 6th International Con-ference on Audio- and Video-Based Biometric Person Authentication(AVBPA), T. Kanade and N. R. (A.K.) Jain, Eds., vol. LNCS 3546, 2005,pp. 426–435.

[161] A. Kumar, D. C. Wong, H. C. Shen, and A. K. Jain, “Personal verifica-tion using palmprint and hand geometrybiometric,” in Proceedings of 4thInternational Conference on Audio- and Video-Based Biometric PersonAuthentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688,2003, pp. 668 – 678.

[162] A. Kundu, Y. He, and P. Bahl, “Recognition of handwritten word: Firstand second order hidden markov model based approach,” Pattern Recog-nition, vol. 22, no. 3, pp. 283–297, 1989.

[163] K. K. Lau, P. C. Yuen, and Y. Y. Tang, Advances in Handwriting Recog-nition. World Scientific, 1999, ch. An efficient Function-based On-lineSignature Recognition System, pp. 559–568.

[164] Y. LeCun, F.-J. Huang, and L. Bottou, “Learning methods for genericobject recognition with invariance to pose and lighting,” in Proceedingsof IEEE CVPR’04 (Computer Vision and Pattern Recognition). IEEEPress, 2004.

[165] L. Lee, T. Berger, and E. Aviczer, “Reliable on-line human signature ver-ification systems,” IEEE Transactions on pattern analysis and machineintelligence, vol. 18, no. 6, pp. 643–647, June 1996.

[166] E. Locard, L’identification des recidivistes. Paris: A. Maloine, 1909.

[167] E. Locard, La preuve judiciaire par les empreintes digitales. Paris:Archives d’anthropologie criminelle de medecine legale et de psychologienormale et pathologique, 1914.

[168] X. Lu, Y. Wang, and A. K. Jain, “Combining classifiers for face recog-nition,” in Proceedings International Conference on Multimedia and Expo(ICME), vol. 3, 2003, pp. 13–16.

[169] R. Luis-Garcia, C. Alberola-Lopez, O. Aghzout, and J. Ruiz-Alzola, “Bio-metric identification systems,” Signal Processing, vol. 83, pp. 2539–2557,2003.

[170] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain,“FVC2000: Fingerprint verification competition,” IEEE Transactions onPattern Analysis and Machine Intelligence, vol. 24, no. 2, pp. 402–412,2000.

145

Page 151: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[171] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain,“FVC2002: Second fingerprint verification competition,” in Proceedingsof the 16th International Conference on Pattern Recognition, vol. 3, 2002,pp. 811– 814.

[172] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain,“FVC2004: Third fingerprint verification competition,” in Proceedings ofthe First International Conference on Biometric Authentication, vol. 3072,2004, pp. 1–7.

[173] D. Maltoni, D. Maio, A. K. Jain, and S. Prabhakar, Handbook of Finger-print Recognition. New-York: Springer-Verlag, 2003.

[174] G. Manoranjan, “JPEG 2000 - a versatile image compression standard,”Keesing’s Journal of Documents & Identity, no. 9, pp. 26–29, 2004.

[175] A. J. Mansfield, G. Kelly, D. Chandler, and J. Kane, “Biometric producttesting: Final report,” National Physical Laboratory, Center fo Mathe-matics and Scientific Computing, Tech. Rep. Issue 1.0, 2002.

[176] A. J. Mansfield and J. L. Waymann, “Best practices in testing and re-porting performance of biometric devices,” National Physical Laboratory,Center fo Mathematics and Scientific Computing, Tech. Rep. Version 2.01,2002.

[177] MAOSCO Limited, MULTOS Standard C API, MAO-DOC-REF-016.London, United Kingdom: MAOSCO Limited, 2004, available fromhttp://www.multos.com.

[178] M. W. Marcellin, M. J. Gormish, A. Bilgin, and M. P. Boliek,“An overviewof JPEG-2000,” in Proceedings of IEEE Data Compression Conference,2000, pp. 523–541.

[179] A. Martin, G. Doddington, T. Kamm, M. Ordowsk, and M. Przybock,“The DET curve in assessment of detection task performance,” in Pro-ceedings of EuroSpeech, vol. 4, 1997, pp. 1895–1898.

[180] A. Martinez and R. Benavente, “The ar face database,” Computer VisionCenter, CVC Technical Report 24, June 1998.

[181] L. Masek,“Recognition of human iris patterns for biometric identification,”Bachelor of Engineering degree, School of Computer Science and SoftwareEngineering, University of Wester Austrialia, 2003.

[182] J. Mathyer, “Quelques remarques sur le problemes de la securite des piecesd’identite et des pieces de legitimation: une solution interessante,” RevueInternationale de Police Criminelle, vol. 35, no. 336, pp. 66–79, 1980.

[183] T. Matsui and S. Furui, “Comparison of text-independent speaker recogni-tion methods using vq-distortion and discrete/continuous hmm’s,” Speechand Audio Processing, IEEE Transactions on, vol. 2, no. 3, pp. 456–459,July 1994.

[184] T. Matsumoto, “Gummy finger and paper iris: An update,” in Proceedingsof the 2004 Workshop on Information Security Research, 2004.

146

Page 152: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[185] T. Matsumoto, K. Matsumto, K. Yamada, and S. Hoshino, “Impact ofartificial gummy fingers on fingerprint systems,” in Proceedings of SPIE,vol. 4677, 2002, pp. 275–289.

[186] G. Medioni and R. Waupotitsch, “Face modeling and recognition in 3-D,” in Proceedings of the IEEE International Workshop on Analysis andModeling of Faces and Gestures (AMFG’03), 2003, pp. 232–233.

[187] H. Melin, “Databases for speaker recognition: Activities in COST250working group 2,” in Proceedings COST250 Workshop on Speaker Recog-nition in Telephony, Rome, Italy, Nov 1999.

[188] J. Menn, “Fraud rings taps into credit data,” Los Angeles Times, 2005.

[189] K. Messer, J. Kittler, M. Sadeghi, M. Hamouz, A. Kostin, F. Cardinaux,S. Marcel, S. Bengio, C. Sanderson, N. Poh, Y. Rodriguez, J. Czyz, L. Van-dendorpe, C. McCool, S. Lowther, S. Sridharan, V. Chandran, R. P. Pala-cios, E. Vidal, L. Bai, L. Shen, Y. Wang, C. Yueh-Hsuan, L. Hsien-Chang,H. Yi-Ping, A. Heinrichs, M. Muller, A. Tewes, C. von der Malsburg,R. Wurtz, Z. Wang, F. Xue, Y. Ma, Q. Yang, C. Fang, X. Ding, S. Lucey,R. Goss, and H. Schneiderman, “Face authentication test on the BANCAdatabase,” in Proceedings 17th International Conference on Pattern Recog-nition (ICPR), vol. 4, Cambridge, United Kingdom, August 2004, pp.523–532.

[190] K. Messer, J. Kittler, M. Sadeghi, M. Hamouz, A. Kostyn, S. Mar-cel, S. Bengio, F. Cardinaux, C. Sanderson, N. Poh, Y. Rodriguez,K. Kryszczuk, J. Czyz, L. Vandendorpe, J. Ng, H. Cheung, and B. Tang,“Face authentication competition on the BANCA database,” in Proceed-ings first International Conference on Biometric Authentication (ICBA),Hong Kong, July 2004, pp. 8–15.

[191] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, “XM2VTSDB:The extended M2VTS database,” in Proceedings of 2nd InternationalConference on Audio- and Video-Based Biometric Person Authentication(AVBPA), 1999, pp. 72–77.

[192] K. Messer, J. Kittler, J. Short, G. Heusch, F. Cardinaux, S. Marcel, Y. Ro-driguez, S. Shan, Y. Su, W. Gao, and X. Chen, “Performance characteri-sation of face recognition algorithms and their sensitivity to severe illumi-nation changes,” in Proceedings of International Conference on Biometrics2006(ICB), D. Zhang and A. Jain, Eds., vol. LNCS 3832, 2005, pp. 1–11.

[193] W. Mikhael and P. Premakanthan, “Speaker recognition employing wave-form based signal representation in nonorthogonal multiple transform do-mains,” in Proceedings IEEE International Symposium on Circuits andSystems (ISCAS’02), vol. 2, May 2002, pp. 608–611.

[194] B. Miller, “Vital signs of identity,” IEEE Spectrum, vol. 31, no. 2, pp.22–30, 1994.

[195] J. Miller, “Dermatoglyphics,” Journal of Investigative Dermatology,vol. 60, no. 6, pp. 435–442, 1973.

147

Page 153: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[196] B. Moghaddam and A. Pentland, “Probabilistic visual learning for objectrepresentation,” IEEE Transactions on Pattern Analysis and Machine In-telligence, vol. 19, no. 7, pp. 696–710, July 1997.

[197] C. Mund and P.-Y. Baumann, “Legal and societal aspects of exchange andstorage of biometric data,” in Biometrics 2005 - Technology and SocietalImpacts, September 2005.

[198] I. Nakanishi, H. Sakamoto, Y. Itoh, and Y. Fukui, “Multi-matcher on-line signature verification system in DWT domain,” in Proceedings 2005IEEE International Conference on Acoustics, Speech, and Signal Process-ing, Philadelphia, USA, March 2005, pp. 965–968.

[199] V. Nalwa, “Automatic on-line signature verification,” Proceedings of theIEEE, vol. 82, no. 2, pp. 215–239, February 1997.

[200] NIST, “NIST standards for biometric accuracy, tamper resistance, andinteroperability,”National Institute of Standards and Technology (NIST),Tech. Rep. 303A Report, Appendix A, 2003.

[201] NIST, “Use of technology standards and interoperable databases withmachine-readable tamper-resistant travel document,” National Instituteof Standards and Technology (NIST), Tech. Rep., 2003.

[202] R. Norton, “The evolving biometric marketplace to 2006,”Biometric Tech-nology Today, vol. 10, no. 9, pp. 7–8, 2002.

[203] OECD, “Biometric-based technologies,” Organisation for Economic Co-operation and Development (OECD), Tech. Rep., 2004.

[204] J. Oglesby, J. Mason, “Speaker recognition with a neural classifier,” inSpeech 88: Proceedings of the 7th Federation of Acoustical. Societies ofEurope (FASE) Symposium, Edinburgh, UK, 1988, pp. 1357–1363.

[205] J. Oglesby, J. Mason, “Speaker recognition with a neural classifier,” inProceedings First IEE International Conference on artificial Neural Net-works, vol. 313, October 1989, pp. 306–309.

[206] J. Oglesby, J. Mason, “Radial basis function networks for speaker recogni-tion,” in Proceedings IEEE International Conference on Acoustics, Speech,and Signal Processing (ICASSP’91), vol. 1, April 1991, pp. 393–396.

[207] J. Ortega-Garcia, J. Fierrez-Aguilar, D. Simon, J. Gonzalez, M. Faundez-Zanuy, V. Espinosa, A. Satue, I. Hernaez, J.-J. Igarza, C. Vivaracho,D. Escudero, and Q.-I. Moro, “MCYT baseline corpus: A bimodal biomet-ric database,” in IEE Proceedings - Vision, Image and Signal Processing,vol. 150, no. 6, 2003, pp. 395–401.

[208] J. Ortega-Garcia, J. Gonzalez-Rodriguez, and V. Marrero-Aguiar, “AHU-MADA: A large speech corpus in spanish for speaker characterization andidentification,” Speech Communication, vol. 31, pp. 255–264, 2000.

148

Page 154: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[209] M. Pandit and J. Kittler, “Feature selection for a DTW-based speakerverification system,” in Proceedings IEEE International Conference onAcoustics, Speech, and Signal Processing (ICASSP’98), vol. 2, 1998, pp.769–772.

[210] M. Parizeau and R. Plamondon, “A comparative analysis of regional cor-relation, dynamic time warping, and skeletal tree matching for signatureverification,” IEEE Transactions on Pattern Analysis and Machine Intel-ligence, vol. 12, no. 7, pp. 710–717, July 1990.

[211] J. Peters, “Enhanced security for chip-based ID cards,” Keesing’s Journalof Documents & Identity, no. 7, pp. 9–11, 2004.

[212] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoff-man, J. Marques, J. Min, and W. Worek, “Overview of the Face Recogni-tion Grand Challenge,” in Proceedings of IEEE Conference on ComputerVision and Pattern Recognition, 2005, to appear.

[213] P. Phillips, P. Grother, R. Micheals, D. BoneBlackburn, E. Tabassi, andM. Bone, “Facial recognition vendor test 2002, Evaluation report,” March2003, available online at http://www.frvt.org/.

[214] P. J. Phillips, A. Martin, C. L. Wilson, and M. Przybocki, “An introduc-tion to evaluation biometric systems,” IEEE Computer, pp. 56–63, 2000.

[215] P. Phillips, H. Moon, S. Rizvi, and P. Rauss, “The FERET evaluationmethodology for face-recognition algorithms,” IEEE Transactions on Pat-tern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104,2000.

[216] P. Phillips, H. Wechsler, J. Huang, and P. Rauss, “The FERET databaseand evaluation procedure for face-recognition algorithms,” Image and Vi-sion Computing Journal, vol. 16, no. 5, pp. 295–306, 1998.

[217] R. Plamondon, J. Brault, and P. Robillard, “Optimizing the design of anaccelerometer pen for signature verification,” in Proceedings 1983 Con-ference on Crime Countermeasures and Security, Lexington, USA, May1983, pp. 35–40.

[218] R. Plamondon and G. Lorette, “On-line signature verification: how manycountries are in the race?” in Proceedings IEEE International CarnahanConference on Security Technology, Zuerich, Switzerland, 1989, pp. 183–191.

[219] F. L. Podio, J. S. Dunn, L. Reinert, C. J. Tilton, L. O’Gorman, M. P. Col-lier, M. Jerde, and B. Wirtz, Common Biometric Exchange File Format(CBEFF) - NISTIR 6529. USA: The National Institute of Standardsand Technology (NIST), 2001.

[220] A. Poritz, “Linear predictive hidden markov models and the speech signal,”in Proceedings IEEE International Conference on Acoustics, Speech, andSignal Processing (ICASSP’82), vol. 7, May 1982, pp. 1291–1294.

149

Page 155: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[221] S. Prabhakar and A. K. Jain, “Decision-level fusion in fingerprint verifica-tion,” Pattern Recognition, vol. 35, no. 4, pp. 861–874, 2002.

[222] S. Prabhakar and A. K. Jain, “Fingerprint matching,” in Automatic Fin-gerprint Recognition Systems, N. Ratha and B. Ruud, Eds. London:Springer-Verlag, 2004, ch. 11, pp. 229–248.

[223] S. Prabhakar, S. Pankanti, and A. K. Jain, “Biometric recognition: Secu-rity and privacy concerns,” IEEE Security and Privacy, vol. 1, no. 2, pp.33–42, 2003.

[224] Presidency of the European Union, “Note - minimum common stan-dards for national identity cards,” Council of the European Union, Note11092/05, 11th of July 2005.

[225] Privacy International et al., “An open let-ter to the ICAO,” March 2004, available athttp://www.privacyinternational.org/issues/terrorism/rpt/icaoletter.pdf.

[226] H. Proenca and L. A. Alexandre, “Ubiris: A noisy iris image database,”Tech. Rep., 2005, iSBN 972-99548-0-1.

[227] Prepose Federal a la Protection des Donnees, 11eme Rapport d’activites2003/2004 du PFPD. Confederation Helvetique, 2004.

[228] Prepose Federal a la Protection des Donnees, 12eme Rapport d’activites2004/2005 du PFPD. Confederation Helvetique, 2005.

[229] J. Putz-Leszczynska and A. Pacut, “Dynamic time warping in subspacesfor on-line signature verification,” in Proc. 12th Biennial Conference ofthe International Graphonomics Society, Salerno, Italy, June 2005, pp.108–112.

[230] N. K. Ratha, J. H. Connell, and R. M. Bolle, “Enhancing security and pri-vacy in biometrics-based authentication systems,” IBM Systems Journal,vol. 40, no. 3, pp. 614–634, 2001.

[231] D. Reynaert and H. Van Brussel, “Design of an advanced computer writingtool,” in Proceedings sixth International Symposium on Micro Machine andHuman Science (MHS ’95), October 1995, pp. 229–234.

[232] D. Reynolds, “Speaker identification and verification using gaussian mix-ture speaker models,” Speech Communication, vol. 17, pp. 91–108, 1995.

[233] D. Reynolds, W. Campbell, T. Gleason, C. Quillen, D. Sturim, P. Torres-Carrasquillo, and A. Adami, “The 2004 MIT Lincoln laboratory speakerrecognition system,” in Proceedings IEEE International Conference onAcoustics, Speech, and Signal Processing (ICASSP 2005), Philadelphia,USA, March 2004, pp. 177–180.

[234] D. Reynolds, T. Quatieri, and R. Dunn, “Speaker verification usingadapted Gaussian mixture models,” Digital Signal Processing, vol. 10, no.1–3, pp. 19–41, 2000.

150

Page 156: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[235] T. Rhee, S. Cho, and J. Kim, “On-line signature verification using model-guided segmentation and discriminative feature selection for skilled forg-eries,” in Proceedings Sixth International Conference on Document Analy-sis and Recognition, September 2001, pp. 645–649.

[236] J. Richiardi and A. Drygajlo, “Gaussian mixture models for on-line sig-nature verification,” in International Multimedia Conference, Proceedings2003 ACM SIGMM workshop on Biometrics methods and applications,Berkeley, USA, Nov. 2003, pp. 115–122.

[237] J. Richiardi, P. Prodanov, and A. Drygajlo, “A probabilistic measureof modality reliability in speaker verification,” in Proceedings IEEE In-ternational Conference on Acoustics, Speech and Signal Processing 2005,Philadelphia, USA, March 2005.

[238] F. Roli, J. Kittler, G. Fumera, and D. Muntoni, “An experimental compar-ison of classifier fusion rules for multimodal personal identity verificationsystems,” in Proceedings of the Third International Workshop on MultipleClassifier Systems, 2002, pp. 325–335.

[239] A. Rosenberg, C. Lee, and S. Gokcen, “Connected word talker verifi-cation using whole word hidden markov models,” in Proceedings IEEEInternational Conference on Acoustics, Speech, and Signal Processing(ICASSP’91), vol. 1, April 1991, pp. 381–384.

[240] P. Rosenzweig, A. Kochems, and A. Schwartz, “Biometric technologies:Security, legal, and policy implications,” Legal Memorandum, vol. 12, pp.1–10, 2004.

[241] A. Ross and A. K. Jain, “Information fusion in biometrics,”Pattern Recog-nition Letters, vol. 24, pp. 2215–2125, 2003.

[242] A. Ross and A. K. Jain,“Multimodal biometrics: An overview,” in Proceed-ings of 12th European Signal Processing Conference, 2004, pp. 1221–1224.

[243] H. Rowley, S. Baluja, and T. Kanade, “Neural network-based face detec-tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 20, no. 1, pp. 23–38, January 1998.

[244] F. Rumsey and T. McCormick, Sound and recording: an introduction,3rd ed. Oxford, UK: Focal press, 1997.

[245] D. Sakamoto, H. Morita, T. Ohishi, Y. Komiya, and T. Matsumoto, “On-line signature verification incorporating pen position, pen pressure and peninclination trajectories,” in Proceedings 2001 IEEE International Confer-ence on Acoustics, Speech and Signal Processing, May 2001, pp. 7–11,dynamic Time Warping.

[246] F. Samaria and A. Harter, “Parameterisation of a stochastic model forhuman face identification,” in Proceedings of 2nd IEEE Workshop on Ap-plications of Computer Vision, Sarasota, United States of America, De-cember 1994.

151

Page 157: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[247] Y. Sato and K. Kogure, “Online signature verification based on shape,motion, and writing pressure,” in Proceedings 6th International Conferenceon Pattern Recognition, 1982, pp. 823–826.

[248] A. Scheenstra, A. Ruifrok, and R. C. Veltkamp,“A survey of 3D face recog-nition methods,” in Proceedings of the International Conference on Audio-and Video-Based Biometric Person Authentication (AVBPA), 2005, to bepublished.

[249] M. Schmidt and H. Gish, “Speaker identification via support vector classi-fiers,” in Proceedings IEEE International Conference on Acoustics, Speech,and Signal Processing (ICASSP’96), vol. 1, May 1996, pp. 105–108.

[250] B. Schneier, Applied Cryptography, 2nd ed. John Wiley and Sons Inc.,1996.

[251] S. A. C. Schuckers, “Spoofing and anti-spoofing measures,” InformationSecurity technical report, vol. 7, no. 4, pp. 56–62, 2002.

[252] R. Schwartz, S. Roucos, and M. Berouti, “The application of probabil-ity density estimation to text-idenpendent speaker identification,” in Pro-ceedings IEEE International Conference on Speech, Acoustics, and SignalProcessing, 1982, pp. 1649–1652.

[253] J. Short, J. Kittler, and K. Messer, “Comparison of photometric normal-isation algorithms for face verification,” in Proceedings 6th IEEE Inter-national Conference on Automatic Face and Gesture Recognition (FGR2004), 2004, pp. 254– 259.

[254] T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, and ex-pression database,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 25, no. 12, pp. 1615–1618, December 2003.

[255] J. Simitian, “SB 682, Identity Information Protection Act of 2005,” Cali-fornia State Senate, March 2005, available at http://www.sen.ca.gov/.

[256] L. Sirovich and M. Kirby, “Low-dimensional procedure for the characteri-zation of human faces,” Journal of the Optical Society of America, vol. 4,no. 3, pp. 519–524, 1987.

[257] R. Snelick, U. Uludag, A. Mink, M. Indovina, and A. K. Jain, “Largescale evaluation of multimodal biometric authentication using state-of-the-art systems,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 27, no. 3, pp. 450–455, 2005.

[258] F. Soong, A. Rosemberg, L. Rabiner, and B. Juang, “A vector quantiza-tion approach to speaker recognition,” in Proceedings IEEE InternationalConference on Speech, Acoustics, and Signal Processing, 1985, pp. 387–390.

[259] C. Soutar, D. Roberge, A. Stoianov, R. Gilroy, and B. V. Kumar,“Biometric encryption using image processing,” in Proc. SPIE OpticalSecurity and Counterfeit Deterrence Techniques II, R. L. van Renesse,Ed., vol. 3314, no. 1. SPIE, 1998, pp. 178–188. [Online]. Available:http://link.aip.org/link/?PSI/3314/178/1

152

Page 158: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[260] S. T. C. (St.-Petersburg), “STC russian speech database,” eLDA cataloguenumber S0050.

[261] J. Stanley and B. Steinhardt,“Drawing a blank: The failure of facial recog-nition technology in Tampa, Florida,” American Civil Liberties Union,”ACLU special report, January 2002.

[262] R. Suikerbuik, H. Tangelder, H. Daanen, and A. Oudenhuijzen, “Auto-matic feature detection in 3D human body scans,” in Proceedings of theConference SAE Digital Human Modelling for Design and Engineering,2004.

[263] K. Sung and T. Poggio, “Example-based learning for view-based humanface detection,” IEEE Trans. Pattern Analysis and Machine Intelligence,vol. 20, pp. 39–51, 1997.

[264] G. Swain, “A smart alternative: an introduction to high-density two-dimensional barcodes,”Keesing’s Journal of Documents & Identity, no. 8,pp. 9–11, 2004.

[265] The London School of Economics & Political Science, “The identityproject: An assessment of the UK Identity Cards Bill & its implications,”The London School of Economics & Political Science, Tech. Rep. InterimReport, March 2005.

[266] C.-L. Tisse, “Contribution a la verification biometrique de personnes parreconnaissance de l’iris,” Ph.D, Universite de Monpellier II, 2003.

[267] F. Tsalakanidou, S. Malassiotis, and M. G. Strintzis, “Integration of 2Dand 3D images for enhanced face authentication,” in Proceedings of 6thIEEE International Conference on Automatic Face and Gesture Recogni-tion (FGR’04), 2004, pp. 266– 271.

[268] M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of Cogni-tive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991.

[269] P. Tuyls and J. Goseling, “Capacity and examples of template protectingbiometric authentication systems,” in Proceedings Biometric Authentica-tion Workshop, Prague, 2004, pp. 158–170, lNCS 4677.

[270] UK Passport Service, “Biometrics enrolment trial,”United Kingdom Pass-port Service,” Report, May 2005.

[271] United Kingdom, Identity Cards Bill. U.K. Home Office, 2004.

[272] C. Van Oel, W. Baare, H. Hulshoff Pol, J. Haag, J. Balazs, A. Dingemans,R. Kahn, and M. Sitskoorn, “Differentiating between low and high suscep-tibility to schizophrenia in twins: The significance of dermatoglyphic in-dices in relation to other determinants of brain development,”Schizophre-nia Research, vol. 52, no. 3, pp. 181–193, 2001.

[273] R. L. van Renesse, “Implications of applying biometrics to travel-documents,” in Proceedings of SPIE, vol. 4677, 2002, pp. 290–298.

153

Page 159: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[274] J. Verbov, “Clinical significance and genetics of epidermal ridges - a reviewof dermatoglyphics,” Journal of Investigative Dermatology, vol. 54, no. 4,pp. 261–271, 1970.

[275] C. Vielhauer, R. Steinmetz, and A. Mayerhofer, “Biometric hash basedon statistical features of online signatures,” in Proc. 16th InternationalConference on Pattern Recognition (ICPR), vol. 1, 2002.

[276] J.-P. Walter, “Quelques aspects de protection des donnees lorsde l’utilisation de donnees biometriques dans le secteur prive,”in 26th international conference of the data protection and pri-vacy commissioners, Wroclaw, Poland, September 2004, available athttp://26konferencja.giodo.gov.pl/program/j/en.

[277] V. Wan and S. Renals, “Speaker verification using sequence discrimi-nant support vector machines,” IEEE Transactions on Speech and AudioProcessing, vol. 13, no. 2, pp. 203–210, March 2005.

[278] Y. Wang, T. Tan, and A. K. Jain, “Combining face and iris biometricsfor identity verification,” in Proceedings of 4th International Conferenceon Audio- and Video-Based Biometric Person Authentication (AVBPA),J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 805–813.

[279] C. Watson, C. Wilson, K. Marshall, M. Indovina, and R. Snelick, “Studiesof one-to-one fingerprint matching with vendor sdk matchers,” NationalInstitute of Standards and Technology (NIST), Tech. Rep., 2003.

[280] J. L. Wayman, A. K. Jain, D. Maltoni, and D. Maio, “An introduction tobiometric authentication systems,” in Biometric Systems: Technology, De-sign and Performance Evaluation, J. L. Wayman, A. K. Jain, D. Maltoni,and D. Maio, Eds. London: Springer-Verlag, 2005, ch. 1, pp. 1–20.

[281] R. Wildes, “Iris recognition,” in Biometric Systems: Technology, Designand Performance Evaluation, J. L. Wayman, A. K. Jain, D. Maltoni, andD. Maio, Eds. London: Springer-Verlag, 2005, ch. 3, pp. 63–95.

[282] R. P. Wildes, “Iris recognition: An emerging biometric technology,” inProceedings of the IEEE, vol. 85, no. 9, 1997, pp. 1348–1363.

[283] R. P. Wildes, J. C. Asmuth, G. L. Green, S. C. Hsu, R. J. Kolczynski, J. R.Matey, and S. E. McBride, “A system for automated iris recognition,” inProceedings of the Second IEEE Workshop on Applications of ComputerVision, 1994, pp. 121–128.

[284] G. O. Williams, “Iris recognition technology,” Iridian technologies, Tech.Rep., 2001.

[285] C. L. Wilson, M. D. Garris, and C. I. Watson, “Matching performancefor the US-VISIT system using flat fingerprints,” National Institute ofStandards and Technology (NIST), Tech. Rep. NISTIR 7110, 2004.

[286] C. Wilson, R. A. Hicklin, H. Korves, B. Ulery, M. Zoepfl, M. Bone,P. Grother, R. Micheals, S. Otto, and C. Watson, “Fingerprint vendortechnology evaluation 2003: Summary of results and analysis report,” Na-tional Institute of Standards and Technology (NIST), Tech. Rep., 2004.

154

Page 160: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[287] C. L. Wilson, C. I. Watson, M. D. Garris, and A. Hicklin, “Studies of fin-gerprint matching using the NIST verification test bed (VTB),” NationalInstitute of Standards and Technology (NIST), Tech. Rep. NISTIR 7020,2004.

[288] L. Wiskott, J.-M. Fellous, N. Kuiger, and C. von der Malsburg, “Facerecognition by elastic bunch graph matching,” IEEE Transactions on Pat-tern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775–779, July1997.

[289] S. S. Wood and C. L. Wilson, “Studies of plain-to-rolled fingerprint match-ing using the NIST algorithmic test bed (ATB),” National Institute ofStandards and Technology (NIST), Tech. Rep. NISTIR 7112, 2004.

[290] J. D. Woodward, “Biometrics: Privacy’s foe or privacy’s friend?” in Pro-ceedings of the IEEE, vol. 85, no. 9, 1997, pp. 1480–1492.

[291] J. D. Woodward, C. Horn, J. Gatune, and A. Thomas, “Biometrics : Alook at facial recognition,”RAND Documented Briefing, Tech. Rep., 2003.

[292] J. Wu, “Rotation invariant classification of 3d surface texture using photo-metric stereo,”Ph.D, Departement of Computer Science, School of Mathe-matical and Computer Sciences, Heriot-Watt University, Edinburgh, UK,2003.

[293] Q.-Z. Wu, I.-C. Jou, and S.-Y. Lee, “On-line signature verification usingLPC cepstrum and neural networks,” IEEE Trans. on systems, man andcybernetics, Part B, vol. 27, no. 1, pp. 148–153, Feb. 1997.

[294] N. Yager and A. Amin, “Fingerprint verification based on minutiae fea-tures: a review,” Pattern Analysis and Application, vol. 17, pp. 94–113,2004.

[295] L. Yang, B. Widjaja, and R. Prasad, “Application of hidden Markov mod-els for signature verification,”Pattern Recognition, vol. 28, no. 2, pp. 161–170, 1995.

[296] B. Yanikoglu and A. Kholmatov, “An improved decision criterion for gen-uine/forgery classification in on-line signature verification,” in ProceedingsICANN/ICONIP 2003, June 2003.

[297] B. Yanikoglu and A. Kholmatov, “Combining multiple biometrics to pro-tect privacy,” in Proceedings of ICPR-BCTP Workshop, 2004.

[298] D.-Y. Yeung, H. Chang, Y. Xiong, S. George, R. Kashi, T. Matsumoto,and G. Rigoll, “SVC2004: First international signature verification compe-tition,” in Proceedings 2004 Biometric Authentication: First InternationalConference, (ICBA 2004), Hong Kong, China, July 2004, pp. 16–22.

[299] H. Yoon, J. Lee, and H. Yang, “An on-line signature verification systemusing Hidden Markov Model in polar space,” in Proceedings Eigth Inter-national Workshop on Frontiers in Handwriting Recognition, Aug. 2002,pp. 329–333.

155

Page 161: Multimodal Biometrics for Identity Documents (MBioID) · claim of simplicity of biometric technology applied to identity documents. We believe that the deployment of such technology

MBioID State-of-the-Art Research Report

[300] K. Yu, J. Mason, and J. Oglesby, “Speaker recognition using hiddenMarkov models, dynamic time warping and vector quantisation,” IEEProceedings - Vision, Image and Signal Processing, vol. 142, pp. 313–318,October 1995.

[301] W. Zhao, R. Chellappa, J. Phillips, and A. Rosenfeld, “Face recognition:A literature survey,”ACM Computing Surveys, vol. 35, no. 4, pp. 399–458,December 2003.

156