WILEY SERIES IN PROBABILITY AND STATISTICS

PROBABILITY AND STATISTICS SECTION

Established by WALTER A. SHEWHART and SAMUEL S. WILKS

Editors: Vic Barnett, Noel A. C. Cressie, Nicholas I. Fisher,
Iain M. Johnstone, J. B. Kadane, David G. Kendall, David W. Scott,
Bernard W. Silverman, Adrian F, M. Smith, JozefL. Teugels;
Ralph A. Bradley, Emeritus, J. Stuart Hunter, Emeritus

A complete list of the titles in this series appears at the end of this volume.

Copyright © 1999 by John Wiley & Sons, Inc.

All rights reserved. Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212) 850-6008, E-Mail: PERMREQ @ WILEY.COM.

For ordering and customer service, call 1-800-CALL-WILEY.

Library of Congress Cataloging In Publication Data:

Billingsley, Patrick.

  Convergence of probability measures / Patrick Billingsley. — 2nd
ed.

  p.cm. — (Wiley series in probability and statistics. Probability and statistics)

  “Wiley-Interscience publication.” Includes bibliographical references and indexes.

  ISBN 0-471-19745-9 (alk. paper)

  1. Probability measures. 2. Metric spaces. 3. Convergence.

I. Title. II. Series: Wiley series in probability and statistics.

Probability and statistics.

QA273.6.B55  1999

519.2—dc21     99-30372

CIP

PREFACE

From the preface to the first edition.Asymptotic distribution theorems in probability and statistics have from the beginning depended on the classical theory of weak convergence of distribution functions in Euclidean space—convergence, that is, at continuity points of the limit function. The past several decades have seen the creation and extensive application of a more inclusive theory of weak convergence of probability measures on metric spaces. There are many asymptotic results that can be formulated within the classical theory but require for their proofs this more general theory, which thus does not merely study itself. This book is about weak-convergence methods in metric spaces, with applications sufficient to show their power and utility.

The second edition.A person who read the first edition of this book when it appeared thirty years ago could move directly on to the periodical literature and to research in the subject. Although the book no longer takes the reader to the current boundary of what is known in this area of probability theory, I think it is still useful as a textbook one can study before tackling the industrial-strength treatises now available. For the second edition I have reworked most of the sections, clarifying some and shortening others (most notably the ones on dependent random variables) by using discoveries of the last thirty years, and I have added some new topics. I have written with students in mind; for example, instead of going directly to the space D[0, ∞), I have moved, in what I hope are easy stages, from C[0, 1] to D[0, 1] to D[0,∞). In an earlier book of mine, I said that I had tried to follow the excellent example of Hardy and Wright, who wrote their Introduction to the Theory of Numbers with the avowed aim, as they say in the preface, of producing an interesting book, and I have again taken them as my model.

Chicago,
January 1999

Patrick Billingsley

For mathematical information and advice, I thank Richard Arratia, Peter Donnelly, Walter Philipp, Simon Tavaré, and Michael Wichura. For essential help on Teχ and figures, I thank Marty Billingsley and Mitzi Nakatsuka.

PB

INTRODUCTION

The De Moivre-Laplace limit theorem says that, if

(1)

is the distribution function of the normalized number of successes in n Bernoulli trials, and if

(2)

is the standard normal distibution function, then

(3)

for all x (n → ∞, the probability p of success fixed).

We say of arbitrary distribution functions Fn and F on the line that Fn converges weakly to F, which we indicate by writing Fn ⇒ F, if (3) holds at every continuity point x of F. Thus the De Moivre-Laplace theorem says that (1) converges weakly to (2); since (2) is everywhere continuous, the proviso about continuity points is vacuous in this case. If Fn and F are defined by

(4)

(I for the indicator function) and

(5)

then again Fn ⇒ F, and this time the proviso does come into play: (3) fails at x = 0.

For a better understanding of this notion of weak convergence, which underlies a large class of limit theorems in probability, consider the probability measures Pn and P generated by arbitrary distribution functions Fn and F. These probability measures, defined on the class of Borel subsets of the line, are uniquely determined by the requirements

Since F is continuous at x if and only if the set {x} consisting of x alone has P-measure 0, Fn ⇒ F means that the implication

(6)

holds for each x.

Let ∂A denote the boundary of a subset A of the line; ∂A consists of those points that are limits of sequences of points in A and are also limits of sequences of points outside A. Since the boundary of (∞, x] consists of the single point x, (6) is the same thing as

(7)

where we have written A for (– ∞, x]. The fact of the matter is that Fn ⇒ F holds if and only if the implication (7) is true for every Borel set A—a result proved in Chapter 1.

Let us distinguish by the term P-continuity set those Borel sets A for which P(∂A) = 0, and let us say that Pn converges weakly to P, and write Pn ⇒ P, if Pn(A)→ P(A) for each P-continuity set A—that is, if (7) holds. As just asserted, Pn ⇒ P if and only if the corresponding distribution functions satisy Fn ⇒ F.

This reformulation clarifies the reason why we allow (3) to fail if F has a jump at x. Without this exemption, (4) would not converge weakly to (5), but this example may appear artificial. If we turn our attention to probability measures Pn and P, however, we see that Pn(A) → P(A) may fail if P(∂A) > 0 even in the De Moivre-Laplace theorem. The measures Pn and P generated by (1) and (2) satisfy

(8)

and

(9)

for Borel sets A. Now if A consists of the countably many points

then Pn(A) = 1 for all n and P(A) = 0, so that P(An) → P(A) is impossible. Since ∂A is the entire real line, this does not violate (7).

Although the concept of weak convergence of distribution functions is tied to the real line (or to Euclidean space, at any rate), the concept of weak convergence of probability measures can be formulated for the general metric space, which is the real reason for preferring the latter concept. Let S be an arbitrary metric space, let S be the class of Borel sets (S is the σ-field generated by the open sets), and consider probability measures Pn and P defined on S. Exactly as before, we define weak convergence Pn ⇒ P by requiring the implication (7) to hold for all Borel sets A. In Chapter 1 we investigate the general theory of this concept and see what it reduces to in various special cases. We prove there, for example, that Pn converges weakly to P if and only if

(10)

holds for all bounded, continuous real-valued functions on S. (In order to conform with general mathematical usage, we take (10) as the defintion of weak convergence, so that (7) becomes a necessary and sufficient condition instead of a definition.)

Chapter 2 concerns weak convergence on the space C = C[0, 1] with the uniform topology; C is the space of all continuous real functions on the closed unit interval [0, 1], metrized by taking the distance between two functions x = x(t) and y — y(t) to be

(11)

An example of the sort of application made in Chapter 2 will show why it is both interesting and useful to develop a general theory of weak convergence—one that goes beyond the Euclidean case. Let ξ₁, ξ₂,… be a sequence of independent, identically distributed random variables defined on some probability space . If the ξn have mean 0 and variance σ², then, by the Lindeberg-Lévy central limit theorem, the distribution of the normalized sum

(12)

converges weakly, as n tends to infinity, to the normal distribution defined by (9).

We can formulate a refinement of the central limit theorem by proving weak convergence of the distributions of certain random functions constructed from the partial sums Sn. For each integer n and each sample point ω, construct on the unit interval the polygonal function that is linear on each of the subintervals [(i – l)/n, i/n], i = 1, 2,…, n, and has the value at the point i/n (S₀(ω) = 0). In other words, construct the function Xn(ω) whose value at a point t of [0, 1] is

(13)

For each ω, Xn(ω) is an element of the space C, a random function. Let Pn be the distribution of Xn(ω) on C, denned for Borel subsets A of C—Borel sets relative to the metric (11)—by

(the definition is possible because the mapping ω → Xn(ω) turns out to be measurable in the right way). In Chapter 2 we prove Donsker’s theorem, which says that

(14)

where W is Wiener measure. We also prove the existence in C of Wiener measure, which describes the probability distribution of the path traced out by a particle in Brownian motion.

If A = [x: x(1) ≤ α], then, since the value of the function Xn(ω) at

It turns out that W(∂A) = 0, so that (14) implies

It also turns out that

so that (14) does contain the Lindeberg-Lévy theorem.

If ξi takes the values +1 and –1 with probability each, Sn can be interpreted as the position at time n in a symmetric random walk. The central limit theorem says that this position, normalized by (σ = 1), is, for n large, approximately distributed as the position at time t = 1 of a particle in Brownian motion. The relation (14) says that the entire path of the random walk during the first n steps is, for n large, distributed approximately as the path up to time t= 1 of a particle in Brownian motion.

To see in a concrete way that (14) contains information going beyond the central limit theorem, consider the set

Again it turns out that W(∂A) = 0, so that (14) implies

(15)

We can evaluate the rightmost member of (15) by finding the limit on the left for the special case of symmetric random walk, which is easy to analyze. And then we have a limit theorem for the distribution of maxk≤n Sk under the hypothesis of the Lindeberg-Lévy theorem.

For another example involving Xn(ω), take A to be the set of x in C for which the set [t: x(t) > 0] has Lebesgue measure at most α (where 0 ≤ α ≤ 1). As before, Pn(A) → P(A). Since the Lebesgue measure of is essentially the fraction of the partial sums S₁, S₂,…, Sn that exceed 0, this argument leads to an arc sine law under the hypotheses of the Lindeberg-Lévy theorem. Chapter 2 contains the details of these derivations.

We can in this way use the theory of weak convergence in C to obtain a whole class of limit theorems for functions of the partial sums S₁, S₂,…, Sn. The fact that Wiener measure W is the weak limit of the distribution over C of the random function Xn(ω) can also be used to prove theorems about W, and W is interesting in its own right.

Chapter 3 specializes the theory of weak convergence to another space of functions on [0, 1]—the space D[0, 1] of functions having only discontinuities of the first kind (jump discontinuities). This is the natural space in which to analyze the Poisson process and other processes with paths that are necessarily discontinuous. We also study spaces of discontinuous functions on [0, ∞) and on certain other sets that play the role of a generalized “time”—for example, the set of convex subsets of the unit square. Chapter 4 concerns weak convergence of the distributions of random functions derived from various sequences of dependent random variables. Chapter 5 has to do with other asymptotic properties of random functions; there we prove Strassens’s theorem, a far-reaching generalization of the law of the iterated logarithm.

Many of the conclusions in Chapters 2 through 5, although not requiring function-space concepts for their statement, could hardly have been derived without function-space methods. Standard measure-theoretic probability and metric-space topology are used from the beginning of the book. Although the point of view throughout is that of functional analysis (a function is a point in a space), nothing of functional analysis is assumed (beyond an initial willingness to view a function as a point in a space). All function-analytic results needed are proved in the text or else in Appendix M at the end of the book. This appendix also gathers together for easy reference some results in metric-space topology, analysis, and probability.

CHAPTER 1

WEAK CONVERGENCE IN METRIC SPACES

SECTION 1. MEASURES ON METRIC SPACES

We begin by studying probability measures on the general metric space. Denote the space by S, and let S be the Borel σ-field, the one generated by the open sets;† its elements are the Borel sets. A probability measure on S is a nonnegative, countably additive set function P satisfying PS = 1.

If probability measures Pn and P satisfy‡ Pnf → Pf for every bounded, continuous real function f on S, we say that Pn converges weakly to P and write Pn ⇒ P. In Chapter 1 we study the basic, general theory of weak convergence, together with the associated concept of convergence in distribution. We first derive some properties of individual measures on (S, S). Although S is sometimes assumed separable or complete, most of the theorems in this chapter hold for all metric spaces.

Measures and Integrals

Theorem 1.1. Every probability measure P on (S, S) is regular, that is, for every S-set A and every there exist a closed set F and an open set G such that F ⊂ A ⊂ G and .

PROOF. Denote the metric on S by ρ(x, y) and the distance from x to A by ρ(x, A) [M1].§ If A is closed, we can take F = A and G = Aδ = [x: ρ(x, A) < δ] for some δ, since the latter sets decrease to A as δ ↓ 0. Hence we need only show that the class of S-sets with the asserted property is a σ-field. Given sets An in , choose closed sets Fn and open sets Gn such that Fn ⊂ An ⊂ Gn and . If , and if , with n₀ so chosen that , then and . Thus is closed under the formation of countable unions; since it is obviously closed under complementation, is a σ-field.

Theorem 1.1 implies that P is completely determined by the values of PF for closed sets F. The next theorem shows that P is also determined by the values of Pf for bounded, continuous f. The proof depends on approximating the indicator IF by such an f, and the function works. It is bounded, and it is continuous, even uniformly continuous, because . And x ∈ F implies f(x) = 1, while implies and hence f (x) = 0. Therefore,

(1.1)

Theorem 1.2. Probability measures P and Q on S coincide if Pf = Qf for all bounded, uniformly continuous real functions f.

PROOF. For the bounded, uniformly continuous f of (1.1), . Letting gives PF ≤ QF, provided F is closed. By symmetry and Theorem 1.1, P = Q.

Because of theorems like this, it is possible to work with measures PA or with integrals Pf, whichever is simpler or more natural. We defined weak convergence in terms of the convergence of integrals of functions, and in the next section we characterize it in terms of the convergence of measures of sets.

Tightness

The following notion of tightness plays a fundamental role both in the theory of weak convergence and in its applications. A probability measure P on (S,S) is tight if for each there exists a compact set K such that . By Theorem 1.1, P is tight if and only if PA is, for each A in S, the supremum of PK over the compact subsets K of A.

Theorem 1.3. If S is separable and complete, then each probability measure on (S,S) is tight.

PROOF. Since S is separable, there is, for each k, a sequence Ak₁, Ak₂, … of open 1/K-balls covering S. Choose nk large enough that . By the completeness hypothesis, the totally bounded set has compact closure K. But clearly .

Some Examples

Here are four metric spaces, together with some facts about them we need further on. Define a subclass of S to be a separating class if two probability measures that agree on necessarily agree also on the whole of S: The values of PA for A in are enough to separate P from all the other probability measures on S. For example, by Theorem 1.1 the closed sets form a separating class. Recall that a class is a π-system if it is closed under the formation of finite intersections and that is a separating class if it is a π-system generating the σ-field S [PM.42].†

Example 1.1. Let Rk denote k-dimensional Euclidean space with the ordinary metric , and denote by Rk the corresponding class of Borel sets—the k-dimensional Borel sets. The distribution function corresponding to a probability measure P on Rk is

(1.2)

Since the sets on the right here form a π-system that generates Rk, they constitute a separating class. Therefore, F completely determines P.

By Theorem 1.3, each probability measure on (Rk,Rk) is tight. But tightness is in this case obvious because the space is σ-compact—is a countable union of compact sets (B(0, n)⁻ ↑ Rk, for example).

Example 1.2. Let R∞ be the space of sequences x = (x₁, x₂,…) of real numbers—the product of countably many copies of R¹. If b(α, β) = 1 ^ |α − β|, then is a metric on equivalent to the usual one, and under it, is complete as well as separable [M4]. Metrize Obviously, () → 0 implies for each ; but by the -test [PM.543], the reverse implication also holds. Under , therefore, has the topology of pointwise convergence: → if and only if for each .

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!

Lesen Sie weiter in der vollständigen Ausgabe!