WHICH LINEAR-FRACTIONAL COMPOSITION OPERATORS ARE ESSENTIALLY NORMAL?

PAUL S. BOURDON, DAVID LEVI, SIVARAM K. NARAYAN, AND JOEL H. SHAPIRO

This paper is dedicated to Professor Lawrence G. Brown, on the occasion of his sixtieth birthday.

Abstract. We characterize the essentially normal composition operators induced on the Hardy space H^{2} by linear fractional maps; they are either compact, normal, or (the nontrivial case) induced by parabolic non-automorphisms. These parabolic maps induce the first known examples of nontrivially essentially normal composition operators. In addition we characterize those linearfractionally induced composition operators on H^{2} that are essentially selfadjoint, and present a number of results for composition operators induced by maps that are not linear fractional.

1. Introduction

We work with holomorphic functions φ that take the unit disc 𝕌 into itself. Each such holomorphic selfmap of 𝕌 induces a linear composition operator C_{φ} on the space Hol (𝕌) of all functions holomorphic on 𝕌 as follows:

C_{φ}f = f ∘ φ (f ∈ Hol (𝕌)).

A classical result of Littlewood [11] asserts that every composition operator restricts to a bounded operator on the Hardy space H^{2} (see also the first chapter of either [9] or [18]). Littlewood’s theorem has, within the last several decades, sparked a lively interaction between function theory and operator theory in which one tries to understand how properties of composition operators relate to the behavior of their inducing maps. Much of this story is told in [7] and [18], and some further developments are surveyed in the conference proceedings [10].

Our work has its roots in the following result, proved over thirty years ago by Howard Schwartz [15]:

A composition operator on H^{2} is normal if and only if it is induced by a dilation z → az for some |a| ≤ 1.

2 PAUL S. BOURDON, DAVID LEVI, SIVARAM K. NARAYAN, AND JOEL H. SHAPIRO

Recall that an operator T on a Hilbert space is called normal if T^{∗}T = T T^{∗}, and essentially normal if T^{∗}T − T T^{∗} is compact. Since compact and normal operators are clearly essentially normal, let us agree to call an operator nontrivially essentially normal if it is essentially normal, but neither normal nor compact. Just recently Nina Zorboska [20] showed that, among the conformal automorphisms of the unit disc, the rotations (which induce unitary composition operators) are the only ones that induce essentially normal composition operators on H^{2}. Zorboska showed further that the composition operators induced on H^{2} by linear-fractional maps fixing no point on the unit circle also fail to be nontrivially essentially normal, and asked if it were possible for any composition operator on H^{2} to be nontrivially essentially normal. The following result answers Zorboska’s question, showing that among the non-automorphic linear-fractional selfmaps of 𝕌 that fix a point of the boundary (a condition that renders their induced composition operators non-compact—see §2.6—and, by Schwartz’s theorem, non-normal), the parabolics induce composition operators that are essentially normal on H^{2}, while the hyperbolics induce operators that are not.

Date: September 7, 2002.

Key words and phrases. Essentially normal, composition operator, linear-fractional map.

Research supported in part by National Science Foundation grants DMS 0100290 and DMS 0100502. The work of the last-named author was carried out at the Department of Mathematics, University of Hawaii where he was a visiting professor during the academic year 2001-02.

Main Theorem. A composition operator induced on H^{2} by a linear-fractional selfmap of the unit disc is nontrivially essentially normal if and only if it is induced by a parabolic non-automorphism.

The proof of this result occupies most of what follows; here is an outline. In the next section we set out some prerequisites on Hardy spaces and on selfmaps of 𝕌—in particular linear-fractional maps. Serious work begins in §3 where we discuss a formula due to Carl Cowen [5] for the adjoint of a linear-fractionally induced composition operator, and use this formula to represent the commutator of such an operator with its adjoint. We show that for linear fractional maps of the disc that fix a point of the boundary, but are not automorphisms, essential normality of the induced composition operator reduces to studying compactness for the commutator of that operator with a closely related composition operator provided by Cowen’s formula. In the parabolic case essential normality follows from the fact that the latter commutator turns out to be zero! In the hyperbolic case the individual terms of this commutator turn out to be composition operators induced by commuting parabolic maps. An analysis of the eigenvalues of those terms reveals that the commutator itself is not compact, so the original composition operator is not essentially normal. This material occupies most of Sections 3–5 and, along with Zorboska’s results, completes the proof of our Main Theorem. In §4, which treats the parabolic case, we also characterize the linear-fractional self-maps of 𝕌 that induce nontrivial essentially self-adjoint composition operators on H^{2}.

In order to make our paper reasonably self-contained, we devote §6 to providing new proofs of Zorboska’s results—as they apply to linear-fractional composition operators; our arguments require less background in operator theory than the original ones. Then, in §7, we move our work beyond the linear-fractional setting, extending our Main Theorem to a class of composition operators that one might describe as “essentially linear-fractional.”

We close with a section that contains a discussion of alternate methods of proving some of our results as well as of a natural question raised by our work. In this final section, we also discuss how the Brown-Douglas-Fillmore Theorem [3] shows that every essentially normal composition operator induced by a linear-fractional mapping has the form “Normal + Compact.” In this regard, note that any operator that is a compact perturbation of a normal operator is clearly essentially normal, but not every essentially normal operator has this form. Consider, for example, the forward shift S on ℓ^{2}, which is essentially normal because its self-commutator S^{∗}S − SS^{∗} has rank one, but is Fredholm of index −1, and therefore cannot be written in the form “normal + compact” since whenever such a sum is Fredholm it must have index 0.

Linear-fractional self-maps of 𝕌 are important in the study of composition operators for two reasons: First, they induce a tractable, yet nontrivial class of operators, and second, they serve as “models” for the most general holomorphic selfmaps of 𝕌. We say more about this latter phenomenon in §8.

2. Prerequisites

Here we collect the fundamental facts about Hardy spaces and linear-fractional maps required for what is to follow. First some notation: In addition to using Hol (𝕌) to denote the space of all holomorphic functions on 𝕌, we write H^{∞} for the space of bounded holomorphic functions on 𝕌, and denote its natural norm by ∥ · ∥_{∞}, i.e.

∥f∥_{∞} := sup_{|z|<1} |f(z)| (f ∈ H^{∞}).

We will also use ∥ · ∥_{∞} to denote the norm in L^{∞} = L^{∞}(∂𝕌), where ∂𝕌 is equipped with Lebesgue arclength measure.

2.1. The space H^{2}. The material of this paragraph occurs in many places; see for example [9, Chapters 1 and 2], or [14, Chapter 17]. For f ∈ Hol (𝕌) we denote by f̂(n) the n-th coefficient of f in its MacLaurin series. The Hardy space H^{2} is the collection of all such functions f for which

∥f∥^{2} = ∑_{n=0}^{∞} |f̂(n)|^{2} < ∞.

The formula above defines a norm that turns H^{2} into a Hilbert space whose inner product is given by

⟨f, g⟩ = ∑_{n=0}^{∞} f̂(n)\overline{ĝ(n)} (f, g ∈ H^{2}).

According to the Riesz-Fisher theorem, if f ∈ H^{2} then the trigonometric series ∑_{0}^{∞} f̂(n)e^{inθ} is the Fourier series of some function f^{∗} ∈ L^{2} = L^{2}(∂𝕌, m) where, here (and henceforth) m denotes arclength measure on ∂𝕌, normalized to have total mass one. The map f → f^{∗} takes H^{2} isometrically onto the closed subspace of L^{2} consisting of functions f with Fourier transform f̂ supported on the non-negative integers. The boundary function f^{∗} turns out to be just the nontangential limit function

f^{∗}(ζ) = ∠ lim_{z→ζ} f(z),

which is known to exist for (m-) almost every point ζ ∈ ∂𝕌. We simplify notation by writing f(ζ) instead of f^{∗}(ζ) for each ζ ∈ ∂𝕌 at which this nontangential limit exists, relying on the context to determine what we mean by the symbol f. With this identification the norm and inner product in H^{2} can be computed on the boundary of the unit disc as:

∥f∥^{2} = ∫_{∂𝕌} |f|^{2} dm and ⟨f, g⟩ = ∫_{∂𝕌} f\overline{g} dm (f, g ∈ H^{2}).

2.2. Angular Derivatives. The behavior of the composition operator C_{φ} : H^{2} → H^{2} is greatly influenced by the angular derivative of φ. For example, if C_{φ} is compact on H^{2}, then φ has finite angular derivatives at no point of the unit circle (see [18, §3.5], for example). Recall that the analytic selfmap φ of 𝕌 has (finite) angular derivative at ζ ∈ ∂𝕌 provided that the nontangential limit φ(ζ) exists, has modulus 1, and

φ′(ζ) := ∠ lim_{z→ζ} \frac{φ(z) − φ(ζ)}{z − ζ} (1)

exists and is finite. By the Julia-Carathéodory Theorem (see [7, Theorem 2.44] or [18, Chapter 4], for example),

|φ′(ζ)| = lim inf_{z→ζ} \frac{1 − |φ(z)|}{1 − |z|} , (2)

where the lim inf is calculated as z approaches ζ unrestrictedly within 𝕌. This statement is to be interpreted in the strongest possible sense: φ has a finite angular derivative at ζ if and only if the right-hand side of (2) is finite. Otherwise the difference quotient in (2) converges uniformly to ∞ as z → ζ in 𝕌, in which case the limit on the right-hand side of (1) is ∞.

2.3. The Denjoy-Wolff Point. Each analytic selfmap φ of 𝕌 that is not an elliptic automorphism of 𝕌 (i.e., not conformally conjugate to a rotation about the origin) has associated with it a unique point ω in the closure of 𝕌 that acts like an attractive fixed point in that φ_{n}(z) → ω as n → ∞, where φ_{n} denotes φ composed with itself n times (φ_{0} being the identity function). The Denjoy-Wolff point of φ may also be characterized as that point ω ∈ \overline{𝕌} such that:

• if |ω| < 1, then φ(ω) = ω and |φ′(ω)| < 1;

• if |ω| = 1, then φ(ω) = ω and 0 < φ′(ω) ≤ 1, where if ω ∈ ∂U then φ(ω) denotes the angular limit of φ at ω, and φ′(ω) denotes the angular derivative of φ at ω. More information about Denjoy-Wolff points and angular derivatives can be found in [7, Chapter 2] or [18, Chapters 4 & 5].

2.4. Toeplitz operators. In this paragraph (up until the last sentence) we identify H^{2} with its space of nontangential limit functions, as described above in §2.1. For each b ∈ L^{∞} = L^{∞}(∂𝕌) the multiplication operator M_{b} : f → bf is a bounded linear operator on L^{2} with ∥M_{b}∥ = ∥b∥_{∞}. Closely related is the Toeplitz operator T_{b} defined on H^{2} by T_{b} = P M_{b}, where P denotes the orthogonal projection of L^{2} onto H^{2}:

P (∑_{−∞}^{∞} f̂(n)e^{inθ}) = ∑_{n=0}^{∞} f̂(n)e^{inθ}.

Clearly T_{b} is a bounded operator on H^{2} with ∥T_{b}∥ ≤ ∥b∥_{∞} (actually, T_{b} has norm equal to ∥b∥_{∞}, see [8, Corollary 7.8, page 179]). If b is the nontangential limit function of a bounded analytic function, also denoted b, then M_{b} takes H^{2} into itself, so the projection is superfluous and T_{b} is the restriction of M_{b} to H^{2}. In this case T_{b} can be identified with the operator of pointwise multiplication by the holomorphic function b, acting on H^{2}, now viewed as a space of functions holomorphic on 𝕌.

If b(z) ≡ z on either 𝕌 or ∂𝕌 then we write T_{z} instead of T_{b}. A routine adjoint computation shows that for each b ∈ L^{∞} we have (T_{b})^{∗} = T\overline{b}. In particular (T_{z})^{∗} = T\overline{z} is easily seen to be the backward shift on H^{2}:

T_{z}^{∗}(z^{n}) = \{ \begin{matrix}z^{n−1} & (n = 1, 2, …) \\ 0 & (n = 0) \end{matrix} (3)

2.5. Reproducing kernels. With each point p ∈ 𝕌 we associate the reproducing kernel

K_{p}(z) = \frac{1}{1 − \overline{p}z} = ∑_{n=0}^{∞} \overline{p}^{n}z^{n} (z ∈ 𝕌). (4)

Each kernel function K_{p} is holomorphic in a neighborhood of the closed unit disc, and so belongs to H^{2}. Moreover for each p ∈ 𝕌 and f ∈ H^{2} the definition of the H^{2}-inner product as a series yields immediately the reproducing property

⟨f, K_{p}⟩ = f(p) (f ∈ H^{2}, p ∈ 𝕌). (5)

Reproducing kernels are crucial to our work because of the following adjoint property

C_{φ}^{∗}K_{p} = K_{φ(p)} (p ∈ 𝕌); (6)

for the proof, just take the inner product of each side of the equation with an arbitrary f ∈ H^{2}, and use (5). There is a companion result, just as easily proven, for “analytic” Toeplitz operators:

T_{b}^{∗} K_{p} = \overline{b(p)} K_{p} (b ∈ H^{∞}, p ∈ 𝕌), (7)

so K_{p} is an eigenfunction for T_{b}^{∗}.

2.6. Linear-fractional selfmaps of 𝕌. We denote by LFT (𝕌) those linear fractional maps that take the open unit disc 𝕌 into itself. The automorphisms of 𝕌, denoted Aut (𝕌), are the maps in LFT (𝕌) that take 𝕌 onto itself. Maps in LFT (𝕌) take the unit disc univalently onto some sub-disc, and the induced composition operator is compact if and only if the closure of this subdisc lies inside 𝕌, i.e., if and only if ∥φ∥_{∞} < 1; see [18, Chapter 2] for example. (The complete story on the compactness problem for composition operators is actually much more interesting than this; see [16], [18, Chapters 3 and 10], or [7, §3.2] for the details.)

Here we are interested solely in non-compact operators, so we consider only maps φ ∈ LFT (𝕌) with ∥φ∥_{∞} = 1. According to the classification in [18, Chapter 0], there are only these possibilities:

(a) φ fixes a point ω ∈ ∂𝕌, whereupon there are two subcases:

(i) φ is parabolic: Here ω is the only fixed point φ possesses in the Riemann sphere ℂ̂. The linear-fractional map τ(z) := (1 + \overline{ω}z)/(1 − \overline{ω}z) takes the unit disc onto the right half-plane Π and sends ω to ∞. Thus Φ := τ ∘ φ ∘ τ^{−1} is a linear-fractional selfmap of Π which fixes only ∞, and so must be the mapping of translation by some number t, where necessarily Re t ≥ 0. Thus

φ(z) = τ^{−1} ∘ Φ ∘ τ(z) = τ^{−1}(τ(z) + t) (z ∈ ℂ); (8)

let us call t the translation number of either φ or Φ. Note that if t is pure imaginary then Φ is an automorphism of Π, and therefore φ ∈ Aut (𝕌). If, on the other hand Re t > 0 then φ ∉ Aut (𝕌). Note further that if two parabolic members of LFT (𝕌) have the same fixed point, then they are both conjugate, by the same map τ, to translations. Since these translations commute under composition, so do the original maps. More generally the same kind of argument shows that every pair of linear-fractional maps with the same fixed point set commutes under composition.

Now every parabolic map of 𝕌 is rotationally conjugate to one that fixes the point 1, and since rotations of the disc induce unitary composition operators on H^{2} it follows that every parabolically induced composition operator on that space is unitarily equivalent to one induced by a map that fixes 1. Thus, in what follows we can always assume ω = 1. For φ ∈ LFT (𝕌) parabolic with fixed point 1 and translation number t, the representation (8) can be rewritten explicitly as:

φ(z) = \frac{(2 − t)z + t}{−tz + (2 + t)} (z ∈ ℂ). (9)

Example: φ(z) = (2 − z)^{−1} is a parabolic non-automorphism of 𝕌 with fixed point 1 and translation number t = 2.

(ii) φ is hyperbolic: In this case φ has an additional fixed point in ℂ̂. By letting τ be a linear fractional map that takes this additional fixed point to zero and the boundary fixed point to ∞ we see that Φ := τ ∘ φ ∘ τ^{−1} is a dilation; Φ(w) = rw for some complex number r. Moreover τ takes 𝕌 to a half-plane on which Φ is a self-map, and this forces r > 0. If both fixed points lie on ∂𝕌 then τ(𝕌) is bounded by a line through the origin, and φ ∈ Aut (𝕌). Examples: φ_{ρ}(z) = (ρ + z)/(1 + ρz) for 0 < ρ < 1 (fixed points ±1).

The other possibility is that one fixed point lies on ∂𝕌 and the other does not, in which case φ ∉ Aut (𝕌). If the fixed point not on ∂𝕌 lies in the exterior of the unit circle, then the one on the boundary is the attractive fixed point. Because τ takes this fixed point to ∞ we see that r > 1 and τ(𝕌) must be a half-plane whose closure does not contain the origin. Example: φ(z) = (1 + z)/2 (r = 2, fixed points 1 and ∞). If, on the other hand, the non-boundary fixed point lies in 𝕌 then it is the attractive one. Since τ takes this point to the origin, we see that r < 1 and the half-plane τ(𝕌) contains the origin. Example: φ(z) = z/(2 − z) (r = 1/2, fixed points 0 and 1).

(b) No fixed point on ∂𝕌. In this case the attractive fixed point lies in 𝕌, and the repulsive one in ℂ̂\\overline{𝕌}, and it is a simple exercise to check that ∥φ ∘ φ∥_{∞} < 1. Example: φ(z) = (1 − z)/2 (fixed points 1/3 and ∞).

2.7. Eigenvalues and eigenvectors. We return to §2.6(a)(i); let φ ∈ LFT (𝕌) be parabolic with fixed point at 1, so that φ is represented by (8), i.e., conjugate via τ(z) = (1 + z)/(1 − z) to a translation mapping Φ(w) = w + t of the right half-plane Π, where t is a complex number with non-negative real part. For λ > 0 let F_{λ}(w) := e^{−λw} for w ∈ Π. Then F_{λ} is a bounded holomorphic function on Π, and F_{λ} ∘ Φ = e^{−λt}F_{λ}. In other words, F_{λ} is an eigenfunction for the operator C_{Φ} acting on Hol (Π), and the corresponding eigenvalue is e^{−λt}. Taking all this back to the disc via τ^{−1} we see that f_{λ} := F_{λ} ∘ τ is an analytic function that is bounded on 𝕌 (in fact it is a singular inner function), hence in H^{2}, and an eigenvector for C_{φ}, which is now viewed as acting on H^{2}. The corresponding C_{φ}-eigenvalue for f_{λ} is still e^{−λt}, hence the spectrum of C_{φ} contains the curve Γ_{t} := {e^{−λt} : λ ≥ 0}.

This is all the spectral information we need to prove the Main Theorem (stated in the Introduction) and to characterize those linear-fractional composition operators that are essentially self–adjoint. For a deeper discussion of such matters we refer the reader to [6] or [7, Theorem 7.41, page 301], where it is shown, for example, that for a parabolic φ ∈ LFT (𝕌) with translation number t the set Γ_{t} ∪ {0} is the spectrum of C_{φ}.

3. Adjoints and commutators

In this section we use Cowen’s representation of the adjoint of a linear-fractionally induced composition operator to reduce the essential normality problem for nonautomorphisms φ ∈ LFT (𝕌) to one of determining the compactness of the commutator of C_{φ} with a related composition operator.

3.1. Cowen’s adjoint formula. In [5] (see also [7, Theorem 9.2, page 322]) Carl Cowen showed that if φ ∈ LFT (𝕌) is given by

φ(z) = \frac{az + b}{cz + d} (ad − bc ≠ 0), (10)

then

(11)  σ(z) := \frac{\overline{a}z − \overline{c}}{−\overline{b}z + \overline{d}} ∈ LFT (𝕌),
(12)  g(z) := \frac{1}{−\overline{b}z + \overline{d}} ∈ H^{∞},

and

C_{φ}^{∗} = T_{g}C_{σ}T_{h}^{∗} ,

where h(z) := cz + d.

The proof is an algebraic manipulation based on the adjoint property (6). The fact that σ maps 𝕌 into itself comes from the easily-checked representation

σ = ρ ∘ φ^{−1} ∘ ρ, where ρ(z) = 1/\overline{z}

(i.e., ρ is the mapping of inversion in the unit circle), and the inverse refers to φ, viewed as a univalent mapping of the Riemann Sphere onto itself. It also follows from this formula that:

(a) the fixed points of σ are the ρ-images of the fixed points of φ; in particular, ρ and σ have the same boundary fixed points, and

(b) σ is an automorphism if and only if φ is, in which case σ = φ^{−1}. Of these statements, perhaps only the last part of (b) requires explanation. If φ ∈ Aut (𝕌) then σ = φ^{−1} on ∂𝕌. Since both σ and φ^{−1} are holomorphic on a neighborhood of \overline{𝕌}, it follows that σ = φ^{−1} on all of 𝕌.

Consider now g and h. The boundedness of g comes from the fact that the linear polynomial that is its denominator has its zero at \overline{d}/\overline{b} = 1/\overline{φ(0)}, which lies outside the closed unit disc. Clearly 1/g ∈ H^{∞}, so in Cowen’s formula the Toeplitz operator T_{g}, which is just the operator on H^{2} of pointwise multiplication by g, is actually invertible on H^{2}. There is, of course, no question that h ∈ H^{∞}, but note also that it takes the value zero only at the point −d/c = 1/\overline{σ(0)}, which also lies outside the closed unit disc. Thus 1/h ∈ H^{∞}, so also T_{h}, and therefore T_{h}^{∗}, is invertible on H^{2}.

For bounded operators A and B on a Hilbert space, we use the notation

[A, B] := AB − BA

for the commutator of A and B; in particular A is essentially normal if and only if [A^{∗}, A] is compact. From Cowen’s formula we derive the following useful representation of [C_{φ}^{∗}, C_{φ}] for φ a linear-fractional selfmap of 𝕌.

3.2. The Commutator Formula. Suppose φ ∈ LFT (𝕌) is given by (10), and σ, g, and h are as in the statement of Cowen’s formula. Then

[C_{φ}^{∗}, C_{φ}] = T_{g} [C_{σ}, C_{φ}] T_{h}^{∗} + T_{g} C_{σ} [T_{h}^{∗}, C_{φ}] + (T_{g} − T_{g∘φ}) C_{σ∘φ} T_{h}^{∗} .

Proof. Using Cowen’s formula and the fact that C_{φ}T_{g} = T_{g∘φ}C_{φ}, we obtain

C_{φ}C_{φ}^{∗} = T_{g∘φ} C_{φ}C_{σ} T_{h}^{∗} = T_{g}C_{φ}C_{σ}T_{h}^{∗} + (T_{g∘φ} − T_{g})C_{φ}C_{σ}T_{h}^{∗} .

In the other direction, Cowen’s formula along with some algebraic manipulation yields

C_{φ}^{∗}C_{φ} = T_{g}C_{σ}C_{φ}T_{h}^{∗} + T_{g}C_{σ}[T_{h}^{∗}, C_{φ}]

so the desired formula now follows upon subtracting the former equation from the latter.

Crucial to our work on non-automorphisms is the following lemma, which reduces the question of essential normality for the composition operators they induce to that of compactness of a commutator of two composition operators.

3.3. Lemma. Suppose b ∈ C(∂𝕌) with b(1) = 0 and suppose further that the function θ → b(e^{iθ}) is differentiable at θ = 0. Then for every non-automorphic φ ∈ LFT (𝕌) with fixed point at 1, the operator T_{b}C_{φ} is compact on H^{2}.

Proof. Let A = T_{b}C_{φ}. We will prove a result stronger than originally advertised: A is a Hilbert-Schmidt operator. Because the sequence of monomials {z^{n}}_{0}^{∞} forms an orthonormal basis for H^{2} it is enough to show that

∥A∥2_{HS} := ∑_{n=0}^{∞} ∥Az^{n}∥^{2} < ∞.

Now computing norms on ∂𝕌:

∥Az^{n}∥_{2}^{2} = ∥P (b φ^{n})∥_{2}^{2} ≤ ∥b φ^{n}∥_{2}^{2}
= \frac{1}{2π} ∫_{−π}^{π} |b(e^{iθ})|^{2} |φ(e^{iθ})|^{2n} dθ

≤ const. ∫_{−π}^{π} θ^{2} |φ(e^{iθ})|^{2n} dθ,

where in the last line we used the differentiability of b(e^{iθ}) at θ = 0 and the fact that b(1) = 0. Upon summing both sides of the resulting inequality, interchanging sum and integral and using the Geometric Series Theorem, we obtain:

∥A∥2_{HS} ≤ const. ∫_{−π}^{π} \frac{θ^{2}}{1 − |φ(e^{iθ})|^{2}} dθ . (13)

Since φ is a non-automorphic linear-fractional selfmap of 𝕌 with a fixed point at 1, it takes ∂𝕌 to a circle tangent to ∂𝕌 at 1, but otherwise lying in 𝕌. If this circle’s radius is denoted by r, then its complex equation is 1 − |z|^{2} = C_{1}|1 − z|^{2}, where C_{1} = (1 − r)/r. Thus for each θ ∈ [−π, π]:

1 − |φ(e^{iθ})|^{2} = C_{1}|1 − φ(e^{iθ})|^{2}
≥ C_{2}|1 − e^{iθ}|^{2}  (since φ(1) = 1)
≥ C_{3} θ^{2} ,

where none of the constants C_{j} (j = 1, 2, 3) depend on θ. Thus the integrand on the right-hand side of (13) is bounded by 1/C_{3}, so the integral is finite, establishing that on H^{2} the operator A = T_{b}C_{φ} is Hilbert-Schmidt, hence compact.

3.4. Proposition. Suppose φ ∈ LFT (𝕌) is not an automorphism, but has a fixed point ω ∈ ∂𝕌. Then C_{φ} is essentially normal if and only if [C_{σ}, C_{φ}] is compact.

Proof. It is enough to show that the last two terms on the right-hand side of the Commutator Formula are compact operators. The last term has the form T_{b}C_{ψ}T_{h}^{∗}, where ψ = σ ∘ φ is a non-automorphic linear-fractional selfmap of 𝕌 that fixes the point ω, and b = g − g ∘ φ is holomorphic on a neighborhood of the closed unit disc and vanishes at ω. Thus T_{b}C_{ψ} satisfies the hypotheses of Lemma 3.3, and so is a Hilbert-Schmidt operator on H^{2}. The same is therefore true of T_{b}C_{ψ}T_{h}^{∗}.

As for the next-to-last term, observe that since h(z) = cz +d we have [T_{h}^{∗}, C_{φ}] = \overline{c}[T_{z}^{∗}, C_{φ}], so it is enough to show that [T_{z}^{∗}, C_{φ}] is compact. Because of (3) (the fact that T_{z}^{∗} is the backward shift on H^{2}) we see after a little computation that if b is defined on the unit circle by b(z) = \overline{z}φ(z) − 1, then [T_{z}^{∗}, C_{φ}] coincides with B := T_{b}C_{φ}T\overline{z} on each monomial z^{n} for each n ≥ 0 (for n = 0: both operators take the constant function 1 to 0), and so the operators coincide on all of H^{2}. Now b(ω) = 0 because φ fixes ω, hence b and φ satisfy the hypotheses of Lemma 3.3, so B = [T_{z}^{∗}, C_{φ}] is compact on H^{2}.

4. Parabolic Non-Automorphisms

In this section we use Proposition 3.4 to show that parabolic non-automorphisms induce composition operators on H^{2} that are nontrivially essentially normal, and to characterize those linear-fractional selfmaps of 𝕌 that induce nontrivially essentially self-adjoint composition operators.

4.1. Theorem. If φ ∈ LFT (𝕌) is a parabolic non-automorphism, then C_{φ} is essentially normal.

Proof. Recall that φ, being parabolic, has just one fixed point in the Riemann Sphere, and in order for φ to map 𝕌 into itself, this fixed point must lie on the unit circle, say at ω. According to the Proposition 3.4 we need only show that [C_{σ}, C_{φ}] is compact. Because σ has the same fixed point set as φ (see §3.1), it too is parabolic, with fixed point at ω. As we noted in §2.6(a)(i), this implies that σ ∘ φ = φ ∘ σ, whereupon

[C_{σ}, C_{φ}] := C_{φ}C_{σ} − C_{σ}C_{φ} = C_{σ∘φ} − C_{φ∘σ} = 0,

which, in view of Proposition 3.4, completes the proof.

To complete the proof of our Main Theorem (stated in the Introduction) we must show that the only linear-fractional maps that induce nontrivially essentially normal composition operators are the parabolic non-automorphisms. This we do in the next two sections. For the rest of this section we assume that the Main Theorem has been proven, and use it to help characterize the essentially self-adjoint composition operators induced by linear-fractional maps.

A bounded operator T on a Hilbert space is said to be essentially self-adjoint if T^{∗} − T is compact. It is easy to check that every such operator is essentially normal. The next result shows, for example, that the map φ(z) = (2−z)^{−1} induces on H^{2} an essentially self-adjoint composition operator.

4.2. Theorem. Suppose φ ∈ LFT (𝕌). Then C_{φ} is nontrivially essentially selfadjoint if and only if φ is parabolic with translation number t > 0.

Proof. Essentially self-adjoint operators are easily seen to be essentially normal, so, by the Main Theorem, we need only consider operators C_{φ} that are nontrivially essentially normal. Thus we may assume that φ is a parabolic non-automorphism, hence the map τ of §2.6(a)(i) conjugates φ to the translation w → w + t with Re t > 0. As we observed in §2.6, we may without loss of generality assume that φ has its fixed point at 1, so φ is given by the explicit formula (9). Cowen’s formula, along with some algebraic manipulation, provides

C_{φ}^{∗} − C_{φ} = T_{g}(C_{σ} − C_{φ})T_{h}^{∗} + (T_{g}T_{h}^{∗} − I)C_{φ} + T_{g}[C_{φ}, T_{h}^{∗}]. (14)

Now the second term on the right is T_{b}C_{φ}, where b(z) = g(z)\overline{h(z)} − 1 on the unit circle. We see from (9) that h(1) = 2 = 1/g(1), so b(1) = 0. Thus b and φ satisfy the hypotheses of Lemma 3.3, so the operator in question is compact. We have already seen (in the proof of Proposition 3.4) that last term on the right is compact, so upon noting once again that T_{g} and T_{h}^{∗} are both invertible operators, we see that: C_{φ} is essentially self-adjoint if and only if C_{σ} − C_{φ} is compact.

In case Im t = 0, i.e. t > 0, then a glance at (9) reveals that σ = φ (we could also have discovered this “coordinate free” by transferring the representation σ = ρ ∘ φ^{−1} ∘ ρ to the right half-plane), so C_{σ} − C_{φ} = 0, hence C_{φ} is essentially self-adjoint.

If Im t ≠ 0 then either (9) or an examination of the situation in the right halfplane reveals that σ is the parabolic map that fixes the point 1 and has translation number \overline{t}. In other words, if φ(z) = τ^{−1}(τ(z) + t) and σ(z) = τ^{−1}(τ(z) + \overline{t}) for z ∈ 𝕌 where, as usual, τ(z) = (1+z)/(1−z). By §2.7, for each positive real number λ, the bounded analytic function f_{λ} is an H^{2}-eigenvector of C_{φ} (resp. C_{σ}) for the eigenvalue e^{−λt} (resp. e^{−λ\overline{t}}). Thus for each λ > 0 the function f_{λ} is an eigenvector of C_{σ} − C_{φ}, with eigenvalue e^{−λ\overline{t}} − e^{−λt} = 2i e^{−λRe t} sin(λ Im t). Because Im t ≠ 0 these eigenvalues fill up a nontrivial interval of the imaginary axis that therefore lies in the spectrum of C_{σ} − C_{φ}. But the Riesz Theory demands (among other things) that the spectrum of a compact operator be at most countable, hence C_{σ} − C_{φ} is not compact, i.e., C_{φ} is not essentially self-adjoint.

Had we been willing to use C^{∗}-algebra methods along with more detailed information about the spectra and essential spectra of parabolically induced composition operators, we could have given a very short proof of this last result that does not depend on Cowen’s formula; see §8 for more details.

5. Hyperbolic non-automorphisms.

As pointed out in §2.6, hyperbolic self-maps of 𝕌 that are not automorphisms have two fixed points: one on the unit circle and the other in the complement of the unit circle. To use Proposition 3.4 in studying the composition operators induced by such maps φ we must examine more closely the relationship between φ and the map σ that occurs in Cowen’s adjoint formula.

5.1. Lemma. Suppose φ ∈ LFT (𝕌) has a fixed point ω ∈ ∂𝕌. Then:

(a) σ′(ω) = \frac{1}{φ′(ω)}.

(b) If φ ∉ Aut (𝕌) then σ ∘ φ and φ ∘ σ are parabolic (with fixed point at ω).

(c) σ ∘ φ commutes with φ ∘ σ.

Proof. (a) If φ is an automorphism then this follows immediately from the fact that σ = φ^{−1} (see the second paragraph of §3.1). For φ ∉ Aut (𝕌), let us write ω = e^{it_{0}} and φ^{−1}(e^{it}) = γ(t). Thus γ is a complex-valued function that is differentiable on a real interval centered at t_{0}. For t in this interval let

β(t) := σ(e^{it}) = ρ(φ^{−1}(ρ(e^{it}))) = ρ(γ(t)),

where the last equality arises from the definition of γ and the fact that the inversion ρ restricts to the identity map on ∂𝕌. Consideration of real and imaginary parts shows the real-valued function t → |γ(t)|^{2} to be differentiable at t_{0}, with derivative at t_{0} equal to twice the real dot product of γ(t_{0}) and γ′(t_{0}) (where complex numbers are now viewed as plane vectors). Now the vector γ′(t_{0}) is tangent to the path of γ at e^{it_{0}}, and since γ(t_{0}) ∈ ∂𝕌, this path is tangent to the unit circle. Thus γ′(t_{0}) is tangent to the unit circle, and so orthogonal to γ(t_{0}), hence \frac{d|γ|^{2}}{dt} (t_{0}) = 0. This, along with the quotient rule for differentiation yields

β′(t_{0}) = \frac{γ′(t_{0})}{|γ(t_{0})|^{2}} = γ′(t_{0}).

By the Chain Rule, the left-hand side of this equation is just ie^{it_{0}}σ′(ω), while the right-hand side is ie^{it_{0}}(φ^{−1})′(ω) = ie^{it_{0}}/φ′(ω). Thus σ′(ω) = 1/φ′(ω), as desired.

(b) Since φ ∉ Aut (𝕌), neither σ ∘ φ nor φ ∘ σ is the identity map. Moreover it follows from (a) and the chain rule that both maps have derivative 1 at their common fixed point ω, hence (see [18, Chapter 0], for example) both are parabolic.

(c) This is clear if φ ∈ Aut (𝕌), since then σ = φ^{−1}. Otherwise the result follows from (b) above; the two maps in question are parabolic, and both have fixed point ω. Thus by §2.6(a)(i) they commute under composition.

We remark that the proof of part (a) above works just as well if we merely assume that φ(ω) = ζ ∈ ∂𝕌. The conclusion then is: σ′(ζ) = 1/φ′(ω).

5.2. Theorem. If φ ∈ LFT (𝕌) is a hyperbolic non-automorphism with a fixed point on ∂𝕌 then C_{φ} is not essentially normal.

Proof. By Proposition 3.4 we need only show that [C_{σ}, C_{φ}] is not compact. Let’s first note that this commutator is not zero, i.e., that φ and σ do not commute. Both φ and σ share a fixed point on ∂𝕌. Since φ is hyperbolic, it has another fixed point p in the Riemann sphere, but not on ∂𝕌 (since φ is not an automorphism of 𝕌). Now σ is also hyperbolic, and its non-boundary fixed point is ρ(p) ≠ p. Thus σ does not commute with φ (else σ(p) would be a fixed point of φ not on ∂𝕌 and not equal to p, thus endowing φ with too many fixed points). It follows that ψ := φ ∘ σ and χ := σ ∘ φ are distinct linear fractional selfmaps of 𝕌 with the same boundary fixed point as φ. By Lemma 5.1(b) ψ and χ are both parabolic, and since they have the same fixed point, they commute, and therefore so do the composition operators C_{ψ} and C_{χ}.

Now the argument proceeds as in the proof of Theorem 4.2: for each positive real number λ, the bounded analytic function f_{λ} is an H^{2}-eigenvector of C_{ψ} (resp. C_{χ}) for the eigenvalue e^{−λs} (resp. e^{−λt}). Thus for each λ > 0, f_{λ} is an eigenvector of [C_{σ}, C_{φ}] = C_{ψ} −C_{χ}, with eigenvalue e^{−λs} −e^{−λt}. The fact that s ≠ t guarantees that these eigenvalues fill up a nontrivial plane curve lying in the spectrum of [C_{σ}, C_{φ}], hence by the Riesz Theory, [C_{σ}, C_{φ}] is not compact, and therefore C_{φ} is not essentially normal.

6. New Proofs for the Remaining Cases

The results of this section come from Zorboska’s paper [20], but our proofs emphasize function theory over operator theory. Together with the work of the previous sections, these results finish the proof of our Main Theorem.

6.1. Theorem. Suppose that φ ∈ LFT (𝕌)\Aut (𝕌) with ∥φ∥_{∞} = 1, and that φ has no fixed point on ∂𝕌. Then C_{φ} is not essentially normal on H^{2}.

Proof. The hypotheses on φ insure that there are points ω, η ∈ ∂𝕌 with ω ≠ η and φ(ω) = η. Upon taking adjoints in Cowen’s theorem we can represent C_{φ} in terms of C_{σ} as follows:

C_{φ} = T_{h} C_{σ}^{∗} T_{g}^{∗}. (15)

For p ∈ 𝕌 recall the H^{2}-reproducing kernel K_{p} for p, given by (4). Using (15) and (7) we see that for each p ∈ 𝕌:

C_{φ}K_{p} = T_{h} C_{σ}^{∗} (T_{g}^{∗}K_{p}) = T_{h} C_{σ}^{∗}(\overline{g(p)} K_{p}) = \overline{g(p)} h C_{σ}^{∗}K_{p} ,

from which (6) yields

C_{φ}K_{p} = \overline{g(p)} h K_{σ(p)} . (16)

We proceed in the spirit of [20, Proposition, page 291] using as test functions the “normalized reproducing kernels” k_{p} := K_{p}/∥K_{p}∥, noting that

∥K_{p}∥^{2} = ⟨K_{p}, K_{p}⟩ = K_{p}(p) = \frac{1}{1 − |p|^{2}} (p ∈ 𝕌). (17)

It follows from this, (16), and (6) that for each p ∈ 𝕌:

∥ [C_{φ}^{∗}, C_{φ}] k_{p}∥ ≥ |⟨[C_{φ}^{∗}, C_{φ}] k_{p}, k_{p}⟩| = |∥C_{φ}k_{p}∥^{2} − ∥C_{φ}^{∗}k_{p}∥^{2}|
= |(1 − |p|^{2})|g(p)|^{2}∥h K_{σ(p)}∥^{2} − (1 − |p|^{2})∥K_{φ(p)}∥^{2}| ,

so that

∥ [C_{φ}^{∗}, C_{φ}] k_{p}∥ ≥ \frac{1 − |p|^{2}}{1 − |φ(p)|^{2}} − const. \frac{1 − |p|^{2}}{1 − |σ(p)|^{2}} (p ∈ 𝕌), (18)

where the constant, which is independent of p, takes into account the positive lower bounds for the moduli of h and g, as discussed in §3.1.

We estimate the first term on the right-hand side of (18) when p approaches ω radially, i.e., p = rω and r → 1−:

\frac{1 − |p|^{2}}{1 − |φ(p)|^{2}} = \frac{1 + r}{1 + |φ(rω)|} \frac{1 − r}{1 − |φ(rω)|}

≥ [1 + o(1)] \frac{|ω − rω|}{|η − φ(rω)|}
→ \frac{1}{|φ′(ω)|} ,

where the second line follows from the reverse triangle inequality and the fact that φ(rω) → η ∈ ∂𝕌. Thus

lim inf_{r→1−} \frac{1 − r^{2}}{1 − |φ(rω)|^{2}} ≥ \frac{1}{|φ′(ω)|} (19)

(actually, it follows from the Julia-Carathéodory Theorem that there is equality here).

For the second term, note that because φ(ω) = η ∈ ∂𝕌 we have (from σ = ρ ∘ φ^{−1} ∘ ρ) that σ(η) = ω, hence σ(ω) = σ(σ(η)). Because φ has no fixed point on ∂𝕌, neither does σ, hence, as noted previously in §2.6(b), ∥σ ∘ σ∥_{∞} < 1. Thus σ(ω) is a point of 𝕌, and so the second term on the right-hand side of (18) converges to zero as p → ω. Thus (18) and (19) yield

lim inf_{r→1−} ∥ [C_{φ}^{∗}, C_{φ}] k_{rω}∥ ≥ \frac{1}{|φ′(ω)|} > 0 . (20)

Now {k_{rω} : 0 ≤ r < 1} is a family of unit vectors in H^{2} that converges weakly to zero as r → 1−. Since compact operators take weakly convergent sequences to norm convergent ones, we see from (20) that [C_{φ}^{∗}, C_{φ}] is not compact, hence C_{φ} is not essentially normal.

The idea behind the proof of Theorem 6.1 also works in the automorphic case. Alternatively, one can use the commutator formula, which simplifies considerably because σ = φ^{−1} (see §3.1). We outline this idea in §6.3.

6.2. Theorem. Essentially normal automorphism-induced composition operators on H^{2} must be normal (i.e., induced by rotations).

Proof. Suppose φ is an automorphism of 𝕌. Then for each positive integer n,

⟨[C_{φ}^{∗}, C_{φ}] z^{n}, z^{n}⟩ = ∥C_{φ}z^{n}∥^{2} − ∥C_{φ}^{∗}z^{n}∥^{2} = ∥φ^{n}∥ − ∥C_{φ}^{∗}z^{n}∥^{2},

hence, because φ is an automorphism (making all its powers unit vectors in H^{2}),

⟨[C_{φ}^{∗}, C_{φ}] z^{n}, z^{n}⟩ = 1 − ∥C_{φ}^{∗}z^{n}∥^{2} (21)

By Cowen’s formula, for each positive integer n:

C_{φ}^{∗}z^{n} = T_{g}C_{σ}T_{h}^{∗}z^{n} = T_{g}C_{σ}(\overline{c}T_{z}^{∗} + \overline{d})z^{n} = T_{g}C_{σ}(\overline{c}z^{n−1} + \overline{d}z^{n}) ,

so that

C_{φ}^{∗}z^{n} = (\overline{c} + \overline{d}σ) σ^{n−1}g . (22)

A little calculation shows that

\overline{c} + \overline{d}σ(z) = \overline{Δ} z g(z) ,

where Δ := ad − bc ≠ 0, so the result of (22) can be rewritten:

C_{φ}^{∗}z^{n} = σ(z)^{n−1}\overline{Δ} z g(z)^{2} (|z| ≤ 1). (23)

Now σ is also an automorphism of 𝕌, so its absolute value is ≡ 1 on ∂𝕌, hence (23) yields, for ζ ∈ ∂𝕌:

|(C_{φ}^{∗}z^{n})(ζ)| = |\overline{Δ}g(ζ)^{2}| = |\frac{\overline{Δ}}{(−\overline{b}ζ + \overline{d})^{2}} | = |σ′(ζ)| .

From this and (21) we see that for each positive integer n:

⟨[C_{φ}^{∗}, C_{φ}] z^{n}, z^{n}⟩ = 1 − ∥σ′∥^{2}. (24)

A routine calculation with power series shows that

∥σ′∥^{2} > \frac{1}{π} ∫_{𝕌} |σ′|^{2} dA,

where dA is two-dimensional Lebesgue measure on the plane. The strict inequality comes from the fact that C_{φ} is not normal, hence (by Schwartz’s Normality Theorem) σ is not a rotation, and so σ′ is not constant. Because σ is univalent, the integral on the right is just the area of σ(𝕌) = 𝕌 (recall that σ is an automorphism), which is π. Thus ∥σ′∥ > 1 which, along with (24), shows that the numbers ⟨[C_{φ}^{∗}, C_{φ}] z^{n}, z^{n}⟩ are non-zero and do not depend on n. However {z^{n}} is a sequence of unit vectors in H^{2} weakly convergent to zero, so if [C_{φ}^{∗}, C_{φ}] were compact this sequence of numbers would converge to zero. Thus [C_{φ}^{∗}, C_{φ}] is not compact, i.e., C_{φ} is not essentially normal.

6.3. Automorphic case via Commutator Formula. Suppose φ(z) = (az + b)/(cz + d) ∈ Aut (𝕌). Because σ = φ^{−1} the Commutator Formula of Theorem 3.2 simplifies to

[C_{φ}^{∗}, C_{φ}] = T_{g}C_{φ}^{−1}[T_{h}^{∗}, C_{φ}] + T_{g−g∘φ}T_{h}^{∗}.

Upon systematically applying the following easily-checked relations:

(a) C_{φ}T_{ψ} = T_{ψ∘φ}C_{φ} if ψ ∈ H^{∞},

(b) T_{ψ}T_{χ} = T_{ψχ} if ψ and χ are in H^{∞}, and

(c) T_{z}^{∗}T_{z} = I, one sees, after some patient calculation, that [C_{φ}^{∗}, C_{φ}]T_{z}T_{σ} = T_{μ}, where

μ(z) = zg(z)(\overline{c} + \overline{d}σ(z)) − σ(z)g(φ(z))(\overline{c} + \overline{d}z) (z ∈ 𝕌).

Thus μ ∈ H^{∞} and

μ(0) = −\overline{c}σ(0)g(φ(0)) = \frac{\overline{c}^{2}d}{\overline{d}(|d|^{2} − |b|^{2})}. (25)

Note that d ≠ 0 (else φ would either be constant or have a pole at the origin).

Now suppose C_{φ} is essentially normal. Then [C_{φ}^{∗}, C_{φ}]T_{z}T_{σ}, a.k.a. T_{μ}, is compact, which renders μ ≡ 0. By (25) this forces c = 0, hence φ is affine. But affine automorphisms are rotations about the origin, hence C_{φ} is normal. □

7. Beyond Linear Fractional

We have seen that if φ ∈ LFT (𝕌) is a parabolic non-automorphism, then C_{φ} is non-trivially essentially normal. To produce further examples of non-trivially essentially normal composition operators, we use the following simple idea: if C_{φ} is essentially normal and ψ is another holomorphic selfmap of 𝕌 for which C_{φ} − C_{ψ} is compact, then C_{ψ} is also essentially normal. Thus we are led to develop criteria that ensure C_{φ} − C_{ψ} is compact. We depend on the following theorem ([17, Theorem 3.2]).

7.1. Difference Theorem. For each pair φ, ψ of distinct holomorphic self-maps of 𝕌, define

I(φ, ψ) = ∫_{∂𝕌} \frac{|φ − ψ|^{2}}{(min\{1 − |φ|, 1 − |ψ|\})^{3}} dm. (26)

If I(φ, ψ) < ∞, then C_{φ} − C_{ψ} is compact; moreover, ∥C_{φ} − C_{φ}∥ ≤ \sqrt{I(φ, ψ)}.

We investigate I(φ, ψ) for selfmaps φ and ψ that extend smoothly to at least one point of ∂𝕌.

7.2. Definition. Let n be a positive integer, let ζ ∈ ∂𝕌, and let 0 ≤ ε < 1. Following [1, p. 50], we say that the self-map φ of 𝕌 belongs to C^{n+ε}(ζ) provided that φ is differentiable at ζ up to order n (viewed as a function with domain 𝕌∪{ζ}) and, for z ∈ 𝕌, has the expansion

φ(z) = ∑_{k=0}^{n} \frac{φ^{(k)}(ζ)}{k!}(z − ζ)^{k} + γ(z),

where γ(z) = o(|z − ζ|^{n+ε}) as z → ζ from within 𝕌.

It is not difficult to show that φ ∈ C^{n}(ζ) whenever φ^{(n)} extends continuously to 𝕌 ∪ {ζ} (but, contrary to the claim made on [1, page 50], the converse is not true).

Theorem 2.2 of [12] shows that a necessary condition for compactness of C_{φ} −C_{ψ} is that φ and ψ have the same first-order boundary data, meaning that if one of φ or ψ has finite angular derivative at ζ ∈ ∂𝕌, then so does the other, and at ζ both functions have the same value and angular derivative.

In [2] it is shown that if extra smoothness assumptions are placed on φ and ψ, then boundary data agreement up to second order derivatives is necessary for compactness of C_{φ} − C_{ψ}. Here we show that in the presence of even more smoothness, along with a boundary-contact restriction, this necessary condition becomes sufficient for C_{φ} − C_{ψ} to be compact.

7.3. Definition. We say that φ and ψ have the same second-order boundary data at ζ ∈ ∂𝕌 provided that both functions belong to C^{2}(ζ), and

(a) φ(ζ) = ψ(ζ),

(b) φ and ψ have the same (finite) angular derivative at ζ, and

(c) φ′′(ζ) = ψ′′(ζ).

Observe that requirement (b) forces the common value of φ(ζ) and ψ(ζ) to have modulus 1; i.e. 1 = |φ(ζ)| = |ψ(ζ)|.

7.4. Definition. For a self-map φ of 𝕌 and for a point ζ ∈ ∂𝕌, let

φ^{−1}({η}) = {ζ ∈ ∂𝕌 : η belongs to the cluster set of φ at ζ}.

Thus ζ belongs to φ^{−1}({η}) if and only if there is a sequence (z_{n}) in 𝕌 with limit ζ such that (φ(z_{n})) has limit η.

7.5. Theorem. Suppose that φ and ψ are self-maps of 𝕌 such that

(a) each takes 𝕌 into a proper subdisc of 𝕌 that is internally tangent to the unit circle at η;

(b) φ^{−1}({η}) = ψ^{−1}({η}) = {ζ};

(c) each belongs to C^{3}({ζ});

(d) φ and ψ have the same second-order boundary data at ζ. Then C_{φ} − C_{ψ} is compact.

Proof. Let E be the full-measure subset of ∂𝕌 consisting of points at which both φ and ψ have radial limit. Upon applying hypotheses (c) and (d), we see that there is a bounded analytic function γ on 𝕌, with γ(z) = o(|z − ζ|^{3}) as z → ζ, such that

φ(z) − ψ(z) = \frac{1}{6}(φ′′′(ζ) − ψ′′′(ζ))(z − ζ)^{3} + γ(z).

Hence, there is a constant C_{1} such that for every z in 𝕌 ∪ E,

|φ(z) − ψ(z)| ≤ C_{1}|ζ − z|^{3}.

Because both φ and ψ are self-maps of 𝕌, φ′(ζ) = ψ′(ζ) is nonzero (by the Julia-Carathéodory Theorem); this, together with hypothesis (b) shows that there is a constant C_{2} such that for every z in 𝕌 ∪ E

\frac{|ζ − z|}{|η − φ(z)|} ≤ C_{2} and \frac{|ζ − z|}{|η − ψ(z)|} ≤ C_{2}.

Finally, by hypothesis (a) there is a constant C_{3} such that for every z in 𝕌 ∪ E,

\frac{|η − φ(z)|^{2}}{1 − |φ(z)|} ≤ C_{3} and \frac{|η − ψ(z)|^{2}}{1 − |ψ(z)|} ≤ C_{3}.

Fix λ ∈ E and suppose for definiteness that |φ(λ)| ≤ |ψ(λ)|. Then the estimates just derived show that the integrand on the right-hand side of (26) is bounded above by

C_{1}^{2} \frac{|ζ − λ|^{6}}{(1 − |ψ(λ)|)^{3}} ≤ C_{1}^{2}C_{2}^{6} \frac{|η − ψ(λ)|^{6}}{(1 − |ψ(λ)|)^{3}} ≤ C_{1}^{2}C_{2}^{6}C_{3}^{3},

and the same is true if |ψ(λ)| ≤ |φ(λ)|. Thus the integrand on the right side of (26) is bounded on E, hence (since E has full measure in ∂𝕌) its integral I(φ, ψ) is finite. By the Difference Theorem (Theorem 7.1 above), C_{φ} − C_{ψ} is therefore compact.

Remarks. (a) With a little more care, one can show establish the conclusion of the preceding theorem under the weaker hypothesis that φ ∈ C^{5/2+ε}(ζ) for some ε > 0. In this case, the integrand on the right-hand side of (26) is bounded by a constant multiple of 1/|ζ − z|^{1−2ε}, a function that is integrable over ∂𝕌.

(b) In similar fashion, by increasing the order of boundary-data agreement in part (d) of the statement of Theorem 7.5, one can increase the order of contact allowed and still obtain a compact difference of composition operators.

(c) The proof of the preceding theorem may easily be modified to show that C_{φ} − C_{ψ} is compact provided (i) F := {ζ : |φ(ζ)| = 1} = {ζ : |ψ(ζ)| = 1} is finite, (ii) φ and ψ are C^{3} at each point in F, (iii) φ and ψ have the same second-order boundary data at each ζ ∈ F, (iv) φ^{−1}({φ(ζ)}) = {ζ} = ψ^{−1}({ψ(ζ)}) for each ζ ∈ F, and (v) there are proper subdiscs D_{φ(ζ)} of 𝕌 for each ζ ∈ F such that D_{φ(ζ)} is internally tangent to ∂𝕌 at φ(ζ) and φ(𝕌) ∪ ψ(𝕌) ⊆ ∪_{ζ∈∂𝕌}D_{φ(ζ)}.

We are now in a position to extend the characterization of essentially normal linear-fractional composition operators provided by our Main Theorem to a class of composition operators that might be described as “essentially linear fractional”. We say that the holomorphic selfmap φ is essentially linear fractional provided that it satisfies the hypotheses of the following theorem.

7.6. Theorem. Let φ be a self-map of 𝕌 with ∥φ∥_{∞} = 1. Suppose that

(a) φ(𝕌) is contained in a proper subdisc of 𝕌 internally tangent to the unit circle at η;

(b) φ^{−1}({η}) consists of one element, say ζ ∈ ∂𝕌; and

(c) φ belongs to C^{3}({ζ}); then there is a linear-fractional mapping ψ having the same second-order boundary data at ζ as does φ, and such that C_{φ} − C_{ψ} is compact.

Proof. Let φ_{1}(z) = η̄φ(ζz) so that φ_{1} is a selfmap of 𝕌 that fixes 1 and belongs to C^{3}({1}). Suppose that there exists ψ ∈ LFT (𝕌) such that C_{φ_{1}} − C_{ψ} is compact. Let ψ_{1} be the linear-fraction map given by ψ_{1}(z) = ηψ(ζ̄z), we have

C_{φ} − C_{ψ_{1}} = C_{ζ̄z}(C_{φ_{1}} − C_{ψ})C_{ηz}

is compact. Moreover, if φ_{1} and ψ have the same second-order boundary data at 1, then the same will be true of φ and ψ_{1} at ζ. Thus, without loss of generality, we may assume that η = ζ = 1.

We will transfer attention from 𝕌 to the right half-plane via the mapping

T (z) = \frac{1 + z}{1 − z} .

Suppose that φ is an analytic self-map of 𝕌 in C^{2}(1) that fixes 1. Then, letting φ′(1) = p and φ′′(1) = a, the right half-plane incarnation of φ, Φ := T ∘ φ ∘ T^{−1}, has the following representation

Φ(w) = \frac{1}{p}w + (\frac{1}{p} − 1 + \frac{a}{p^{2}} ) + Γ(w), (27)

where Γ(w) = o(1) as |w| → ∞. Now suppose, in addition, that φ maps 𝕌 into a proper subdisc of 𝕌 that is internally tangent to ∂𝕌 at 1. Translated to the right half-plane, this means that there is positive constant c such that Re Φ(w) > c whenever Re w > 0. Hence, using representation (27), we see that Re (1/p − 1 + a/p^{2}) ≥ c > 0 with this additional assumption on the way φ contacts ∂𝕌 at 1. Note that if p ≥ 1, then the contact assumption yields Re a > 0 (i.e., Re φ′′(1) > 0).

Now let φ be an essentially linear fractional map that fixes the point 1. Let

(28) Ψ(w) = \frac{w}{p} + (\frac{1}{p} − 1 + \frac{a}{p^{2}} ) , where p = φ′(1) and a = φ′′(1),

and let ψ(z) = (T^{−1} ∘Ψ∘T)(z). The work of the preceding paragraph shows that ψ is a linear fractional self-map of 𝕌 whose image is a proper subdisc of 𝕌 internally tangent to ∂𝕌 at 1 and whose second-order boundary data at 1 agrees with that of φ at 1. Thus Theorem 7.5 shows that C_{φ} − C_{ψ} is compact, as desired.

7.7. Corollary. Suppose that φ is an essentially linear fractional selfmap of 𝕌 with Denjoy-Wolff point ω. If ω ∈ 𝕌 or if ω ∈ ∂𝕌 and φ′(ω) < 1, then C_{φ} is not essentially normal.

Proof. Suppose ω ∈ ∂𝕌. Matching up with the statement of Theorem 7.6, we have ω = ζ = η. Suppose that C_{φ} − C_{ψ} is compact, where ψ is the linear-fractional map provided by Theorem 7.6. Because ψ has the same first-order boundary data as does φ and maps 𝕌 into a proper subdisc of 𝕌, ψ is a hyperbolic non-automorphism and thus Theorem 5.2 shows that C_{ψ} is not essentially normal. Because C_{φ} − C_{ψ} is compact, C_{φ} is also not essentially normal.

If ω ∈ 𝕌, then, because φ is essentially linear-fractional, there are points ζ and η in ∂𝕌 such that the hypotheses of Theorem 7.6 are satisfied. Note that because φ’s Denjoy-Wolff point lies in 𝕌, if ζ = η (that is, φ fixes ζ), then |φ′(ζ)| > 1. Let ψ be the linear-fractional map of Theorem 7.6. Because ψ’s first-order data at ζ agrees with that of φ, ψ is hyperbolic and either Theorem 5.2 (when ζ = η) or Theorem 6.1 (when ζ ≠ η) shows that C_{ψ} is not essentially normal; hence, neither is C_{φ}.

7.8. Corollary. Suppose that φ is an essentially linear fractional selfmap of 𝕌 with Denjoy-Wolff point ω ∈ ∂𝕌. If φ′(ω) = 1, then C_{φ} is essentially normal.

Proof. This corollary follows from Theorem 7.6 and Theorem 4.1 in the same way that the “ω ∈ ∂𝕌” case of Corollary 7.7 followed from Theorem 7.6 and Theorem 5.2.

The preceding corollary provides nontrivially essentially normal composition operators induced by selfmappings of 𝕌 that are not univalent. For example, let Ψ be the self-map of the right halfplane given by

Ψ(w) = w + 6 + \frac{4}{w + 1}.

and let φ = T ∘ Ψ ∘ T^{−1} where T (z) = (1 + z)/(1 − z). Then it’s easy to check that φ is not univalent, yet is essentially linear-fractional: C_{φ} − C_{ψ} is compact where ψ is the parabolic member of LFT (𝕌) given by ψ(z) = T^{−1}(T (z) + 6).

However valence restrictions do play some role in the story of essential normality. The following theorem shows, for example, that if C_{φ} is not compact, then φ(z^{2}) cannot induce an essentially normal composition operator on H^{2}.

7.9. Theorem. Suppose φ and ν are holomorphic self-maps of 𝕌, with C_{φ} noncompact and ν inner. If ν ∉ Aut (𝕌) then C_{φ∘ν} is not essentially normal.

Proof. Let ∥T ∥_{e} denote the essential norm of a Hilbert space operator T, i.e., its distance, measured in the operator norm, to the closed subspace of compact operators. For composition operators on H^{2} a formula for the essential norm involving function-theoretic properties of the inducing function was given in [16], and, as pointed out by Cima and Matheson in [4], the derivation of this formula showed that

lim sup_{|p|↑1} ∥C_{φ}k_{p}∥ = ∥C_{φ}∥_{e}, (29)

where k_{p} is the “normalized reproducing kernel” that first appeared in the proof of Theorem 6.1. Suppose first that ν(0) = 0. In this case Nordgren [13] has shown that C_{ν} is an isometry on H^{2}; in particular, C_{φ∘ν} = C_{ν}C_{φ} inherits the non-compactness of C_{φ}.

Let χ := φ ∘ ν. By (29) there is sequence (p_{n}) in U that converges to some point η ∈ ∂𝕌, and is such that lim_{n} ∥C_{χ}k_{p_{n}}∥ = ∥C_{χ}∥_{e} > 0, where the positivity of the essential norm reflects the non-compactness just noted for C_{χ}. As in the proof of Theorem 6.1 we have:

∥[C_{χ}^{∗}, C_{χ}]k_{p_{n}}∥ ≥ ⟨[C_{χ}^{∗} C_{χ}]k_{p_{n}}, k_{p_{n}}⟩
= ∥C_{χ}k_{p_{n}}^{2}∥ − ∥C_{χ}^{∗}k_{p_{n}}∥^{2}
= ∥C_{χ}∥_{e}^{2} + ε_{n} − \frac{1 − |p_{n}|^{2}}{1 − |χ(p_{n})|^{2}} ,

where ε_{n} → 0 as n → ∞.

Suppose χ does not have a finite angular derivative at η. By the Julia-Carathéodory theorem (§2.2), (1 − |χ(p)|^{2})/(1 − |p|^{2}) → ∞ as p → η, so the last calculation shows that

lim sup_{|p|→1−} ∥[C_{χ}^{∗}, C_{χ}]k_{p}∥ ≥ ∥C_{χ}∥_{e}^{2} > 0

and therefore [C_{χ}^{∗}, C_{χ}] is not compact.

If, on the other hand, χ does have a finite angular derivative at η, then Julia-Carathéodory asserts that lim inf_{n}(1 − |χ(p_{n})|^{2})/(1 − |p_{n}|^{2}) ≥ |χ′(η)|, whereupon our calculation results in:

lim sup_{|p|→1−} ∥[C_{χ}^{∗}, C_{χ}]k_{p}∥ ≥ ∥C_{χ}∥_{e}^{2} − \frac{1}{|χ′(η)|}. (30)

Because ν(0) = 0 and ν is not an automorphism, it follows from [7, Lemma 7.33] that |ν′(η)| > 1. This, along with an argument (which we omit), again based on the Julia-Carathéodory theorem, shows that χ passes along the finiteness of its angular derivative at η to ν, and also that the angular derivative of φ at ν(η) ∈ ∂𝕌 is finite. By the chain rule for angular derivatives [18, §4.8, page 74, Exercise 10],

|χ′(η)| = |φ′(ν(η))| |ν′(η)| > |φ′(ν(η))|.

Now it is known [16, Theorem 3.3, page 385] that sup_{ζ∈∂𝕌} |φ′(ζ)|^{−1/2} ≤ ∥C_{φ}∥_{e}, hence the last estimate yields

|χ′(η)|^{−1} < |φ′(η(ν))|^{−1} ≤ ∥C_{φ}∥_{e}^{2} = ∥C_{χ}∥_{e}^{2},

where we have used (29) and the fact that C_{ν} is an isometry to obtain the final equality. Therefore the right-hand side of (30) is strictly positive, which once again establishes the non-compactness of [C_{χ}^{∗} , C_{χ}].

So far we have proven “non-essential-normality” for C_{χ} = C_{φ∘ν} under the additional assumption that ν(0) = 0. If ν(0) ≠ 0, set ψ(z) = (ν(0) − z)/(1 − \overline{ν(0)}z), so that ψ ∈ Aut (𝕌), ψ(ν(0)) = 0, and ψ is its own compositional inverse. Let ν_{1} := ψ ∘ ν and φ_{1} := φ ∘ ψ, so that φ ∘ ν = φ_{1} ∘ ν_{1}. Note that ν_{1} is inner, not an automorphism, and ν_{1}(0) = 0; also the non-compactness of C_{φ} transfers to C_{φ_{1}} = C_{ψ}C_{φ} because C_{ψ}, being an isomorphism of H^{2}, is bounded below. Thus the result of the last paragraph shows that C_{φ∘ν} = C_{φ_{1}∘ν_{1}} is not essentially normal.

8. Final Remarks and Further Directions

8.1. Essentially self-adjoint operators. (a) At the end of §4 we commented that Theorem 4.2 could be proven by abstract methods. The idea is to identify C_{φ} with its coset in the Calkin algebra, the quotient of the algebra of bounded operators on H^{2} by the closed ideal of compacts. This is a C^{∗}-algebra in the involution inherited from the bounded operators (“send an operator to its adjoint”) [8, Theorem 5.38, page 139], and to say that an operator on H^{2} is essentially selfadjoint (resp. essentially normal) means that its coset in the Calkin algebra is self-adjoint (resp. normal) with respect to this involution.

Now a normal element of a C^{∗} algebra is self-adjoint if and only if its spectrum lies in the real line, so an essentially normal composition operator C_{φ} is essentially self-adjoint if and only if its essential spectrum (the spectrum of its coset in the Calkin algebra) is real. We pointed out at the end of §2.7 that the spectrum of C_{φ} consists of the curve Γ_{t} := {e^{−λt} : λ ≥ 0} along with the origin. If Im t ≠ 0 then Γ_{t} spirals into the origin, while if Im t = 0 (i.e., if t > 0) then Γ_{t} is the real segment (0, 1]. In all cases the spectrum of C_{φ} has no interior, and therefore coincides with the essential spectrum. Conclusion: C_{φ} is essentially self-adjoint if and only if t > 0.

(b) An element of a C^{∗}-algebra is called positive if it is self-adjoint and its spectrum lies in the non-negative real axis. Let us call an operator on Hilbert space essentially positive if it is essentially self-adjoint with essential spectrum in [0, ∞). The proof above shows that, among the composition operators induced by linearfractional selfmaps of 𝕌, the essentially self-adjoint ones are actually essentially positive.

8.2. Compact perturbations of normal operators. A consequence of the celebrated Brown-Douglas-Fillmore theorem [3] is that:

An essentially normal Hilbert-space operator T is a compact perturbation of a normal operator if and only if its Fredholm index function i(T − λI) is trivial (i.e., identically 0 on the essential resolvent of T).

As we discussed in part (a) of the preceding subsection, if φ ∈ LFT (𝕌) is parabolic, then the essential spectrum of C_{φ} has connected complement and empty interior. It follows that the index function of C_{φ} is trivial so that C_{φ} is a compact perturbation of a normal operator. We can restate this observation as follows:

Every essentially normal, linear-fractionally induced composition operator is a compact perturbation of a normal operator.

The results of §7 generalize this to composition operators induced by maps φ that are “essentially linear-fractional.”

8.3. Linear-fractional models. One of the deepest (and perhaps most underappreciated) results about holomorphic selfmaps of the unit disc is that each one has a linear-fractional model. For univalent maps this has a particularly attractive statement:

If the holomorphic map φ : 𝕌 → 𝕌 is a univalent then there is a univalent map τ mapping 𝕌 onto a simply connected domain G and a linearfractional map Φ with Φ(G) ⊂ G such that φ = τ^{−1} ∘ Φ ∘ τ.

In case φ has no fixed point in 𝕌 it turns out that Φ can be chosen to belong to LFT (𝕌), with no fixed point in 𝕌, hence either hyperbolic or parabolic. For certain problems it has been possible to use this result or variants of it to “transfer” results about composition operators induced by linear-fractional maps to more general situations; see [6], [1], or [18, Chapter 8] for more on this. We have already used a very special case of this idea, modeling general parabolic self-maps of 𝕌 by translations of the right half-plane. Is it possible that our Main Theorem might extend to all holomorphic self-maps of 𝕌 in the sense that the only such maps that induce nontrivially essentially normal composition operators on H^{2} are the ones with parabolic non-automorphic models? The work of §7 provides some evidence that this may be the case.

References

[1] P. S. Bourdon and J. H. Shapiro, Cyclic Phenomena for Composition Operators, Memoirs Amer. Math. Soc. #596, Vol. 125, 1997.

[2] P. S. Bourdon, Components of linear-fractional composition operators, preprint.

[3] L. G. Brown, R. Douglas, and P. Fillmore, Unitary equivalence modulo the compact operators and extensions of C^{∗}-algebras, Proceedings of a Conference on Operator Theory (Dalhousie Univ., Halifax, N.S., 1973), Springer Lecture Notes in Mathematics 345 (1973), 58–128.

[4] J. A. Cima and A. Matheson, Essential norms of composition operators and Alexandrov measures, Pacific J. Math., 179 (1997), 59–64.

[5] C. C. Cowen, Linear fractional composition operators on H^{2}, Integral Eqns. Op. Th. 11 (1988), 151–160.

[6] C. C. Cowen, Composition operators on H^{2}, J. Operator Th. 9 (1983), 77–106.

[7] C. C. Cowen and B. D. MacCluer, Composition Operators on Spaces of Analytic Functions, CRC Press 1995.

[8] R. G. Douglas, Banach Algebra Techniques in Operator Theory, Academic Press 1972 (2nd ed. Springer 1998).

[9] P. L. Duren, Theory of H^{p} Spaces, Academic Press 1970.

[10] F. Jafari et al., editors, Studies on Composition Operators, Contemp. Math. Vol. 213, American Math. Soc. 1998.

[11] J. E. Littlewood, On inequalities in the theory of functions, Proc. London Math. Soc. 23 (1925), 481–519.

[12] B. D. MacCluer, Components in the space of composition operators, Integral Equations Operator Theory 12 (1989), 725–738.

[13] E. A. Nordgren, Composition operators, Canad. J. Math. 20 (1968), 442–449.

[14] W. Rudin, Real and Complex Analysis, third edition, McGraw-Hill 1987.

[15] H. J. Schwartz, Composition operators on H^{p}, Thesis: University of Toledo 1969.

[16] J. H. Shapiro, The essential norm of a composition operator, Annals of Math. 125 (1987), 375–404.

[17] J. H. Shapiro and C. Sundberg, Isolation amongst the composition operators, Pacific J. Math. 145 (1990), 117–151.

[18] J. H. Shapiro, Composition Operators and Classical Function Theory, Springer Verlag 1993.

[19] J. H. Shapiro and P. D. Taylor, Compact, nuclear, and Hilbert-Schmidt composition operators on H^{2}, Indiana Univ. Math. J. 125 (1973), 471–496.

[20] N. Zorboska, Closed range essentially normal composition operators are normal, Acta Sci. Math. (Szeged) 65 (1999) 287–292.

Washington and Lee University, Lexington, VA 24450

E-mail address: pbourdon@wlu.edu

Central Michigan University, Mount Pleasant, MI 48859

E-mail address: sivaram.narayan@cmich.edu
E-mail address: levibks@power-net.net

Michigan State University, East Lansing, MI 48824, USA

E-mail address: shapiro@math.msu.edu