In matrix analysis, there are several different matrix norms that you might use depending on the context of your particular problem. If you are treating the matrix as an operator acting on a the complex vector space Cⁿ, then you would likely use the operator norm. If you are considering the matrix as a density operator (i.e., if you’re a quantum information nerd like me) then you might want to use the trace norm. If you just want something that’s easy to calculate, you might be better off going with the Frobenius norm. These are three of the most well-studied and well-used matrix norms, and they have one very important thing in common — they are unitarily invariant. That is, if X ∈ M_n, then

Unitarily-invariant norms are particularly “nice” in that they satisfy submultiplicativity as well as various other desirable properties. Here I will present two particular families of unitarily-invariant norms, briefly discuss some of their applications, and then define a (new?) family of norms that encompass all of the other norms mentioned in this post as special cases.

Before proceeding, recall that for any matrix X ∈ M_n we can define the absolute value |X| of X to be the positive matrix square root of X^*X. Then the singular values of X, s₁(X), s₂(X), …, s_n(X), are defined to be the eigenvalues of |X|. Throughout this post we will assume that the singular values are ordered from largest to smallest (this is pretty standard practice when dealing with singular values):

Ky Fan Norms

Given a natural number k such that 1 ≤ k ≤ n, the Ky Fan k-norm of a matrix X ∈ M_n is defined to be the sum of the k largest singular values of X:

While Ky Fan norms aren’t extremely well-known, they have applications is matrix theory as well as quantum information theory. For example, they have recently appeared in [1] as a tool for determining whether a linear map from M_n to M_m is k-positive, which is one of the difficult open problems in quantum information. If P_k ⊆ M_n denotes the space of rank-k orthogonal projections (i.e., matrices such that P² = P^* = P), then it is not difficult to show that

Several properties of these norms are obvious from the definition — for example, the Ky Fan k-norm is upper-bounded by the Ky Fan (k+1)-norm and each Ky Fan norm is unitarily-invariant. One property that isn’t immediately obvious, however, is the following very cool result:

Fan Dominance Theorem [2, Section IV.2]. Let X, Y ∈ M_n. Then

if and only if

Schatten Norms

Given a real number p ≥ 1, the Schatten p-norm of a matrix X ∈ M_n is defined to be the standard vector p-norm of the vector of singular values of X:

There are numerous applications of Schatten norms in quantum information theory. For example, they are used to define completely bounded norms for linear maps acting on matrices, which are probably the most important norms for maps in that quantum information (see [3] for a particular paper that deals with these norms). As with the Ky Fan norms, the Schatten norms are unitarily-invariant and can be equivalently defined via an expression involving the trace:

One of the other nice properties of the Schatten p-norms is a modified submultiplicativity result, which states that if X,Y ∈ M_n then

Everything In Between

We have now seen two families of norms based on the singular values of a matrix, both of which are very important in matrix analysis as well as quantum information theory. The Ky Fan norms are given by summing the first k singular values, while the Schatten norms are given by computing the standard vector p-norm of the vector of singular values. So why have I never seen the natural generalization of these two families of norms — the vector p-norm of the first k singular values — defined?

Definition. Let X ∈ M_n, p ≥ 1 and 1 ≤ k ≤ n, with k a natural number. Then I define the (p,k)-singular norm of X to be

Notice that these norms are also unitarily-invariant, and as with the previously-defined norms, they are given by a relatively simple trace expression:

One particular case of these norms — the p = 2 case — actually appeared implicitly in [1], though they were (incorrectly? or just rug-sweepingly?) referred to as Ky Fan norms. I have also found a need for the p = 2 case of these norms in a recent project of mine that will hopefully be wrapped up in the next month or so.

I will finish by pointing out some special cases of this norm:

• If we allow p = ∞ by taking the limit as p → ∞ in the above definition, then the (∞,k)-singular norm coincides with the standard operator norm, regardless of k.
• When p = 1, the (1,k)-singular norm is exactly the Ky Fan k-norm.
• When k = n, the (p,n)-singular norm is exactly the Schatten p-norm.
• When p = 1, k = n (i.e., the Schatten 1-norm, which equals the Ky Fan n-norm), we recover exactly the trace norm.
• When p = 2, k = n (i.e., the Schatten 2-norm), we recover exactly the Frobenius norm.
• When p = 1, k = 1 (i.e., the Ky Fan 1-norm), we again obtain the operator norm.

References

1. D. Chruscinski, A. Kossakowski, Spectral Conditions for Positive Maps. Commun. Math. Phys. 290, 10511064 (2009). arXiv:0809.4909 [quant-ph]
2. R. Bhatia, Matrix analysis. Volume 169 of Graduate texts in mathematics (1997).
3. I. Devetak, M. Junge, C. King, M. B. Ruskai, Multiplicativity of completely bounded p-norms implies a new additivity result. Commun. Math. Phys. 266, 37-63 (2006). arXiv:quant-ph/0506196v2

Recall for matrices that a similarity transformation is one that takes a (square) matrix A to SAS^-1, where S is some invertible matrix of the same size as A. Standard linear algebra courses like to beat into students just how important similarity transformations are, which makes sense from their point of view because linear algebra is really more about studying operators than matrices themselves, and similarity transformations correspond to just a change of basis. The downside is that many people end up thinking of similarity as the equivalence relation among matrices and hence the Jordan canonical form as the canonical form. This month’s Lemma of the Month is about the canonical form under a different equivalence relation: *-congruence.

What is *-Congruence?

Two square matrices A and B are said to be *-congruent if there is an invertible matrix S such that SAS^* = B. Even though no inverses are present in that equation, invertibility really is required for any nice results to be obtained (for example, *-congruence would not even define an equivalence relation if S were allowed to be singular). For people who like knowing applications of mathematical tools before knowing the tools themselves, *-congruence has an important role when dealing with quadratic forms. Also, the inertia of a matrix, which will be defined in the next paragraph, comes into play when looking at sign pattern matrices (apologies for the lack of a wiki link; it seems as though sign pattern matrices are at that ugly stage after there are lots of papers written about them but before there is a Wikipedia page written about them).

Sylvester’s Law of Inertia

By far the most well-known result about *-congruence is Sylvester’s Law of Inertia, which gets its name from the seldom-used inertia of a matrix. The inertia of a Hermitian matrix A is defined to be the tuple

where n₊ is the number of positive eigenvalues of A, n₀ is the number of zero eigenvalues of A, and n_- is the number of negative eigenvalues of A (recall that Hermitian matrices have real eigenvalues, so this definition makes sense). Sylvester’s Law of Inertia is as follows:

Theorem [1,2]. Let A, B ∈ M_n be Hermitian. Then A and B are *-congruent if and only if they have the same inertia.

One obvious corollary of this theorem is that every Hermitian matrix A is *-congruent to a diagonal matrix with n₊ ones, n₀ zeroes, and n_- negative ones on the diagonal; this is the canonical form for Hermitian matrices under *-congruence.

Generalizations of Sylvester’s Law

If you’re a linear algebra nerd like me, then you might be thinking to yourself that Sylvester’s Law of Inertia seems trivial in light of the Spectral Decomposition Theorem; for any Hermitian A we could find a unitary so that UAU^* is real and diagonal, and then we could find a diagonal D such that DUAU^*D^* is scaled appropriately to satisfy the theorem. Setting S = DU then shows us that we can always arrive at the canonical form, and the remaining steps in the proof are all more or less trivial (this proof is provided in much more detail in the PDF attached to the end of this post).

Notice, however, that the Spectral Decomposition Theorem applies not only to Hermitian matrices, but to normal matrices as well. Thus, using the same logic outlined in the previous paragraph, it stands to reason that the canonical form under *-congruence for normal matrices should be a diagonal matrix in which each diagonal entry either has modulus one or zero. Indeed, this is essentially the content of the following theorem of Ikramov, which was proved in 2001.

Theorem [3]. Let A, B ∈ M_n be normal. Then A and B are *-congruent if and only if they have the same number of eigenvalues on each open ray from the origin in the complex plane.

This theorem truly generalizes Sylvester’s Law of Inertia; in the Hermitian case, the only two open rays that eigenvalues can lie on are the positive real line and the negative real line. What I would like to know, though, is why on Earth this theorem was first proved/published so recently — it can be proved in about a paragraph using only ideas from an undergraduate linear algebra course, and I’m pretty sure that anyone who uses the Spectral Decomposition Theorem semi-regularly would see the main idea of the proof more or less instantly. Is it just a hole in the literature that wasn’t noticed until recently?

Anyway, for the really interested reader, there is a generalization of these results to the case when A and B need not even be normal, but it’s quite technical so I won’t present it here. See [4] if you’re curious.

Thanks to Roger Horn for his great series of lectures at the Summer School and Advanced Workshop on Trends and Developments in Linear Algebra, which introduced me to the generalizations of Sylvester’s Law.

Related Links:

• Lemma of the Month #3: Sylvester’s Law of Inertia and *-Congruence [pdf]

References:

1. Sylvester, J. J., A demonstration of the theorem that every homogeneous quadratic polynomial is reducible by real orthogonal substitutions to the form of a sum of positive and negative squares. Philosophical Magazine IV: 138–142 (1852).
2. Horn, R. and Johnson, C., Matrix analysis. Cambridge University Press (1990).
3. Ikramov, Kh. D., On the inertia law for normal matrices. Doklady Math. 64 (2001) 141-142.
4. Horn, R. and Sergeichuk, V., Canonical forms for complex matrix congruence and *congruence. Linear Algebra Appl. 416 (2006) 1010-1032. arXiv:0709.2473v1 [math.RT]

Given a completely positive linear map E: M_n → M_n, its multiplicative domain, denoted MD(E), is an algebra defined as follows:

Roughly speaking, MD(E) is the largest subalgebra of M_n on which E behaves multiplicatively. It will be useful to make this notion precise:

Definition. Let A be a subalgebra of M_n and let π : A → M_n. Then π is said to be a *-homomorphism if π(ab) = π(a)π(b) and π(a^*) = π(a)^* for all a,b ∈ A.

Thus, MD(E) is roughly the largest subalgebra of M_n such that, when E is restricted to it, E is a *-homomorphism (I keep saying “roughly speaking” because of the “∀b ∈ M_n” in the definition of MD(E) — the definition of a *-homomorphism only requires that the multiplicativity hold ∀b ∈ A). Probably the most well-known result about the multiplicative domain is the following theorem of Choi [1,2], which shows how the multiplicative domain simplifies when E is such that E(I) = I (i.e., when E is unital):

Theorem [Choi]. Let E: M_n → M_n be a completely positive map such that E(I) = I. Then

One thing in particular that this theorem shows is that, when E(I) = I, the multiplicative domain of E only needs to be multiplicative within MD(E) (i.e., we can remove the “roughly speaking” that I spoke of earlier).

MD(E) in Quantum Error Correction

Before moving onto how MD(E) plays a role in quantum error correction, let’s consider some examples to get a better feeling for what the multiplicative domain looks like.

• If E is the identity map (that is, it is the map that takes a matrix to itself) then MD(E) = M_n, the entire matrix algebra.
• If E(a) = Diag(a) (i.e., E simply erases all of the off-diagonal entries of the matrix a), then MD(E) = {Diag(a)}, the set of diagonal matrices.

Notice that in the first example, the map E is very well-behaved (as well-behaved as a map ever could be); it preserves all of the information that is put into it. We also see that MD(E) is as large as possible. In the second example, the map E does not preserve information put into it (indeed, one nice way to think about matrices in the quantum information setting is that the diagonal matrices are “classical” and rest of the matrices are “quantum” — thus the map E(a) = Diag(a) is effectively removing all of the “quantumness” of the input data). We also see that MD(E) is tiny in this case (too small to put any quantum data into).

The above examples then hint that if the map E preserves quantum data, then MD(E) should be large enough to store some quantum information safely. This isn’t quite true, but the intuition is right, and we get the following result, which was published as Theorem 11 in this paper:

Theorem. Let E: M_n → M_n be a quantum channel (i.e., a completely positive map such that Tr(E(a)) = Tr(a) for all a ∈ M_n) such that E(I) = I. Then MD(E) = UCC(E), the algebra of unitarily-correctable codes for E.

What this means is that, when E is unital, its multiplicative domain encodes exactly the matrices that we can correct via a unitary operation. This doesn’t tell us anything about correctable codes that are not unitarily-correctable, though (i.e., matrices that can only be corrected by a more complicated correction operation). To capture these codes, we have to generalize a bit.

Generalized Multiplicative Domains

In order to generalize the multiplicative domain, we can require that the map E be multiplicative with another map π that is already a *-homomorphism, rather than require that it be multiplicative with itself. This is the main theme of this paper, which was submitted for publication this week. We define generalized multiplicative domains as follows:

Definition. Let A be a subalgebra of M_n, let E : M_n → M_n be completely positive, and let π : A → M_n be a *-homomorphism. Then the multiplicative domain of E with respect to π, denoted MD_π(E), is the algebra given by

It turns out that these generalized multiplicative domains are reasonably well-behaved and generalize the standard multiplicative domain in exactly the way that we wanted: they capture all correctable codes for arbitrary quantum channels (see Theorem 11 of the last paper I mentioned). Furthermore, there are even some characterizations of MD_π(E) analogous to the theorem of Choi above (see Theorems 5 and 7, as well as Corollary 12).

References:

1. M.-D. Choi, A Schwarz inequality for positive linear maps on C*-algebras. Illinois Journal of Mathematics, 18 (1974), 565-574.
2. V. I. Paulsen, Completely Bounded Maps and Operator Algebras, Cambridge Studies in Advanced Mathematics 78, Cambridge University Press, Cambridge, 2003.

This month’s Lemma of the Month comes all the way from the Summer School and Advanced Workshop on Trends and Developments in Linear Algebra in Trieste, Italy, where Professor Rajendra Bhatia presented a lecture that introduced several simple yet endlessly interesting matrix inequalities. I will briefly present the various results here without proof, though the proof of the first “stepping stone” lemma is provided in the PDF attached to the bottom of this post. The truly interested reader can find full proofs in Professor Bhatia’s notes (follow the link above) or in [1].

Recall that one of the defining properties of a matrix norm is that it satisfies the triangle inequality:

So what can we say about generalizing the backward triangle inequality to matrices? We can of course replace A by A – B in the above equation to find the following backward triangle inequality:

However, what happens if we swap the roles of the absolute value and the matrix norm on the left-hand side? That is, if we recall that |A| is the positive semidefinite part of A (i.e., the square root of A^*A), then can we say something like

It turns out that the answer to this question is heavily norm-dependent, so we will focus on the norm that gives the simplest answer: the Frobenius norm, which I will denote by ||•||₂.

Theorem [Araki-Yamagami]. Let A, B ∈ M_n. Then

Building Up to the Result

In order to prove the result, one can proceed by proving a series of simple commutant-type matrix norm inequalities, which are interesting in their own right.

Lemma. Let A, B ∈ M_n be positive semi-definite and let X ∈ M_n be arbitrary. Then

Lemma. Let A ∈ M_n be Hermitian. Then

Lemma. Let A, B ∈ M_n be Hermitian. Then

Finally, it just wouldn’t seem right to post from Italy without sharing a bit of it. Thus, I leave you with a taste of the highlight of the trip (excepting the mathematics, of course).

The colosseum as of eight hours ago

Related Links

• Lemma of the Month #2: A Backward Triangle Inequality for Matrices [pdf]

References

1. H. Araki and S. Yamagami, An inequality for the Hilbert-Schmidt norm, Commun. Math. Phys., 81 (1981) 89-98.

The Collatz conjecture (or hailstone problem or 3n+1 problem) is a problem that is so simple to state that grade-schoolers can understand it, yet has been approached from an innumerable variety of angles and has resisted mathematicians for decades now. The problem is as follows:

Pick a positive integer. If it’s even then divide it by 2. If it’s odd, multiply it by 3 and add 1. Now apply this procedure to the result. Repeat. Will you always eventually hit 1 if you continue in this way?

For example, if you start with 11 then you multiply by 3 and add 1 to get 34, divide by 2 to get 17, and continuing similarly gets you the rest of the sequence: 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1. Before you go trying to find a counterexample to the conjecture, know that it has been verified by computer search for all starting numbers up to 20 × 2⁵⁸ ≈ 5.764 × 10¹⁸. The way that we are going to look at this problem today is the “pretty” approach; we’re going to extend the Collatz conjecture over the complex plane and look at the fractal defined by its iteration.

Define the Collatz function f(x) as follows:

Take a moment or two to convinve yourself that if x is a positive integer, then f(x) is the next number after x in its Collatz sequence (e.g., f(11) = 34, f(34) = 17, and so on). To extend this function to the real numbers, simply recall that (-1)^x = cos(πx). In fact, this gets us an extension to the complex numbers at the same time, and after some simplification we arrive at:

Well hey, that’s a holomorphic function so it has a notion of a Julia set and we can study the fractal that its iterates induce. Indeed, we can obtain the following image pretty easily with standard fractal-generation software:

The Collatz f(z) fractal

The fractal is located on the complex plane, and the horizontal line through the middle of the image is the real line. Black regions are regions in which the orbit of that number is bounded, while other colours indicate that the orbit of that number is unbounded (notice the large region of bounded numbers around z = 0). The big “spikes” that occur along the real line are, as we would expect, located at the integers (the image above is wide enough that you can see the spikes at z = -2, -1, 1, and 2). Instead of proceeding with the Collatz function as I have defined it, I’m going to introduce a modified Collatz function g(z) as follows:

Observe that this function, like f(z), always maps natural numbers to terms that appear later in their Collatz sequence (e.g., g(11) = 17, g(17) = 13). The benefit of this function is that it has the additional property that g(1) = 1. That is, 1 is a fixed point of g(z), whereas it is part of a period-3 cycle of f(z). The fractal induced by g(z) is:

The Collatz g(z) fractal

Close-up of the g(z) fractal at z = 1

I originally moved to using g(z) instead of f(z) because the plot of the f(z) fractal from earlier indicated to me that there may be a ball of black (i.e., boundedness) of non-zero radius around each of the integers, but proving this for f(z) seemed to be quite difficult (as we would expect, since it would basically imply half of the Collatz conjecture). Somewhat strangely, even though there appear to be balls around the integers in the fractal induced by f(z), these balls vanish in the fractal induced by g(z). The image to the right is a close-up of the above fractal (the point of convergence is z = 1).

Nonetheless, the real line still seems to behave reasonably nicely under the action of g(z); it’s not difficult to prove that the fractal contains the real line segment [0,N] for some large number N that is similar in magnitude to the least M such that the conjecture is true for all n ≤ M (known to be at least 5×10¹⁸ or so, as mentioned earlier). However, many (non-natural number) points in that interval do not converge to 1.

So what now? If the Collatz conjecture is true, then z = 1 is the unique natural number fixed point of g(z), yet there seem to be points arbitrarily close to z = 1 that don’t even converge under iteration of g(z). Why are there smaller spikes between the integers in both of these fractals, and where are the spikes centered? Does the restriction of the Collatz function to the numbers at the center of those spikes have any simple interpretation? Who knows, I’ll explore some more in the future. Until then, enjoy some pretty pictures.

g(z) fractal at z = 4

g(z) fractal at z = 8

g(z) fractal at z = 16

g(z) fractal at z = -8

There’s an old math paradox that says that all natural numbers are interesting, since otherwise there would have to be a smallest uninteresting number, and that in itself is pretty interesting. Of course, this is meant to show that ideas in the English language do not always translate to well-defined mathematical concepts, but let’s ignore our better mathematical sense and tinker with the idea of how interesting different numbers are a little bit. In particular, I claim that 11630 is utterly bland and uninteresting.

Why 11630?

Before saying why 11630 is uninteresting, I should probably say what I consider “interesting” to even mean. Interesting, to me, means that it has some (semi-unique) mathematical property that sets it apart from other numbers. 11 is interesting because it is prime, 16 is interesting because it is a perfect power (16 = 4²), and so on. Clearly, there is some ambiguity in this definition, since one could consider composite numbers interesting, just as I considered prime numbers interesting. Additionally, do we consider 2719 interesting simply because it is prime? I’d say no, since there are hundreds of prime numbers that come before it — perhaps only the first few numbers that satisfy a given property should be considered interesting as a result of it?

Using these ideas, it seems like determining how interesting a number is would be a task perfectly suited to the Online Encyclopedia of Integer sequences (OEIS). If you’re unfamiliar with it (i.e., if you’re not a math person and have no place reading my blog), the OEIS is a database containing thousands of (you guessed it) integer sequences that have been submitted by users over the last decade or so (such as the sequence of prime numbers 2, 3, 5, 7, 11,… and the sequence of perfect powers 1, 4, 8, 9, 16,…). Presumably, if an integer is interesting then it will appear in at least one or two of the 159437 sequences contained in the database, right? Indeed, it seems that we can get a rough idea of how interesting a number is by looking at how many sequences that number appears in in the database compared to other numbers of similar size.

11630 is the first number that is not listed in a single sequence in the OEIS. It is not prime, nor is it highly composite (11630 = 2×5×1163). It doesn’t have any particularly notable residue properties, and it doesn’t come up in counting problems. It’s boring in every way, and it seems as though not a single mathematician has found a use for it in the last dozen or so years (let me know if you’ve discovered otherwise).

What Numbers are Interesting?

First off, I’m not going to deal with particularly small numbers (say in the range of 1 – 50) since, as the strong law of small numbers quips, these numbers will appear all over the place just because they’re small. You could probably argue that most (if not all) of them are interesting, so I’ll instead take a look at a couple larger numbers that are particularly interesting.

The number 421 appears in some 1894 sequences, while most numbers that size appear in about 940 sequences. This seems to indicate that 421 is a particularly interesting number, but why? What’s so special about 421? Well, it’s prime (in fact, it’s a twin prime, Pythagorean prime, cuban prime, lucky number of Euler prime, additive prime, and irregular prime), it’s congruent to 1 mod 2,3,4, 5, 6,7, 10, 12, it’s the sum of five primes, and 421² = 420² + 29². Similarly, 512 appears in 2116 sequences even though most numbers around 512 appear in about 800 sequences. This is perhaps less surprising than 421, since 512 = 2⁹ = 8³ = 16² + 16² is a number that somehow seems “nice” due to it being a perfect power. Additionally, 512 is a Leyland number, Harshad number, and it comes up in all sorts of counting problems.

What of the Paradox?

Recalling the paradox from earlier, we are now forced to ask ourselves whether or not 11630 is now interesting as a result of it being the first number not included in the OEIS. Rather than come up with an answer, I’m going to take the easy way out and let the OEIS decide. The sequence of uninteresting numbers is 11630, 12067, 12407, 12887, 13258, 13794, 13882, 13982, 14018, 14163,… Let’s submit that to the OEIS and see if they consider it to be interesting or not.

Update [June 13, 2009]: I got word back via e-mail today that this sequence didn’t make the cut. So there you have it — these numbers truly are uninteresting.

Update [November 12, 2009]: It looks like 11630 is now listed in the OEIS. Additionally, 12067 was recently added, meaning that 12407 is now the first uninteresting number.

Downloads:

• OEIS counts (1 – 50000) [Excel spreadsheet — 6.81MB]
• OEIS counts (1 – 50000) [.txt — 444kB]