# Karhunen Loeve Transform in Python

In the theory of stochastic processes, the Karhunen–Loève theorem (named after Kari Karhunen and Michel Loève), also known as the Kosambi–Karhunen–Loève theorem is a representation of a stochastic process as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval.

Here’s an exercise I did a week back to understand it better.

# Karhunen Loeve Transform in Python¶

Karhunen Loeve Transform relies on the covariance matrix of a set of observation vectors. The basic algotithm is explained in these excellent lectures

And to see the KL transform in action visit wolfram.com

In [1]:
```import numpy as np
```
In [2]:
```def KLT(a):
"""
Returns Karhunen Loeve Transform of the input and the transformation matrix and eigenval

Ex:
import numpy as np
a  = np.array([[1,2,4],[2,3,10]])

kk,m = KLT(a)
print kk
print m

# to check, the following should return the original a
print np.dot(kk.T,m).T

"""
val,vec = np.linalg.eig(np.cov(a))
klt = np.dot(vec,a)
return klt,vec,val
```

### Example 1¶

In [3]:
```a  = np.array([[1,2,4],[2,3,10]])
print a,a.shape
```
```[[ 1  2  4]
[ 2  3 10]] (2, 3)

```
In [4]:
```kk,m,val = KLT(a)
```
In [5]:
```print kk
print m
print np.dot(kk.T,m).T
print val
```
```[[-1.59602949 -2.86686851 -7.03449892]
[-1.56610659 -2.18656465 -8.15572344]]
[[-0.94564854 -0.32519048]
[ 0.32519048 -0.94564854]]
[[  1.   2.   4.]
[  2.   3.  10.]]
[  0.09810743  21.23522591]

```

### Example 2¶

In [6]:
```a= np.random.rand(20,5)*100
kk,m,val = KLT(a)
print val
```
```[  6.03748782e+03 +0.00000000e+00j   5.36177007e+03 +0.00000000e+00j
3.94526413e+03 +0.00000000e+00j   2.80078918e+03 +0.00000000e+00j
-2.80723774e-13 +2.73566185e-13j  -2.80723774e-13 -2.73566185e-13j
3.15308859e-13 +2.18787274e-13j   3.15308859e-13 -2.18787274e-13j
-3.21526974e-13 +3.59336478e-14j  -3.21526974e-13 -3.59336478e-14j
4.10392909e-13 +0.00000000e+00j   3.28554436e-13 +0.00000000e+00j
1.93536473e-13 +8.16943559e-14j   1.93536473e-13 -8.16943559e-14j
-1.00036413e-13 +1.08479990e-13j  -1.00036413e-13 -1.08479990e-13j
5.89399042e-14 +0.00000000e+00j  -1.01830548e-13 +0.00000000e+00j
-3.35517071e-14 +5.19296828e-14j  -3.35517071e-14 -5.19296828e-14j]

```
In [7]:
```print kk[:,0]
```
```[ -19.88571370 +5.78613631j  -26.92980925 +1.1405507j
-71.38367367+16.51826228j   41.42575279 +1.77169163j
24.74718060 +7.73570152j   79.94127422 -8.9347779j
-16.92612673-11.09979081j  -21.37461321 +2.7563032j
7.03333553-10.45710615j   46.62805945+18.89607343j
4.58608380 -5.40369704j  146.04910424 +0.5710281j
-22.22228105 -2.42342784j  -16.60076029 -6.5282612j
-38.29552191 +9.07952442j   -6.48539692 +8.84928717j
-99.68353378 +9.9974666j   -65.96782860 -7.75414299j
-10.08531081-25.27053509j -101.15451537 -1.9404897j ]

```

### Example 3¶

In [8]:
```a=np.array([[0, 0, 1], [0, 1, 1]])
kk,m,val = KLT(a)
```
In [9]:
```print kk
```
```[[  0.00000000e+00  -7.07106781e-01  -1.11022302e-16]
[  0.00000000e+00   7.07106781e-01   1.41421356e+00]]

```
In [10]:
```print m
```
```[[ 0.70710678 -0.70710678]
[ 0.70710678  0.70710678]]

```
In [11]:
```print val
```
```[ 0.5         0.16666667]

```

Visit wolfram.com for some great practical results

Here’s the python function for KLT

```def KLT(a):
"""
Returns Karhunen Loeve Transform of the input and
the transformation matrix and eigenval

Ex:
import numpy as np
a  = np.array([[1,2,4],[2,3,10]])

kk,m = KLT(a)
print kk
print m

# to check, the following should return the original a
print np.dot(kk.T,m).T

"""
val,vec = np.linalg.eig(np.cov(a))
klt = np.dot(vec,a)
return klt,vec,val

```

See a better version of the post here