Karhunen Loeve Transform in Python

In the theory of stochastic processes, the Karhunen–Loève theorem (named after Kari Karhunen and Michel Loève), also known as the Kosambi–Karhunen–Loève theorem is a representation of a stochastic process as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval.

Here’s an exercise I did a week back to understand it better.

Karhunen Loeve Transform in Python

Karhunen Loeve Transform relies on the covariance matrix of a set of observation vectors. The basic algotithm is explained in these excellent lectures

And to see the KL transform in action visit wolfram.com

In [1]:
import numpy as np
In [2]:
def KLT(a):
    """
    Returns Karhunen Loeve Transform of the input and the transformation matrix and eigenval
    
    Ex:
    import numpy as np
    a  = np.array([[1,2,4],[2,3,10]])
    
    kk,m = KLT(a)
    print kk
    print m
    
    # to check, the following should return the original a
    print np.dot(kk.T,m).T
        
    """
    val,vec = np.linalg.eig(np.cov(a))
    klt = np.dot(vec,a)
    return klt,vec,val

Example 1

In [3]:
a  = np.array([[1,2,4],[2,3,10]])
print a,a.shape
[[ 1  2  4]
 [ 2  3 10]] (2, 3)

In [4]:
kk,m,val = KLT(a)
In [5]:
print kk
print m
print np.dot(kk.T,m).T
print val
[[-1.59602949 -2.86686851 -7.03449892]
 [-1.56610659 -2.18656465 -8.15572344]]
[[-0.94564854 -0.32519048]
 [ 0.32519048 -0.94564854]]
[[  1.   2.   4.]
 [  2.   3.  10.]]
[  0.09810743  21.23522591]

Example 2

In [6]:
a= np.random.rand(20,5)*100
kk,m,val = KLT(a)
print val
[  6.03748782e+03 +0.00000000e+00j   5.36177007e+03 +0.00000000e+00j
   3.94526413e+03 +0.00000000e+00j   2.80078918e+03 +0.00000000e+00j
  -2.80723774e-13 +2.73566185e-13j  -2.80723774e-13 -2.73566185e-13j
   3.15308859e-13 +2.18787274e-13j   3.15308859e-13 -2.18787274e-13j
  -3.21526974e-13 +3.59336478e-14j  -3.21526974e-13 -3.59336478e-14j
   4.10392909e-13 +0.00000000e+00j   3.28554436e-13 +0.00000000e+00j
   1.93536473e-13 +8.16943559e-14j   1.93536473e-13 -8.16943559e-14j
  -1.00036413e-13 +1.08479990e-13j  -1.00036413e-13 -1.08479990e-13j
   5.89399042e-14 +0.00000000e+00j  -1.01830548e-13 +0.00000000e+00j
  -3.35517071e-14 +5.19296828e-14j  -3.35517071e-14 -5.19296828e-14j]

In [7]:
print kk[:,0]
[ -19.88571370 +5.78613631j  -26.92980925 +1.1405507j
  -71.38367367+16.51826228j   41.42575279 +1.77169163j
   24.74718060 +7.73570152j   79.94127422 -8.9347779j
  -16.92612673-11.09979081j  -21.37461321 +2.7563032j
    7.03333553-10.45710615j   46.62805945+18.89607343j
    4.58608380 -5.40369704j  146.04910424 +0.5710281j
  -22.22228105 -2.42342784j  -16.60076029 -6.5282612j
  -38.29552191 +9.07952442j   -6.48539692 +8.84928717j
  -99.68353378 +9.9974666j   -65.96782860 -7.75414299j
  -10.08531081-25.27053509j -101.15451537 -1.9404897j ]

Example 3

In [8]:
a=np.array([[0, 0, 1], [0, 1, 1]])
kk,m,val = KLT(a)
In [9]:
print kk
[[  0.00000000e+00  -7.07106781e-01  -1.11022302e-16]
 [  0.00000000e+00   7.07106781e-01   1.41421356e+00]]

In [10]:
print m
[[ 0.70710678 -0.70710678]
 [ 0.70710678  0.70710678]]

In [11]:
print val
[ 0.5         0.16666667]

Visit wolfram.com for some great practical results

Here’s the python function for KLT

def KLT(a):
    """
    Returns Karhunen Loeve Transform of the input and 
    the transformation matrix and eigenval
    
    Ex:
    import numpy as np
    a  = np.array([[1,2,4],[2,3,10]])
    
    kk,m = KLT(a)
    print kk
    print m
    
    # to check, the following should return the original a
    print np.dot(kk.T,m).T
        
    """
    val,vec = np.linalg.eig(np.cov(a))
    klt = np.dot(vec,a)
    return klt,vec,val

See a better version of the post here

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s