Weeks3 & 4: PCA and covariance

 

so I wrote this little widget to help you visualize it. So here is the same graph, and what we have here is basically the approximation of the function that we can play with, we can change from, we can change it using these sliders. So right now, we have the approximation that is zero everywhere, so it's basically the zero function, and it's clearly very far from the, from our target function, x minus four absolute value. So let's see what happens if we increase this coefficient. We see that we move, we move the constant to the middle of the function, more or less. Then when we add the second one, we add a sinusoid, and the sinusoid helps us get closer to the function, in some of the regions. It doesn't help us at all here, right, because it's zero, and it stays equidistance, but now if we add the cosine, that helps us also there. So what you see here is as you go and you add more and more components, you get closer and closer to the original, to the target function, and basically you can say, okay, these coefficients that I wrote that are down here are a representation for the approximation of the function. Okay, so that's a nice thing in and of itself. But now let's see a particular use of that, which is recovering from noise. So suppose I have a function that is really a sum of two sinusoids, so two times the vector one minus four times the vector five, but then I add to it noise. Okay, so this is random, normal  noise, so this is just what's called white noise. We add that, and what we get is something like this. So what we get is the orange line. Underlying it, the function that was really without noise is the blue line. So the question is can we recover that? Now if this function is really based on just the sinusoid and on a few of them, then we can do a very good job of that, and that's what we have here, okay, so, let's see, what we have here basically is that, if I remove these, okay. So now we have essentially the function zero. So this is, the approximation is zero, but now if we just look at the coefficients that corresponded to what was really in the original function, we see that that adds a little bit and now if we add this one, then we essentially get a perfect, a perfect reconstruction, almost perfect reconstruction. So of course we can ask what is happening if we change the others? And they will change the function a little bit, but they are not going to change it a lot. So what this means is, the essence of the function is captured by the coefficient c1 and c5, and the rest of them are much less important, and so the numbers in front of them are much smaller. Okay, so that basically lets us detect inside the noise what is really what we call the signal. Okay, so to summarize, functions can be thought of as vectors, and vectors can be thought of as functions. Okay, so you can go back and forth. The Fourier basis is a set of orthonormal functions made out of sines and cosines, and orthonormal functions can be used to remove noise that is added, that is added to a function that has a small representation in the basis we're using.

PCA is a way to analyze the covariance correlation of the elements of a vector

High Dimensional Vectors

1. Functions can be thought of as vectors and vice versa

2. Fourier Basis is a set of orthonormal functions consisting of sine and cosine

3. Orthonormal functions can help to remove noise added to functions

Computing PCA using RDD

PCA- Principal component Analysis is a way to analyze the covariance correlations between the elements of a vector. 

1

Goodness of fit- How well do the first three components represent the data (Eigen vectors may be)

Box Plots show 25% of the data to 75% of the data in Box along with Median inside the box. The line corresponds to 5% to 95% and the other points are outliers.

- Okay, so, we looked at some initial graphs that showed us the average and the standard deviation of different measurements, and now we want to dig a little bit deeper and look at correlation between the measurements on different days. So the way that we're going to do it, is we're going to use the PCA that we computed before, and see how we can explain things using the PCA. We'll start it here and we'll continue in the next video as well. Okay, so, the first and most important thing to look at is how much do the top eigenvectors explain? How much of the variance, of the total variance do they explain? So what we see here is that for tmin, tobserve, tmax, it is modest. It's basically the first one explains about 0.2, and then you get smaller, and then the top five explain about 0.3, here the top  

The best approximation that is the single vector is the mean.