Server Help

[Muskrat]

Here's a question for math people:

The common practice in function approximation is to the use the minimized sum of squares to find error. Why not absolute value?

Using squares gives more weight to big screw-ups and less to small screw-ups. Why not just give the amount of weight due to them? I know this will depend on someone's definition of "best fit," but I need some ideas.

[Dr Brain]

MMSE ususally results in a nicer closed form. Especially with normally (aka Gaussian) distributed functions.
_________________
Hyperspace Owner

Smong> so long as 99% deaths feel lame it will always be hyperspace to me

[SamHughes]

Let's step down a dimension, from functions in R->R to real numbers themselves, i.e. functions in {0} -> R, you could say.

The analogue to a least squares approximation is the average. The average, m = (a_1 + ... + a_n) / n, is the value that minimizes the sum, ((a_1 - m)^2 + ... + (a_n - m)^2).

Suppose that, instead, we wanted the value for m that minimizes the expression (|a_1 - m| + ... + |a_n - m|).

What value is that? Well, if n is odd, the value's the median. If n is even, you can use any value between the two closest to the median. Is the median what we want? It depends on what we want.

Now let's go back to the least squares approximation. It turns out that the same thing happens: if you want to find the line such that the absolute value distance is minimized, you'll have half the points above the line, and half the points below the line, and there isn't a unique solution.

But why not something in between? Why not minimize (|y_1 - f(x_1)|^1.5 + ... + |y_n - f(y_n)|^1.5) or something else? People prefer minimizing the sum of squares because that makes the computations easy and fast -- you can use simple principles to compute the solution -- you're really just finding the minimum of some paraboloid. And there is a huge mathematical framework built around this.

So people use least squares a lot because it makes the math easy, and beacuse it fits with other assumptions (that are preferred because they make the math easy), and people _definitely_ don't use minimal sum of absolute distances because there is not one solution. If you have sample points (0, 0), (100, -1), (100, 1), least squares makes you split the uprights, while sum of absolute distances only cares if you get it in.

[Cheese]

|3| = 3
|-3| = 3
_________________
SSC Distension Owner
SSCU Trench Wars Developer

[Samapico]

[rootbear75]

the perfect number: 6

[Samapico]

28

[SamHughes]

Why was 6 afraid of 20? Because 28 496.

[Samapico]

...

What?

What does 20 have to do with that?