Regression when x and y each have uncertaintiesn h Zzr 8 8
I have a set of $N$ points $(x_i,y_i)$. $X$ and $Y$ both have some noise associated with them due to measurement inaccuracy however the relationship of the underlying true values (i.e. if we could remove the noises) of these points should be of the form $y = mx +c$ where $m$ and $c$ are constants.
However, due to measurement inaccuracies in both $X$ and $Y$, I will get uncertainties in both my $m$ and $c$ values.
1)If I assume that my measurement inaccuracies for both $X$ and $Y$ are Gaussian distributed $\\epsilon$ ~ $N(0,\\sigma)$ how do I obtain the most likely $(m,c)$ and the uncertainties/confidence in both.
2) If I instead know that the uncertainties are different for $X$ and $Y$ such that $\\sigma_x \\neq \\sigma_y$ where $\\epsilon_x$ ~ $N(0,\\sigma_x)$, $\\epsilon_y$ ~ $N(0,\\sigma_y)$ can I get a different estimate of $(m,c)$ and the uncertainties/confidence in both.
2 Answers
In both cases you want to use Deming regression. Case 1 is a special case of Deming regression called orthogonal regression, which minimizes the sum of squared perpendicular distances from the data points to the regression line. For case 2, the general case, you will need an estimate of the ratio $\\delta = \\sigma^2_y / \\sigma^2_x$ for the problem to be solvable.
-
$\\begingroup$ Interesting. Thanks for your response - I will look at the links and see if they answer my question. $\\endgroup$ – piccolo 7 hours ago
-
$\\begingroup$ Thanks. The wiki page doesn't mention anything about calculating the uncertainties in the estimates of $m$ and $c$. $\\endgroup$ – piccolo 6 hours ago
-
2$\\begingroup$ I've heard this recommendation before, but this paper (Smith 2009) argues that error in X is a bad criterion for choosing RMA regression (which I think is the same as Deming regression). It argues instead that symmetry is a better criterion for whether to choose OLS or RMA, and proposes a few alternatives for dealing with the problem of error in X. $\\endgroup$ – mkt 6 hours ago
As a general concept the problem of error in X is called measurement error.
In linear regression analysis it causes attenuation bias, which is considered as one of the sources of engogeneity. Measuremet error shrinks coefficient of the right-hand-side variable measured with an error towards zero. It causes not uncertainty of an estimator, but its inconsistency instead.
While mentioned in other answer deming regression is two-variable concept, the multivariate solutions include instrumental variable method as preferred option.
Formulas for attenuation bias in case of linear regression are precisely derived, for example in here. This means if you have a clue of the variation of the error, you might estimate severity of the problem and possibly correct for it.
Measurement in Y, in case of linear regression is less of the problem, as linear regression assumes random error in the dependent variable. It causes worse prediction and higher residual variance, but do not biased coefficients in any way.
-
$\\begingroup$ Thanks. How would you get an uncertainty in the estimates of $m$ and $c$ when you use Deming regression? $\\endgroup$ – piccolo 6 hours ago
-
1$\\begingroup$ As for deming regression, here are some nice notes which include variance formulas - if a variance or sd is a measure of uncertainty of what you are looking for: ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/… If looking for technical solution, I would suggest R: r-bloggers.com/deming-and-passing-bablok-regression-in-r $\\endgroup$ – cure 6 hours ago
-
$\\begingroup$ The last link suggest also Passing Bablok method. I did not mention it in my answer, but I guess it would be an interesting area of investigation here. $\\endgroup$ – cure 6 hours ago
-
$\\begingroup$ A word of warning about Deming regression: confidence intervals for the slope parameter are not guaranteed to be bounded, i.e. they can include $\\pm{\\infty}$. This can occur when the signal-to-noise ratio in the data is too low. $\\endgroup$ – Estacionario 5 hours ago