0703-bias from wrong covariance matrix estimation

Full Covariance Matrix Case

1. Chi-Squared Function: The chi-squared ($\chi^2$) function is defined by:

   $$\chi^2 = (\mathbf{d} - \mathbf{m}(\mathbf{p}))^T \mathbf{C}^{-1} (\mathbf{d} - \mathbf{m}(\mathbf{p}))$$

   where:

   • $\mathbf{d}$ is the data vector.

   • $\mathbf{m}(\mathbf{p})$ is the model prediction vector dependent on parameters $\mathbf{p}$.

   • $\mathbf{C}$ is the covariance matrix of the data.

2. Derivative Calculation: To compute the gradient $\nabla_{\mathbf{p}} \chi^2$, consider the residual vector $\mathbf{r} = \mathbf{d} - \mathbf{m}(\mathbf{p})$. The derivative of $\chi^2$ with respect to a parameter $p_i$ is:

   $$\frac{\partial \chi^2}{\partial p_i} = \frac{\partial}{\partial p_i} \left( \mathbf{r}^T \mathbf{C}^{-1} \mathbf{r} \right)$$

   Applying the product rule, this becomes:

   $$\frac{\partial \chi^2}{\partial p_i} = 2 (\mathbf{C}^{-1} \mathbf{r})^T \frac{\partial \mathbf{r}}{\partial p_i}$$

   Since $\mathbf{r} = \mathbf{d} - \mathbf{m}(\mathbf{p})$, it follows:

   $$\frac{\partial \mathbf{r}}{\partial p_i} = -\frac{\partial \mathbf{m}(\mathbf{p})}{\partial p_i}$$

   Therefore, the gradient expression simplifies to:

   $$abla_{\mathbf{p}} \chi^2 = -2 \mathbf{J}^T \mathbf{C}^{-1} \mathbf{r}$$

   where $\mathbf{J}$ is the Jacobian matrix of $\mathbf{m}(\mathbf{p})$ with respect to $\mathbf{p}$.

Diagonal Covariance Matrix Case

If $\mathbf{C}$ is purely diagonal, then:

$$\mathbf{C} = \text{diag}(\sigma_1^2, \sigma_2^2, \ldots, \sigma_n^2)$$

$$\mathbf{C}^{-1} = \text{diag}\left(\frac{1}{\sigma_1^2}, \frac{1}{\sigma_2^2}, \ldots, \frac{1}{\sigma_n^2}\right)$$

The chi-squared function simplifies to:

$$\chi^2 = \sum_{i=1}^n \frac{(d_i - m_i(\mathbf{p}))^2}{\sigma_i^2}$$

The derivative with respect to $p_j$ is:

$$\frac{\partial \chi^2}{\partial p_j} = \sum_{i=1}^n \frac{-2 (d_i - m_i(\mathbf{p}))}{\sigma_i^2} \frac{\partial m_i(\mathbf{p})}{\partial p_j}$$

In summary, ignoring non-diagonal elements of the covariance matrix when they are significant (i.e., when data points are correlated) simplifies the calculation but can lead to incorrect parameter estimates. This approach assumes all measurements are independent, potentially overlooking crucial structural information about the data's variability and leading to biased results in parameter estimation.