0703-bias from wrong covariance matrix estimation

Full Covariance Matrix Case

  1. Chi-Squared Function: The chi-squared ($\chi^2$) function is defined by:

    χ2=(dm(p))TC1(dm(p))\chi^2 = (\mathbf{d} - \mathbf{m}(\mathbf{p}))^T \mathbf{C}^{-1} (\mathbf{d} - \mathbf{m}(\mathbf{p}))

    where:

    • $\mathbf{d}$ is the data vector.

    • $\mathbf{m}(\mathbf{p})$ is the model prediction vector dependent on parameters $\mathbf{p}$.

    • $\mathbf{C}$ is the covariance matrix of the data.

  2. Derivative Calculation: To compute the gradient $\nabla_{\mathbf{p}} \chi^2$, consider the residual vector $\mathbf{r} = \mathbf{d} - \mathbf{m}(\mathbf{p})$. The derivative of $\chi^2$ with respect to a parameter $p_i$ is:

    χ2pi=pi(rTC1r)\frac{\partial \chi^2}{\partial p_i} = \frac{\partial}{\partial p_i} \left( \mathbf{r}^T \mathbf{C}^{-1} \mathbf{r} \right)

    Applying the product rule, this becomes:

    χ2pi=2(C1r)Trpi\frac{\partial \chi^2}{\partial p_i} = 2 (\mathbf{C}^{-1} \mathbf{r})^T \frac{\partial \mathbf{r}}{\partial p_i}

    Since $\mathbf{r} = \mathbf{d} - \mathbf{m}(\mathbf{p})$, it follows:

    rpi=m(p)pi\frac{\partial \mathbf{r}}{\partial p_i} = -\frac{\partial \mathbf{m}(\mathbf{p})}{\partial p_i}

    Therefore, the gradient expression simplifies to:

    ablapχ2=2JTC1rabla_{\mathbf{p}} \chi^2 = -2 \mathbf{J}^T \mathbf{C}^{-1} \mathbf{r}

    where $\mathbf{J}$ is the Jacobian matrix of $\mathbf{m}(\mathbf{p})$ with respect to $\mathbf{p}$.

Diagonal Covariance Matrix Case

If $\mathbf{C}$ is purely diagonal, then:

C=diag(σ12,σ22,,σn2)\mathbf{C} = \text{diag}(\sigma_1^2, \sigma_2^2, \ldots, \sigma_n^2)
C1=diag(1σ12,1σ22,,1σn2)\mathbf{C}^{-1} = \text{diag}\left(\frac{1}{\sigma_1^2}, \frac{1}{\sigma_2^2}, \ldots, \frac{1}{\sigma_n^2}\right)

The chi-squared function simplifies to:

χ2=i=1n(dimi(p))2σi2\chi^2 = \sum_{i=1}^n \frac{(d_i - m_i(\mathbf{p}))^2}{\sigma_i^2}

The derivative with respect to $p_j$ is:

χ2pj=i=1n2(dimi(p))σi2mi(p)pj\frac{\partial \chi^2}{\partial p_j} = \sum_{i=1}^n \frac{-2 (d_i - m_i(\mathbf{p}))}{\sigma_i^2} \frac{\partial m_i(\mathbf{p})}{\partial p_j}

In summary, ignoring non-diagonal elements of the covariance matrix when they are significant (i.e., when data points are correlated) simplifies the calculation but can lead to incorrect parameter estimates. This approach assumes all measurements are independent, potentially overlooking crucial structural information about the data's variability and leading to biased results in parameter estimation.

Last updated