Tjyylli

Create chunks from an array

What is better: yes / no radio, or simple checkbox?

Formatting a table to look nice

A bug in Excel? Conditional formatting for marking duplicates also highlights unique value

How do I deal with being envious of my own players?

Book about a time-travel war fought by computers

What could be a means to defeat a childrens’ nightmare?

What is a term for a function that when called repeatedly, has the same effect as calling once?

What is the minimum amount of skill points per HD?

Is every open circuit a capacitor?

Where is this quote about overcoming the impossible said in "Interstellar"?

I can't die. Who am I?

Why doesn't "adolescent" take any articles in "listen to adolescent agonising"?

How can neutral atoms have exactly zero electric field when there is a difference in the positions of the charges?

PTIJ: What’s wrong with eating meat and couscous?

School performs periodic password audits. Is my password compromised?

How can I highlight parts in a screenshot

Can a space-faring robot still function over a billion years?

Caulking a corner instead of taping with joint compound?

Are there other characters in the Star Wars universe who had damaged bodies and needed to wear an outfit like Darth Vader?

GPL code private and stolen

How does signal strength relate to bandwidth?

How to merge row in the first column in LaTeX

Why won't the strings command stop?

Inverse of the covariance matrix of a multivariate normal distribution

Second moment of draws from a multivariate normal covariance matrixWhy do we use the determinant of the covariance matrix when using the multivariate normal?Multivariate normal with singular covarianceUnderstanding the marginal distribution of multivariate normal distributionInference for multivariate normal when the sample covariance matrix is not invertibleIf an inverse covariance matrix is sparse, what can I say about the covariance matrix?Comparison between analytic gradient and numerical gradient for multivariate normal distribution wrt mean and covarianceIn multivariate normal distribution, what does it imply if the covariance matrix does not have full ranksampling from a multivariate normal distributionEstimating means and covariance matrix of a multivariate normal distribution

4

1

$begingroup$

Is the covariance matrix of a multivariate normal distribution always invertible?

normal-distribution covariance-matrix matrix linear-algebra

share|cite|improve this question

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
  $endgroup$
  – Xi'an
  yesterday
  
  
  

  
  
  
  

add a comment | 

4

1

$begingroup$

Is the covariance matrix of a multivariate normal distribution always invertible?

normal-distribution covariance-matrix matrix linear-algebra

share|cite|improve this question

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
  $endgroup$
  – Xi'an
  yesterday
  
  
  

  
  
  
  

add a comment | 

$begingroup$

Is the covariance matrix of a multivariate normal distribution always invertible?

normal-distribution covariance-matrix matrix linear-algebra

share|cite|improve this question

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

$endgroup$

share|cite|improve this question

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

share|cite|improve this question

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

edited yesterday

kjetil b halvorsen

31k983222

edited yesterday

kjetil b halvorsen

31k983222

asked yesterday

DadooDadoo

375

asked yesterday

DadooDadoo

375

2 Answers
2

active

oldest

votes

8

$begingroup$

If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=begin{bmatrix}sigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 end{bmatrix}$$

and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.

It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=begin{bmatrix}sigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 end{bmatrix}$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.

share|cite|improve this answer

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
  $endgroup$
  – gunes
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much for the clarification.
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  6
  

  
  
  
  
  
  

  
  $begingroup$
  This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
  $endgroup$
  – whuber♦
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
  $endgroup$
  – statmerkur
  yesterday
  
  
  

  
  
  
  

add a comment | 

4

$begingroup$

No.

The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix{1 & 1 \1 & 1}$, which is not invertible.

share|cite|improve this answer

answered yesterday

MKRMKR

1563

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395766%2finverse-of-the-covariance-matrix-of-a-multivariate-normal-distribution%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

8

$begingroup$

If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=begin{bmatrix}sigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 end{bmatrix}$$

and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.

It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=begin{bmatrix}sigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 end{bmatrix}$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.

share|cite|improve this answer

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
  $endgroup$
  – gunes
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much for the clarification.
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  6
  

  
  
  
  
  
  

  
  $begingroup$
  This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
  $endgroup$
  – whuber♦
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
  $endgroup$
  – statmerkur
  yesterday
  
  
  

  
  
  
  

add a comment | 

8

$begingroup$

If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=begin{bmatrix}sigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 end{bmatrix}$$

and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.

It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=begin{bmatrix}sigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 end{bmatrix}$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.

share|cite|improve this answer

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
  $endgroup$
  – gunes
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you very much for the clarification.
  $endgroup$
  – Dadoo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  6
  

  
  
  
  
  
  

  
  $begingroup$
  This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
  $endgroup$
  – whuber♦
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
  $endgroup$
  – statmerkur
  yesterday
  
  
  

  
  
  
  

add a comment | 

$begingroup$

If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=begin{bmatrix}sigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 end{bmatrix}$$

and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.

It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=begin{bmatrix}sigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 end{bmatrix}$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.

share|cite|improve this answer

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

$endgroup$

share|cite|improve this answer

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

edited yesterday

whuber♦

205k33448816

edited yesterday

whuber♦

205k33448816

answered yesterday

gunesgunes

5,7601115

answered yesterday

gunesgunes

5,7601115

4

$begingroup$

No.

The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix{1 & 1 \1 & 1}$, which is not invertible.

share|cite|improve this answer

answered yesterday

MKRMKR

1563

$endgroup$

add a comment | 

4

$begingroup$

No.

The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix{1 & 1 \1 & 1}$, which is not invertible.

share|cite|improve this answer

answered yesterday

MKRMKR

1563

$endgroup$

add a comment | 

$begingroup$

No.

The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix{1 & 1 \1 & 1}$, which is not invertible.

share|cite|improve this answer

answered yesterday

MKRMKR

1563

$endgroup$

share|cite|improve this answer

answered yesterday

MKRMKR

1563

answered yesterday

MKRMKR

1563

answered yesterday

MKRMKR

1563