Hi Mike In the lecture on ANCOVA, we covered the theory on LSMEANS for the post hoc testing. What we did not do is the actual post hoc testing itself. Please could you give us some idea of how to do this - i.e. which table we get our stats from and the post hoc testing step. The reason we do not have this is that, although we covered ANCOVA in one of the labs, we do not have the tables and results for that tut so we cannot work out the procedure from there. Thanks! Laura
The procedure isdentical to that of 2 way anova and the tables look the same. You set out your hypotheses as usual noting that the result is after accounting for the covariate. The table will have the LSMEANS lidsted with a number and the pvalues in a matrix. i.e. to compare group one to group 4 you find the p-value in the 1st row 4th column or 4throw first column.
From email 1. In the 2007 exam, I am having problems with Question 4 and don't know how to go about it. I suspect its a one-way ANOVA with several covariates i.e. supervisor support, organisation support, contract violation, and we are interested in tenure vs contract difference? You are correct.
s the Chi-Square box, Box's M statistic? Yes, it comes under the heading test of homogeneity of covariances. and then for the F value which statistic do we use - Wilks Lambda etc? as noted in class it doesn’t generally matter but for consistency we will use wilk’s lambda. and for effect size, where do we get X1-X2 from? In general you are not always given all of the info. In that instance you need to state what else needs to be done and what info must be presented. As an aside the R2 in the table is the same as eta sq for anova.
3. Do we need to know how to do a stepwise forward, backward regression? And if so, can you give an example of how to do it? You need to know these conceptually, but are unlikely to encounter them as such. In any case the procedure is identical to any other regression. The only new table is the one before the final regression starts which summarise which variables have gone in and out of the regression (see tut 4.) After that you simply have the ordinary regression results for the selected model.
4. In the test we did in April, Question 1 D - only Type III sum of squares was presented, is this appropriate and why? Do we always need to get Type I and Type III to see if its balanced or not? and it is only in ANCOVA that we use Type I ? No, balanced means that all groups have the same number of subjects. You could be told that or have a table which indicates it. For this course the rule is sound, but given the reasons we discussed for using type 1 in ancova you may be able to see that the choice could be appropriate in some way for anova.
5. How do we classify the design of the tutorial we did on the bakeries? Do we say its a two-way ANOVA with ANCOVA or do we just say ANCOVA because they are only interested in finding out the results when you take out size of bakery as a covariate? 2 Way ANCOVA
DV Hours lost IVs Train and BreadOnly Covariate Bakeries.
Note the labelling is define as (change anova to ancova in the anova definition of what it would be if there was no covariate)
The answer is not a simple set of rules but the following principal apply. 1. The correlation is the default and will be used if strong motivation for a covariance matrix is not evident. 2. To that end to use a covariance matrix the following must be true a) All variables/items must be on the same scale. b) Some relevance must be presence in the size of the variance. That is a variable/item with larger variance must be considered more important than one with small variance.
if a and b are true an argument can be made for the covariance matrix.
1. Formula for Cohens D (paired t-test): where do you get the s1 and s2 values from? These would be given, they are the standard deviations of the two variables. They are typically not shown in the printout for a paired t-test and as such would have to be in a separate printout. It is assume that you do not need special help to find basic descriptive statistic and they could be presented in any format. 2. Effect size - independent t-test: where do you get these values from? They are all in the table at the top with descriptive statistics. See note above. 3. T-tests: If variance is not equal, do we assume it is and carry on or do we stop? NO. Use the appropriate test for unequal variances, which is part of the printout (The one labelled unequal/Satterthwaite.) 4. Effect size - 1 sample t-test: where do you get sample std dev (s) value from? From the mean table see note above. 5. 1-way Anova: If we reject equality of variance, do we do a non-parametric test or assume equality of variance and carry on with parametric test?
To some extent this will depend on the way the question is phrased. In general if there is a problem with assumptions you carry on regardless and note the problem. However for such simple procedures questions have been asked in such a way as to require a choice between one or the other analysis. In this instance it would be presented as "which of these analyses are most appropriate - justify your choice.)
: For the exemplar - how do we interpret the means in bonferronis, tukeys and fishers tests? Means are means for the most part irrepsective of the adjustment to the hypothesis test.
: If fail to reject main F-statistic, we dont do post-hoc but do we still do effect size even though its not significant?
As noted in the lectures we generally do effect sizes for significant effects but there are sensible reasons to do them for non signioficant effects. In the instance above you may look at eta squared for exampe but not the cohens d values.
6. Is confidence interval only used for the t-tests?
Confidence intervals appear in many situations are are always interpreted in the same way.
7. 2-way/n-way Anova: When do we do post-hoc? (If only the interaction is significant or no matter what the main effect is?)
Post hoc test are done for significant effects. Hoever if there is an interaction effect this supercedes the effects for the main effects that are part of that interaction effect.
8. Ancova: 8.1 Do we wish to remove the effect of the extraneous variables only or of moderator and mediator variables? Anocova is appropriate for Extraneous and mediator variables. Moderator variables are examine dby means of Anova's (Interactions) or cross product terms in moderated multiple regression.
In the case of mediators we expect to see no relationship with the IV and the DV after the mediator effect is accounted for.
8.2 Are the assumptions of Ancova the same as Anova with the addition of these 3: 1) IVs shouldnt afffect covariate; 2) regression slopes; 3) linearity Not sure what you mean by point 1. P
Points 2 and 3 are the same. So yes but only tha the covariats are linearly related to the DV's
8.3 Why do we use Type 1 in Ancova? Because Type III in effect undoes the muticollinearity effect which is the whole point of Ancova. (This is one of those bits where I've drastically simplified the truth as sas handles covariates in quite a complex way.) 8.4 What do we do after the post-hoc tests? Depends what you've done before. In essence you do in ancova everything you would do in an anova. 9. Simple regression: 9.1 Does parameter estimates come at the end and what is the significance of it? You look at the parameter estimates to establish the direction of the relationship. 9.2 What is X in the equation?
Have to see the equation but presuming
Y = Beta0 + Beta1 x X + epsilon
X here is the independent variable. 10. Multiple regression: 10.1 When do we carry on to multicollinearity? You always do multicollineary. 10.2 Do we do stepwise etc. or just state what steps we will use? Stepwise will either be done for you or it would more likely be some you would suggest if appropriate. 10.3 If we do stepwise etc, how do we do it?! The computer will already have done it.
Most members of the class feel that their notes are very haphazard and messy resulting in confusing steps for some of the procedures. Is it at all possible that we can go through the steps of each hypothesis test on Tuesday? We realise that this may take a long time so if you would prefer not to could you post some general notes on each test on the blog?
Many thanks Sherianne (on behalf of the Honours class)
The exams question blog wont let me in so I just decided to e-mail you. I have been going over multivariate analysis and I have a few questions:
-If a 1 sample Hotellings T squared is significant do we do post-hoc tests with 1 sample t tests to see which variables within the vector are significant? Yes
-Do we need to consider liberal/conservative tests with regards to the post hoc tests of multivariate analyses? Again, one of those bonus thing. Not covered in the method.
-If we are not given LS mean tables (Anova) or confidence intervals (t-tests) for the univariate test statistics how do we see the directions for all the significant post-hoc tests? You can’t. will suggest that they are needed.
-Do we only do effect sizes for the significant post-hoc tests or ALL the post-hoc tests for multivariate analysis? Again this is not a yes/no answer. As will all effect size measures, there are questions you can ask both ways, but typically you would focus on significant differences. -You told us to always use a Mu with a tilder for the hypothesis tests/population means on multivariate analyses. You did not in class today but we are supposed to...right?
Yeah, as I said in class notation in this field is notoriously diverse and sometimes I forget my own rules.
12 comments:
Hi Mike
In the lecture on ANCOVA, we covered the theory on LSMEANS for the post hoc testing. What we did not do is the actual post hoc testing itself. Please could you give us some idea of how to do this - i.e. which table we get our stats from and the post hoc testing step. The reason we do not have this is that, although we covered ANCOVA in one of the labs, we do not have the tables and results for that tut so we cannot work out the procedure from there. Thanks!
Laura
Hi Laura,
The procedure isdentical to that of 2 way anova and the tables look the same. You set out your hypotheses as usual noting that the result is after accounting for the covariate. The table will have the LSMEANS lidsted with a number and the pvalues in a matrix. i.e. to compare group one to group 4 you find the p-value in the 1st row 4th column or 4throw first column.
From email
1. In the 2007 exam, I am having problems with Question 4 and don't know how to go about it. I suspect its a one-way ANOVA with several covariates i.e. supervisor support, organisation support, contract violation, and we are interested in tenure vs contract difference?
You are correct.
s the Chi-Square box, Box's M statistic? Yes, it comes under the heading test of homogeneity of covariances.
and then for the F value which statistic do we use - Wilks Lambda etc? as noted in class it doesn’t generally matter but for consistency we will use wilk’s lambda.
and for effect size, where do we get X1-X2 from?
In general you are not always given all of the info. In that instance you need to state what else needs to be done and what info must be presented. As an aside the R2 in the table is the same as eta sq for anova.
3. Do we need to know how to do a stepwise forward, backward regression? And if so, can you give an example of how to do it?
You need to know these conceptually, but are unlikely to encounter them as such. In any case the procedure is identical to any other regression. The only new table is the one before the final regression starts which summarise which variables have gone in and out of the regression (see tut 4.) After that you simply have the ordinary regression results for the selected model.
4. In the test we did in April, Question 1 D - only Type III sum of squares was presented, is this appropriate and why? Do we always need to get Type I and Type III to see if its balanced or not? and it is only in ANCOVA that we use Type I ?
No, balanced means that all groups have the same number of subjects. You could be told that or have a table which indicates it.
For this course the rule is sound, but given the reasons we discussed for using type 1 in ancova you may be able to see that the choice could be appropriate in some way for anova.
5. How do we classify the design of the tutorial we did on the bakeries? Do we say its a two-way ANOVA with ANCOVA or do we just say ANCOVA because they are only interested in finding out the results when you take out size of bakery as a covariate?
2 Way ANCOVA
DV Hours lost
IVs Train and BreadOnly
Covariate Bakeries.
Note the labelling is define as (change anova to ancova in the anova definition of what it would be if there was no covariate)
Hi Mike,
Regarding factor analysis, please can you explain when we use the covariance as against the correlation matrix.
Many thanks, Dorit
To Dorit,
The answer is not a simple set of rules but the following principal apply.
1. The correlation is the default and will be used if strong motivation for a covariance matrix is not evident.
2. To that end to use a covariance matrix the following must be true
a) All variables/items must be on the same scale.
b) Some relevance must be presence in the size of the variance. That is a variable/item with larger variance must be considered more important than one with small variance.
if a and b are true an argument can be made for the covariance matrix.
From Email
1. Formula for Cohens D (paired t-test): where do you get the s1 and s2 values from?
These would be given, they are the standard deviations of the two variables. They are typically not shown in the printout for a paired t-test and as such would have to be in a separate printout. It is assume that you do not need special help to find basic descriptive statistic and they could be presented in any format.
2. Effect size - independent t-test: where do you get these values from?
They are all in the table at the top with descriptive statistics. See note above.
3. T-tests: If variance is not equal, do we assume it is and carry on or do we stop?
NO. Use the appropriate test for unequal variances, which is part of the printout (The one labelled unequal/Satterthwaite.)
4. Effect size - 1 sample t-test: where do you get sample std dev (s) value from?
From the mean table see note above.
5. 1-way Anova: If we reject equality of variance, do we do a non-parametric test or assume equality of variance and carry on with parametric test?
To some extent this will depend on the way the question is phrased. In general if there is a problem with assumptions you carry on regardless and note the problem. However for such simple procedures questions have been asked in such a way as to require a choice between one or the other analysis. In this instance it would be presented as "which of these analyses are most appropriate - justify your choice.)
: For the exemplar - how do we interpret the means in bonferronis, tukeys and fishers tests?
Means are means for the most part irrepsective of the adjustment to the hypothesis test.
: If fail to reject main F-statistic, we dont do post-hoc but do we still do effect size even though its not significant?
As noted in the lectures we generally do effect sizes for significant effects but there are sensible reasons to do them for non signioficant effects. In the instance above you may look at eta squared for exampe but not the cohens d values.
6. Is confidence interval only used for the t-tests?
Confidence intervals appear in many situations are are always interpreted in the same way.
7. 2-way/n-way Anova: When do we do post-hoc? (If only the interaction is significant or no matter what the main effect is?)
Post hoc test are done for significant effects. Hoever if there is an interaction effect this supercedes the effects for the main effects that are part of that interaction effect.
8. Ancova:
8.1 Do we wish to remove the effect of the extraneous variables only or of moderator and mediator variables?
Anocova is appropriate for Extraneous and mediator variables. Moderator variables are examine dby means of Anova's (Interactions) or cross product terms in moderated multiple regression.
In the case of mediators we expect to see no relationship with the IV and the DV after the mediator effect is accounted for.
8.2 Are the assumptions of Ancova the same as Anova with the addition of these 3:
1) IVs shouldnt afffect covariate; 2) regression slopes; 3) linearity
Not sure what you mean by point 1. P
Points 2 and 3 are the same. So yes but only tha the covariats are linearly related to the DV's
8.3 Why do we use Type 1 in Ancova?
Because Type III in effect undoes the muticollinearity effect which is the whole point of Ancova. (This is one of those bits where I've drastically simplified the truth as sas handles covariates in quite a complex way.)
8.4 What do we do after the post-hoc tests?
Depends what you've done before. In essence you do in ancova everything you would do in an anova.
9. Simple regression:
9.1 Does parameter estimates come at the end and what is the significance of it?
You look at the parameter estimates to establish the direction of the relationship.
9.2 What is X in the equation?
Have to see the equation but presuming
Y = Beta0 + Beta1 x X + epsilon
X here is the independent variable.
10. Multiple regression:
10.1 When do we carry on to multicollinearity?
You always do multicollineary.
10.2 Do we do stepwise etc. or just state what steps we will use?
Stepwise will either be done for you or it would more likely be some you would suggest if appropriate.
10.3 If we do stepwise etc, how do we do it?!
The computer will already have done it.
Hi Mike,
Regarding Factor Analysis, what is the cut pff point to see whether a factor loads or not? Is it 5.0 or 4.0?
Sorry not a factor, to see if a PC loads onto a factor..
Thanks
Most correctly it would be items/variables loading onto a factor. The guideline is 0.4. Cutoff is perhaps to strong a word.
Mike
Hi Mike,
Most members of the class feel that their notes are very haphazard and messy resulting in confusing steps for some of the procedures. Is it at all possible that we can go through the steps of each hypothesis test on Tuesday? We realise that this may take a long time so if you would prefer not to could you post some general notes on each test on the blog?
Many thanks
Sherianne (on behalf of the Honours class)
I will provide an example of each significant procedure. This should be enough to address your concerns.
The exams question blog wont let me in so I just decided to e-mail you. I have been going over multivariate analysis and I have a few questions:
-If a 1 sample Hotellings T squared is significant do we do post-hoc tests with 1 sample t tests to see which variables within the vector are significant?
Yes
-Do we need to consider liberal/conservative tests with regards to the post hoc tests of multivariate analyses?
Again, one of those bonus thing. Not covered in the method.
-If we are not given LS mean tables (Anova) or confidence intervals (t-tests) for the univariate test statistics how do we see the directions for all the significant post-hoc tests?
You can’t. will suggest that they are needed.
-Do we only do effect sizes for the significant post-hoc tests or ALL the post-hoc tests for multivariate analysis?
Again this is not a yes/no answer. As will all effect size measures, there are questions you can ask both ways, but typically you would focus on significant differences.
-You told us to always use a Mu with a tilder for the hypothesis tests/population means on multivariate analyses. You did not in class today but we are supposed to...right?
Yeah, as I said in class notation in this field is notoriously diverse and sometimes I forget my own rules.
Post a Comment