Multivariable Analyses
Background to Questions 1 and 2
A randomised controlled trial was conducted in 800 adults to compare a new influenza vaccine to the current standard vaccine. The effect of the vaccine is measured by the within-person change in antibody titre between pre-vaccination and 30 days post-vaccination, and the log fold change is calculated by:
log fold change=log_{10}
The Excel file “Antibody data” contains the following variables:
PID: patient identifier
Treat: randomised treatment: 0=current vaccine, 1=new vaccine
Female: Sex: 0=male, 1=female
Elderly: 0=age <70 years at randomisation, 1=age ≥70 years at randomisation
Log_fold: log fold change calculated using the formula above
Question 1
- Calculate appropriate summary statistics for the variables female and elderly and describe them in sentences.
- Calculate appropriate summary statistics for log fold change both overall and by randomised treatment and describe them in sentences.
- Calculate a 95% confidence interval for the effect of the new vaccine on the mean log fold change.
- Calculate the p-value to test the null hypothesis that the new vaccine does not affect mean log fold change.
- What do you conclude about the impact of the intervention on the mean log fold change?
- Explain how the result from the confidence interval is related to the result of the hypothesis test.
- It is believed that there may be a relationship between response to vaccination, age and sex, so the investigators wish to perform adjusted analyses. Perform a further analysis to estimate the effect of the intervention adjusted for sex and explain the result. Does this alter your conclusion from the previous analysis?
- Perform another adjusted analysis to estimate the effects of new intervention, elderly status and sex on log fold change. Give the estimated effects of each predictor, their confidence intervals and p-values and interpret the levels of evidence.
- Compare the results of the univariable and two adjusted analyses. Do your conclusions about the effect of the intervention differ? Suggest a reason why this would be the case.
Question 2
A published study of a similar intervention used a dichotomous outcome to categorise each participant as a vaccine responder based on the change in antibody levels. They defined a fold change of ≥4 as a “vaccine response” and fold change of <4 as a non-response. This corresponds to log fold change of ≥0.6021 versus <0.6021.
- Generate the outcome ‘vaccine response’ for the current study and summarise it in each of the treatment groups.
- Calculate an estimate of the effect of the new vaccine on the outcome of vaccine response and give the corresponding 95% confidence interval. Interpret these results.
- Calculate the p-value for a test of the hypothesis that there is no association between the new vaccine and this outcome. Are the assumptions for this test satisfied in this dataset?
- Perform the two adjusted analyses analogous to those in Question 1 (parts 7 and 8) and interpret their results.
- Compare the results of the analyses obtained in questions 1 and 2. What do you conclude about the effect of the new vaccine on these two outcomes? Has the study demonstrated that the new vaccine is associated with increased antibody response?
Question 3
You will be given SPSS outputs for two multivariable analyses. Describe the results of the analyses in words and interpret the effects of each predictor on the outcome. The aim here is to describe the results of each analysis in a single paragraph. Please give as much information as possible to draw a conclusion about the effect of each predictor on the outcome. It should be written in a form suitable for inclusion in the “results” section of a publication.
- 3a
The following outputs from a multivariable linear regression describe the effect of randomised treatment, surgery type and age on pain scores in a clinical trial testing a new post-surgical analgesia versus placebo.
The outcome is the patient’s self-reported pain score 24 hours after surgery.
- Treat=randomised treatment: 0=placebo, 1=new analgesia (categorical predictor, reference is placebo)
- Surgery_type: 0 =laparoscopic, 1=laparotomy (categorical predictor, reference is laparoscopic)
- Age is the patient’s age (continuous predictor in years)
Describe the results of this analysis and indicate any further information you would request to help your interpretation.
- 3b
An observational study of a group of children suffering from epilepsy was conducted to examine predictors of drug-resistant epilepsy (DRE). In a multivariable logistic regression the outcome of DRE was modelled by the following predictors:
- Status_epilepticus: seizures lasting 5 minutes or more, or very close together without recovery between seizures: 1=yes, 0=no
- Age_onset: Age at onset of seizures (years)
- AED_GT_3: Use of more than three anti-epileptic drugs 1 = yes; 0 = no
Describe the results of this analysis and indicate any further information you would request to help your interpretation.
Solution
Q1.
1.
Female | |||
Frequency | Percent | ||
Male | 413 | 51.6 | |
Female | 387 | 48.4 | |
Total | 800 | 100.0 |
Elderly | |||
Frequency | Percent | ||
Age less than 70 years | 593 | 74.1 | |
Age more than or equal to 70 years | 207 | 25.9 | |
Total | 800 | 100.0 |
Out of 800 adults on which the trial was conducted, 51.6% of them are males and 48.4% of them are females. There is almost equal representation of males and females in the trial.
Out of 800 adults on which the trial was conducted, 74.1% of them are aged less than 70 years and 25.9% of them have age more than or equal to 70 years.
2.
Log fold change | |||
Statistic | |||
log_fold | Mean | .6228966 | |
95% Confidence Interval for Mean | Lower Bound | .6074754 | |
Upper Bound | .6383179 | ||
Median | .6212950 | ||
Variance | .049 | ||
Std. Deviation | .22220706 | ||
Minimum | -.22883 | ||
Maximum | 1.44180 |
Log fold change by randomised trial | ||||
Treat | Statistic | |||
log_fold | Current vaccine | Mean | .5288525 | |
95% Confidence Interval for Mean | Lower Bound | .5086295 | ||
Upper Bound | .5490754 | |||
Median | .5434700 | |||
Variance | .041 | |||
Std. Deviation | .20181615 | |||
Minimum | -.22883 | |||
Maximum | 1.08920 | |||
New vaccine | Mean | .7101424 | ||
95% Confidence Interval for Mean | Lower Bound | .6904475 | ||
Upper Bound | .7298374 | |||
Median | .6981400 | |||
Variance | .042 | |||
Std. Deviation | .20410767 | |||
Minimum | .20200 | |||
Maximum | 1.44180 |
The mean log fold change is 0.6228 with a standard deviation of 0.222
The mean log fold change is 0.52885 for the persons using current vaccine and it is 0.71014 for the persons using new vaccine.
For the persons using current vaccine, the range of log fold change is -0.2288 to 1.0892 with a median of 0.54347 whereas for the persons using new vaccine, the range of log fold change is 0.202 to 1.4418 with median of 0.69814
The log fold change seems to be higher in the persons using new vaccine as compared to the current vaccine
3.
Pairwise Comparisons | ||||||
Dependent Variable: log_fold | ||||||
(I) Treat | (J) Treat | Mean Difference (I-J) | Std. Error | Sig.^{b} | 95% Confidence Interval for Difference^{b} | |
Lower Bound | Upper Bound | |||||
Current vaccine | New vaccine | -.181^{*} | .014 | .000 | -.209 | -.153 |
New vaccine | Current vaccine | .181^{*} | .014 | .000 | .153 | .209 |
Based on estimated marginal means | ||||||
*. The mean difference is significant at the .05 level. | ||||||
b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments). |
95% confidence interval for the effect of the new vaccine on the mean log fold change is 0.153 – 0.209
4.
Test for effect of intervention | ||||||
Dependent Variable: log_fold | ||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared |
Intercept | 306.590 | 1 | 306.590 | 7439.276 | .000 | .903 |
Treat | 6.564 | 1 | 6.564 | 159.272 | .000 | .166 |
Error | 32.887 | 798 | .041 | |||
Total | 349.852 | 800 |
The p – value for testing the null hypothesis that the new vaccine does not affect mean log fold change is 2.0121 *
5.
The impact of intervention of new vaccine on the mean log fold is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.
6.
The result obtained from the hypothesis testing confirms the statistical significance of the confidence interval for the effect of the new vaccine on the mean log fold change.
The 95% confidence interval for the effect of the new vaccine on the mean log fold change does not include 0.
The 95% confidence interval for the effect of the new vaccine on the mean log fold change is 0.153 – 0.209 which means that the effect of the new vaccine is that it increases the log fold change as compared to the current vaccine and the result from the hypothesis test reveals that this effect is statistically significant.
7.
Test for effect of intervention adjusted for sex | ||||||
Dependent Variable: log_fold | ||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared |
Intercept | 305.683 | 1 | 305.683 | 7431.347 | .000 | .903 |
Treat | 6.599 | 1 | 6.599 | 160.424 | .000 | .168 |
Female | .141 | 1 | .141 | 3.424 | .065 | .004 |
Treat * Female | .006 | 1 | .006 | .140 | .709 | .000 |
Error | 32.743 | 796 | .041 | |||
Total | 349.852 | 800 |
When the test for the effect of intervention is performed adjusted for sex, the effect due to sex and the interaction effect between sex and treatment on the log fold change are found to be statistically insignificant.
The p -value for the effect of the intervention on the log fold change is 1.2616 *
Thus, this does not alter the result from previous analysis and the effect of the intervention on the log fold change adjusted for sex is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.
8.
Test for effect of intervention adjusted for sex and elderly status | ||||||
Dependent Variable: log_fold | ||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared |
Intercept | 211.875 | 1 | 211.875 | 5491.226 | .000 | .874 |
Treat | 4.210 | 1 | 4.210 | 109.120 | .000 | .121 |
Female | .066 | 1 | .066 | 1.720 | .190 | .002 |
Elderly | 2.025 | 1 | 2.025 | 52.476 | .000 | .062 |
Treat * Female | .049 | 1 | .049 | 1.281 | .258 | .002 |
Treat * Elderly | .069 | 1 | .069 | 1.779 | .183 | .002 |
Female * Elderly | .009 | 1 | .009 | .232 | .630 | .000 |
Treat * Female * Elderly | .083 | 1 | .083 | 2.148 | .143 | .003 |
Error | 30.559 | 792 | .039 | |||
Total | 349.852 | 800 |
Parameter Estimates | |||||
Dependent Variable: log_fold | |||||
Parameter | B | Sig. | 95% Confidence Interval | ||
Lower Bound | Upper Bound | ||||
Intercept | .613 | .000 | .562 | .663 | |
[Treat=0] | -.187 | .000 | -.259 | -.115 | |
[Female=0] | -.013 | .744 | -.090 | .064 | |
[Elderly=0] | .121 | .000 | .061 | .181 |
The p – value for the test of effect of intervention on the mean log fold change is 5.1 *
The effect of intervention on the mean log fold change is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.
9.
The p – value for the tests of effect of intervention on the mean log fold change increases when it is adjusted for the factor elderly but in all the 3 cases, the p – value for the test of effect of intervention on the mean log fold change is extremely low and hence, the effect of intervention on the mean fold change is statistically significant in the univariable as well as the two adjusted analysis and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.
Q2.
1.
vaccine_response | |||
Frequency | Percent | ||
Non response | 367 | 45.9 | |
Vaccine response | 433 | 54.1 | |
Total | 800 | 100.0 |
vaccine_response * Treat Crosstabulation | ||||
Count | ||||
Treat | Total | |||
Current vaccine | New vaccine | |||
vaccine_response | Non response | 245 | 122 | 367 |
Vaccine response | 140 | 293 | 433 | |
Total | 385 | 415 | 800 |
2.
Testing the effect of intervention on vaccine response | |||||||||
B | Wald | df | Sig. | Exp(B) | 95% C.I.for EXP(B) | ||||
Lower | Upper | ||||||||
Step 1^{a} | Treat | 1.436 | .151 | 90.278 | 1 | .000 | 4.203 | 3.126 | 5.652 |
Constant | -.560 | .106 | 27.901 | 1 | .000 | .571 | |||
a. Variable(s) entered on step 1: Treat. |
An estimate of the effect of the new vaccine on the outcome of vaccine response is 4.203 which means that the odds of having vaccine response is 4.203 times greater for new vaccine as opposed to the current vaccine and the corresponding 95% confidence interval is 3.126 – 5.652
3.
The p-value for a test of the hypothesis that there is no association between the new vaccine and this outcome is 2.069 * which means that there is statistically significant association between the new vaccine and this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine. The assumptions of this test are met in this data set.
4.
Testing effect of intervention adjusted for sex on vaccine response | ||||||||
B | Wald | df | Sig. | Exp(B) | 95% C.I.for EXP(B) | |||
Lower | Upper | |||||||
Step 1^{a} | Treat | 1.441 | 90.616 | 1 | .000 | 4.226 | 3.141 | 5.685 |
Female | -.173 | 1.310 | 1 | .252 | .841 | .625 | 1.131 | |
Constant | -.478 | 14.145 | 1 | .000 | .620 | |||
a. Variable(s) entered on step 1: Treat, Female. |
The odds of having vaccine response is 4.226 times greater for new vaccine as opposed to the current vaccine when adjusted for sex and the corresponding 95% confidence interval is 3.141 – 5.685
The p-value for the effect of new vaccine on this outcome is 1.7445 * which means that there is statistically significant effect of new vaccine on this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.
Testing effect of treatment adjusted for sex and elderly on vaccine response | ||||||||
B | Wald | df | Sig. | Exp(B) | 95% C.I.for EXP(B) | |||
Lower | Upper | |||||||
Step 1^{a} | Treat | 1.488 | 90.441 | 1 | .000 | 4.429 | 3.259 | 6.019 |
Female | -.098 | .395 | 1 | .530 | .907 | .668 | 1.230 | |
Elderly | -1.091 | 36.553 | 1 | .000 | .336 | .236 | .479 | |
Constant | -.255 | 3.667 | 1 | .056 | .775 | |||
a. Variable(s) entered on step 1: Treat, Female, Elderly. | ||||||||
The odds of having vaccine response is 4.429 times greater for new vaccine as opposed to the current vaccine when adjusted for sex and the corresponding 95% confidence interval is 3.259 – 6.019
The p-value for the effect of new vaccine on this outcome is 1.905 * which means that there is statistically significant effect of new vaccine on this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.
5.
From the results of the analysis of both the questions, it is evidently clear that there is statistically significant effect of intervention of new vaccine on the log folds change and the vaccine response variable generated as a dichotomous measure of the log folds change (log fold change of ≥0.6021 as a “vaccine response” and fold change of <0.6021 as a “non-response”) with a very low p-value for all the tests of hypothesis whether unadjusted or adjusted for sex and elderly in both the cases.
This means that the log folds change and vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.
Thus, the study has clearly demonstrated that the new vaccine is associated with increased antibody response.
Q3.
(a)
The patient’s self-reported pain score after 24 hours of surgery can be expressed as:
Pain = 24.339 + 1.573 * I(surgery type = laparotomy) – 1.593 * I(treatment = new analgesia) + 0.072 * Age.
The mean pain score is 1.593 less if the treatment given to the patient is new analgesia as compared to placebo and the effect of this treatment on the pain score is statistically significant.
The mean pain score is 1.573 more if the patient has received a surgery of type laparotomy as compared to a patient who has received a surgery of type laparoscopic and the effect of the type of the surgery on the pain score of the patient is statistically significant.
The mean pain score increases 0.072 for an increase in the age in the age of the patient by 1 year but this effect of the age on the pain score of the patient is not statistically significant.
To help the interpretation of this regression model, I would request to know the adjusted R-squared value of the regression model and the estimates of effect size.
(b)
The logistic regression model is performed to examine the predictors of DRE.
The drug resistant epilepsy DRE was found to be 4.533 times higher in case of seizures lasting 5 minutes or more than compared to the other case and this predictor is statistically significant in predicting DRE (p-value = 0.043). The DRE was found to be 2.351 times higher if there has been a use of more than three anti-epileptic drug and this predictor is not statistically significant in predicting DRE (p-value = 0.264). The DRE is found to be decreasing with an increase in the age of the onset of seizures but this predictor is not statistically significant in predicting the DRE (p-value = 0.253).
To help the interpretation of this model, I would request to know the Cox & Snell R Square and Nagelkerke R Square values, which are both methods of calculating the explained variation of the model and the classification table.