Multivariable Analyses

Multivariable Analyses

Background to Questions 1 and 2

A randomised controlled trial was conducted in 800 adults to compare a new influenza vaccine to the current standard vaccine. The effect of the vaccine is measured by the within-person change in antibody titre between pre-vaccination and 30 days post-vaccination, and the log fold change is calculated by:

log fold change=log10 

The Excel file “Antibody data” contains the following variables:

PID: patient identifier

Treat: randomised treatment: 0=current vaccine, 1=new vaccine

Female: Sex: 0=male, 1=female

Elderly: 0=age <70 years at randomisation, 1=age ≥70 years at randomisation

Log_fold: log fold change calculated using the formula above 

Question 1

  1. Calculate appropriate summary statistics for the variables female and elderly and describe them in sentences.
  2. Calculate appropriate summary statistics for log fold change both overall and by randomised treatment and describe them in sentences.
  3. Calculate a 95% confidence interval for the effect of the new vaccine on the mean log fold change.
  4. Calculate the p-value to test the null hypothesis that the new vaccine does not affect mean log fold change.
  5. What do you conclude about the impact of the intervention on the mean log fold change?
  6. Explain how the result from the confidence interval is related to the result of the hypothesis test.
  7. It is believed that there may be a relationship between response to vaccination, age and sex, so the investigators wish to perform adjusted analyses. Perform a further analysis to estimate the effect of the intervention adjusted for sex and explain the result. Does this alter your conclusion from the previous analysis?
  8. Perform another adjusted analysis to estimate the effects of new intervention, elderly status and sex on log fold change. Give the estimated effects of each predictor, their confidence intervals and p-values and interpret the levels of evidence.
  9. Compare the results of the univariable and two adjusted analyses. Do your conclusions about the effect of the intervention differ? Suggest a reason why this would be the case. 

Question 2

A published study of a similar intervention used a dichotomous outcome to categorise each participant as a vaccine responder based on the change in antibody levels. They defined a fold change of ≥4 as a “vaccine response” and fold change of <4 as a non-response. This corresponds to log fold change of ≥0.6021 versus <0.6021.

  1. Generate the outcome ‘vaccine response’ for the current study and summarise it in each of the treatment groups.
  2. Calculate an estimate of the effect of the new vaccine on the outcome of vaccine response and give the corresponding 95% confidence interval. Interpret these results.
  3. Calculate the p-value for a test of the hypothesis that there is no association between the new vaccine and this outcome. Are the assumptions for this test satisfied in this dataset?
  4. Perform the two adjusted analyses analogous to those in Question 1 (parts 7 and 8) and interpret their results.
  5. Compare the results of the analyses obtained in questions 1 and 2. What do you conclude about the effect of the new vaccine on these two outcomes? Has the study demonstrated that the new vaccine is associated with increased antibody response? 

Question 3

You will be given SPSS outputs for two multivariable analyses. Describe the results of the analyses in words and interpret the effects of each predictor on the outcome. The aim here is to describe the results of each analysis in a single paragraph. Please give as much information as possible to draw a conclusion about the effect of each predictor on the outcome. It should be written in a form suitable for inclusion in the “results” section of a publication. 

  1. 3a

The following outputs from a multivariable linear regression describe the effect of randomised treatment, surgery type and age on pain scores in a clinical trial testing a new post-surgical analgesia versus placebo.

The outcome is the patient’s self-reported pain score 24 hours after surgery.

  • Treat=randomised treatment: 0=placebo, 1=new analgesia (categorical predictor, reference is placebo)
  • Surgery_type: 0 =laparoscopic, 1=laparotomy (categorical predictor, reference is laparoscopic)
  • Age is the patient’s age (continuous predictor in years)

Describe the results of this analysis and indicate any further information you would request to help your interpretation. 

  1. 3b

An observational study of a group of children suffering from epilepsy was conducted to examine predictors of drug-resistant epilepsy (DRE). In a multivariable logistic regression the outcome of DRE was modelled by the following predictors:

  • Status_epilepticus: seizures lasting 5 minutes or more, or very close together without recovery between seizures: 1=yes, 0=no
  • Age_onset: Age at onset of seizures (years)
  • AED_GT_3: Use of more than three anti-epileptic drugs 1 = yes; 0 = no

Describe the results of this analysis and indicate any further information you would request to help your interpretation. 

Solution 

Q1.

1.

Female
Frequency Percent
Male 413 51.6
Female 387 48.4
Total 800 100.0
Elderly
Frequency Percent
Age less than 70 years 593 74.1
Age more than or equal to 70 years 207 25.9
Total 800 100.0

Out of 800 adults on which the trial was conducted, 51.6% of them are males and 48.4% of them are females. There is almost equal representation of males and females in the trial.

Out of 800 adults on which the trial was conducted, 74.1% of them are aged less than 70 years and 25.9% of them have age more than or equal to 70 years.

2.

Log fold change
Statistic
log_fold Mean .6228966
95% Confidence Interval for Mean Lower Bound .6074754
Upper Bound .6383179
Median .6212950
Variance .049
Std. Deviation .22220706
Minimum -.22883
Maximum 1.44180
      Log fold change by randomised trial
Treat Statistic
log_fold Current vaccine Mean .5288525
95% Confidence Interval for Mean Lower Bound .5086295
Upper Bound .5490754
Median .5434700
Variance .041
Std. Deviation .20181615
Minimum -.22883
Maximum 1.08920
New vaccine Mean .7101424
95% Confidence Interval for Mean Lower Bound .6904475
Upper Bound .7298374
Median .6981400
Variance .042
Std. Deviation .20410767
Minimum .20200
Maximum 1.44180

The mean log fold change is 0.6228 with a standard deviation of 0.222

The mean log fold change is 0.52885 for the persons using current vaccine and it is 0.71014 for the persons using new vaccine.

For the persons using current vaccine, the range of log fold change is -0.2288 to 1.0892 with a median of 0.54347 whereas for the persons using new vaccine, the range of log fold change is 0.202 to 1.4418 with median of 0.69814

The log fold change seems to be higher in the persons using new vaccine as compared to the current vaccine

3.

Pairwise Comparisons
Dependent Variable:   log_fold
(I) Treat (J) Treat Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb
Lower Bound Upper Bound
Current vaccine New vaccine -.181* .014 .000 -.209 -.153
New vaccine Current vaccine .181* .014 .000 .153 .209
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).

95% confidence interval for the effect of the new vaccine on the mean log fold change is      0.153 – 0.209

4.

Test for effect of intervention
Dependent Variable:   log_fold
Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared
Intercept 306.590 1 306.590 7439.276 .000 .903
Treat 6.564 1 6.564 159.272 .000 .166
Error 32.887 798 .041
Total 349.852 800

The p – value for testing the null hypothesis that the new vaccine does not affect mean log fold change is 2.0121 *

5.

The impact of intervention of new vaccine on the mean log fold is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.

6.

The result obtained from the hypothesis testing confirms the statistical significance of the confidence interval for the effect of the new vaccine on the mean log fold change.

The 95% confidence interval for the effect of the new vaccine on the mean log fold change does not include 0.

The 95% confidence interval for the effect of the new vaccine on the mean log fold change is 0.153 – 0.209 which means that the effect of the new vaccine is that it increases the log fold change as compared to the current vaccine and the result from the hypothesis test reveals that this effect is statistically significant.

7.

Test for effect of intervention adjusted for sex
Dependent Variable:   log_fold
Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared
Intercept 305.683 1 305.683 7431.347 .000 .903
Treat 6.599 1 6.599 160.424 .000 .168
Female .141 1 .141 3.424 .065 .004
Treat * Female .006 1 .006 .140 .709 .000
Error 32.743 796 .041
Total 349.852 800

When the test for the effect of intervention is performed adjusted for sex, the effect due to sex and the interaction effect between sex and treatment on the log fold change are found to be statistically insignificant.

The p -value for the effect of the intervention on the log fold change is 1.2616 *

Thus, this does not alter the result from previous analysis and the effect of the intervention on the log fold change adjusted for sex is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.

8.

Test for effect of intervention adjusted for sex and elderly status
Dependent Variable:   log_fold
Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared
Intercept 211.875 1 211.875 5491.226 .000 .874
Treat 4.210 1 4.210 109.120 .000 .121
Female .066 1 .066 1.720 .190 .002
Elderly 2.025 1 2.025 52.476 .000 .062
Treat * Female .049 1 .049 1.281 .258 .002
Treat * Elderly .069 1 .069 1.779 .183 .002
Female * Elderly .009 1 .009 .232 .630 .000
Treat * Female * Elderly .083 1 .083 2.148 .143 .003
Error 30.559 792 .039
Total 349.852 800
Parameter Estimates
Dependent Variable:   log_fold
Parameter B Sig. 95% Confidence Interval
Lower Bound Upper Bound
Intercept .613 .000 .562 .663
[Treat=0] -.187 .000 -.259 -.115
[Female=0] -.013 .744 -.090 .064
[Elderly=0] .121 .000 .061 .181

The p – value for the test of effect of intervention on the mean log fold change is 5.1 *

The effect of intervention on the mean log fold change is statistically significant and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.

9.

The p – value for the tests of effect of intervention on the mean log fold change increases when it is adjusted for the factor elderly but in all the 3 cases, the p – value for the test of effect of intervention on the mean log fold change is extremely low and hence, the effect of intervention on the mean fold change is statistically significant in the univariable as well as the two adjusted analysis and the log fold change is statistically significantly higher in case of using new vaccine as compared to the current vaccine.

Q2.

1.

vaccine_response
Frequency Percent
Non response 367 45.9
Vaccine response 433 54.1
Total 800 100.0
vaccine_response * Treat Crosstabulation
Count
Treat Total
Current vaccine New vaccine
vaccine_response Non response 245 122 367
Vaccine response 140 293 433
Total 385 415 800

2.

Testing the effect of intervention on vaccine response
B Wald df Sig. Exp(B) 95% C.I.for EXP(B)
Lower Upper
Step 1a Treat 1.436 .151 90.278 1 .000 4.203 3.126 5.652
Constant -.560 .106 27.901 1 .000 .571
a. Variable(s) entered on step 1: Treat.

An estimate of the effect of the new vaccine on the outcome of vaccine response is 4.203 which means that the odds of having vaccine response is 4.203 times greater for new vaccine as opposed to the current vaccine and the corresponding 95% confidence interval is 3.126 – 5.652

3.

The p-value for a test of the hypothesis that there is no association between the new vaccine and this outcome is 2.069 *  which means that there is statistically significant association between the new vaccine and this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine. The assumptions of this test are met in this data set.

4.

Testing effect of intervention adjusted for sex on vaccine response
B Wald df Sig. Exp(B) 95% C.I.for EXP(B)
Lower Upper
Step 1a Treat 1.441 90.616 1 .000 4.226 3.141 5.685
Female -.173 1.310 1 .252 .841 .625 1.131
Constant -.478 14.145 1 .000 .620
a. Variable(s) entered on step 1: Treat, Female.

The odds of having vaccine response is 4.226 times greater for new vaccine as opposed to the current vaccine when adjusted for sex and the corresponding 95% confidence interval is 3.141 – 5.685

The p-value for the effect of new vaccine on this outcome is 1.7445 *  which means that there is statistically significant effect of new vaccine on this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.

Testing effect of treatment adjusted for sex and elderly on vaccine response
B Wald df Sig. Exp(B) 95% C.I.for EXP(B)
Lower Upper
Step 1a Treat 1.488 90.441 1 .000 4.429 3.259 6.019
Female -.098 .395 1 .530 .907 .668 1.230
Elderly -1.091 36.553 1 .000 .336 .236 .479
Constant -.255 3.667 1 .056 .775
a. Variable(s) entered on step 1: Treat, Female, Elderly.

The odds of having vaccine response is 4.429 times greater for new vaccine as opposed to the current vaccine when adjusted for sex and the corresponding 95% confidence interval is 3.259 – 6.019

The p-value for the effect of new vaccine on this outcome is 1.905 *  which means that there is statistically significant effect of new vaccine on this vaccine response outcome. The vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.

5.

From the results of the analysis of both the questions, it is evidently clear that there is statistically significant effect of intervention of new vaccine on the log folds change and the vaccine response variable generated as a dichotomous measure of the log folds change (log fold change of ≥0.6021 as a “vaccine response” and fold change of <0.6021 as a “non-response”) with a very low p-value for all the tests of hypothesis whether unadjusted or adjusted for sex and elderly in both the cases.

This means that the log folds change and vaccine response will be statistically significantly higher in the case of new vaccine as compared to the current vaccine.

Thus, the study has clearly demonstrated that the new vaccine is associated with increased antibody response.

Q3.

(a)

The patient’s self-reported pain score after 24 hours of surgery can be expressed as:

Pain = 24.339 + 1.573 * I(surgery type = laparotomy) – 1.593 * I(treatment = new analgesia) + 0.072 * Age.

The mean pain score is 1.593 less if the treatment given to the patient is new analgesia as compared to placebo and the effect of this treatment on the pain score is statistically significant.

The mean pain score is 1.573 more if the patient has received a surgery of type laparotomy as compared to a patient who has received a surgery of type laparoscopic and the effect of the type of the surgery on the pain score of the patient is statistically significant.

The mean pain score increases 0.072 for an increase in the age in the age of the patient by 1 year but this effect of the age on the pain score of the patient is not statistically significant.

To help the interpretation of this regression model, I would request to know the adjusted R-squared value of the regression model and the estimates of effect size.

(b)

The logistic regression model is performed to examine the predictors of DRE.

The drug resistant epilepsy DRE was found to be 4.533 times higher in case of seizures lasting 5 minutes or more than compared to the other case and this predictor is statistically significant in predicting DRE (p-value = 0.043). The DRE was found to be 2.351 times higher if there has been a use of more than three anti-epileptic drug and this predictor is not statistically significant in predicting DRE (p-value = 0.264). The DRE is found to be decreasing with an increase in the age of the onset of seizures but this predictor is not statistically significant in predicting the DRE (p-value = 0.253).

To help the interpretation of this model, I would request to know the Cox & Snell R Square and Nagelkerke R Square values, which are both methods of calculating the explained variation of the model and the classification table.