Downstream methods
Many packages provide tools for downstream processing — e.g. plot
diagnostics, model comparisons, visual plotting, and regression tables –
that are compatible with merMod objects. This vignette
provides instructions on some recommended packages with examples. Rather
than attempting an exhaustive survey, We aimed to include the more
popular packages.
Diagnostic Plots
While plot(merMod_object) provides some diagnostic plots
(see ?lme4::plot.merMod), other packages provide more
complete functionality.
This section emphasizes the performance and
DHARMa package.
library(lme4)
# Example of a linear mixed-effects model (LMM)
mod_ss <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
# Assume a non-mixed model for comparison
mod_ss2 <- lm(Reaction ~ Days, sleepstudy)
# Another (LMM) example, and others for comparison
mod_cw = lmer(weight ~ Time + Diet + (1|Chick), data = ChickWeight, REML = FALSE)
mod_cw2 = lm(weight ~ Time + Diet, data = ChickWeight)
mod_cw3 = lmer(weight ~ Time + (1|Chick), data = ChickWeight, REML = FALSE)
# Example of a Poisson generalized linear mixed-effects model (GLMM)
form <- TICKS~YEAR+HEIGHT+(1|BROOD)+(1|INDEX)+(1|LOCATION)
mod_gt <- glmer(form, family="poisson",data=grouseticks)
DHARMa
The DHARMa
package uses a simulation-based method for residual checks for
generalized linear (mixed) models.
simulationOutput <- DHARMa::simulateResiduals(fittedModel = mod_ss)
plot(simulationOutput)
The left plot performs three tests, the Kolmogorov–Smirnov test,
simulation-based dispersion tests (see
??DHARMa::testDispersion), and simulation-based outlier
tests (see ??DHARMa::testOutliers).
Car
The car package
was designed for “An R Companion to Applied Regression” to provide
functions that are applied to a fitted regression model. The
car::influencePlot() creates a diagnostic bubble plot that
visualizes influential observations in a regression by plotting
Studentized residuals against leverage (hat values), with bubble size
representing Cook’s distance.
car::influencePlot(mod_ss)
## StudRes Hat CookD
## 10 1.6589487 0.2931772 0.5707626
## 20 0.4498749 0.2931772 0.0419733
## 57 5.5270053 0.1218763 2.1198891
## 60 -4.5704359 0.2931772 4.3321637
Model Comparison & Performance
Most model comparisons for merMod objects will focus on
the fixed effects, and the ones shown will be just for fixed effects
unless otherwise stated.
Pbkrtest
The pbkrtest
package implements three tests to examine fixed effects from linear
mixed-effects models (LMM): - Parametric bootstrap test
pbkrtest::PBmodcomp() - Kenward-Roger-type F-test
pbkrtest::KRmodcomp() - Satterthwaite-type F-test
pbkrtest::SATmodcomp()
fm0 <- lmer(weight ~ Time + (1|Chick), data = ChickWeight)
fm1 <- update(fm0, .~.-Time)
pbkrtest::PBmodcomp(fm0, fm1)
## Bootstrap test; time: 21.32 sec; samples: 1000; extremes: 0;
## large : weight ~ Time + (1 | Chick)
## stat df p.value
## LRT 926.47 1 < 2.2e-16 ***
## PBtest 926.47 0.000999 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
pbkrtest::KRmodcomp(fm0, fm1)
## large : weight ~ Time + (1 | Chick)
## small : weight ~ (1 | Chick)
## stat ndf ddf F.scaling p.value
## Ftest 2471.07 1.00 530.53 1 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
pbkrtest::SATmodcomp(fm0, fm1)
## large : weight ~ Time + (1 | Chick)
## small : weight ~ (1 | Chick)
## statistic ndf ddf p.value
## [1,] 2471.7 1.0 530.93 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Note: the pbkrtest::KRmodcomp() function was package was
intended for lmerMod objects only, as the
Kenward-Roger-type F-test was designed for fitted linear mixed
models.
MuMIn
The MuMIn
package provides tools for model selection by automating the process
through subsets of the maximum model.
options(na.action = "na.fail")
mumin_compare = MuMIn::dredge(mod_cw)
mumin_compare
## Global model call: lmer(formula = weight ~ Time + Diet + (1 | Chick), data = ChickWeight,
## REML = FALSE)
## ---
## Model selection table
## (Intrc) Diet Time df logLik AICc delta weight
## 4 11.23 + 8.718 7 -2802.600 5619.4 0.00 0.996
## 3 27.84 8.726 4 -2811.172 5630.4 11.02 0.004
## 2 101.80 + 6 -3265.144 6542.4 923.04 0.000
## 1 121.00 3 -3274.405 6554.9 935.46 0.000
## Models ranked by AICc(x)
## Random terms (all models):
## 1 | Chick
MuMIn::dredge() doesn’t print the fitted model object,
so to find which the labelled models 1, 2, etc., correspond to, use
get.models(mumin_compare, subset = TRUE). The output tends
to be long and is omitted for space reasons.
RLRsim
A common way to test the significance of random effects is through a
likelihood ratio test. The RLRsim package includes
fast simulation-based exact tests that is compatible with
lmerMod objects.
# First model is one with the random effects,
# the second is must be a lm-object.
RLRsim::exactLRT(m = mod_cw, m0 = mod_cw2)
## No restrictions on fixed effects. REML-based inference preferable.
##
## simulated finite sample distribution of LRT. (p-value based on 10000 simulated values)
##
## data:
## LRT = 172.41, p-value < 2.2e-16
Multcomp
The multcomp
package was designed for performing general linear hypotheses in
parametric models.
glh_cw = multcomp::glht(mod_cw)
summary(glh_cw)
##
## Simultaneous Tests for General Linear Hypotheses
##
## Fit: lmer(formula = weight ~ Time + Diet + (1 | Chick), data = ChickWeight,
## REML = FALSE)
##
## Linear Hypotheses:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) == 0 11.2311 5.5780 2.013 0.17828
## Time == 0 8.7175 0.1753 49.742 < 0.001 ***
## Diet2 == 0 16.2193 9.0788 1.787 0.28082
## Diet3 == 0 36.5527 9.0788 4.026 < 0.001 ***
## Diet4 == 0 30.0255 9.0855 3.305 0.00447 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)
We can also look at confidence intervals for the pairwise difference
between groups:
Car
car::Anova() can be used traditional ANOVA-style tables
using Wald \(\chi^{2}\) statistics on
the fixed effects only.
## Analysis of Deviance Table (Type II Wald chisquare tests)
##
## Response: weight
## Chisq Df Pr(>Chisq)
## Time 2474.247 1 < 2.2e-16 ***
## Diet 20.466 3 0.0001359 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Insight
As its name suggests, the insight
package was designed to easily gather information from statistical
models from a variety of different object types. Their documentation
already covers examples of merMod objects here.
Plotting & Prediction
When interpreting results, we may be interested in seeing the
coefficient plots or effect plots. Usually, the first step requires
predicting the response and then fitting the model (using
plot() or ggplot()).
Dotwhisker
The dotwhisker package
was designed to easily make dot-and-whisker plots from regression
results. We recommend scaling by two standard deviations as recommended
by Gelman (2008)..
effects_gt <- broom.mixed::tidy(mod_cw, effect = "fixed", conf.int = TRUE)
dotwhisker::dwplot(effects_gt, by_2sd = TRUE)
Notice that we combined the usage of broom.mixed package to
extract the fixed effects (for the dot-and-whisker plots). Details can
be found by clicking the url for broom.mixed or see this section.
Emmeans
As its name implies, the emmeans package obtains
the estimated marginal means.
More details and examples for lmerMod objects can be
found here.
plot(emmeans::emmeans(mod_cw, "Diet"))
SjPlot
The sjPlot
package was built for easier plotting and table outputs.
sjPlot::plot_model(mod_cw)
ggEffects
The ggEffects
package also computes the marginal effects, returning ready-to-graph
results via the ggplot2
package.
The ggEffects
documentation already includes more details and examples for
lmerMod objects.
predict_ss <- ggeffects::predict_response(mod_ss, "Days [0:9]")
ggplot(predict_ss, aes(x, predicted)) +
geom_line() + labs(x = "Days", y = "Predicted Reaction") +
geom_ribbon(aes(ymin = conf.low, ymax = conf.high), alpha = 0.1)
The ggEffects package is supposed to be superseded by modelbased, however,
it may run errors when using lmerMod objects.
Marginaleffects
As its package name implies, the goal of marginaleffects is to compute
the marginal means as well as various other model comparisons for a lot
of different classes, merMod objects being one of them.
More details and examples for mixed models can be found here.
library(marginaleffects)
# Note: datagrid depends on marginaleffects
pred_ss <- predictions(mod_ss, newdata = datagrid(Days = 0:9))
# Plot using ggplot2
ggplot(pred_ss, aes(x = Days, y = estimate)) +
geom_line() +
geom_ribbon(aes(ymin = conf.low, ymax = conf.high), alpha = 0.1) +
labs(x = "Days", y = "Predicted Reaction")
Other
This section covers other external packages that may be useful but
does not fall under the category of previous sections.
Broom.mixed
The broom package takes
output from built-in functions in R and converts them into a tibble, which is an alternative
to R’s built in data.frame.
Broom.mixed
is a spin-off of broom where it takes outputs from outputs from
specifically mixed models from various popular mixed model packages, and
of course lme4 is one of them.
broom.mixed::tidy(mod_ss)
## # A tibble: 6 × 6
## effect group term estimate std.error statistic
## <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 fixed <NA> (Intercept) 251. 6.82 36.8
## 2 fixed <NA> Days 10.5 1.55 6.77
## 3 ran_pars Subject sd__(Intercept) 24.7 NA NA
## 4 ran_pars Subject cor__(Intercept).Days 0.0656 NA NA
## 5 ran_pars Subject sd__Days 5.92 NA NA
## 6 ran_pars Residual sd__Observation 25.6 NA NA
Equatiomatic
The equatiomatic
package takes a fitted model and writes the equation for you, formatted
in \(\LaTeX\).
equatiomatic::extract_eq(mod_ss)
The output provides \(\LaTeX\) code
that outputs the following equation: \[
\begin{aligned}
\operatorname{Reaction}_{i} &\sim N \left(\alpha_{j[i]} +
\beta_{1j[i]}(\operatorname{Days}), \sigma^2 \right) \\
\left(
\begin{array}{c}
\begin{aligned}
&\alpha_{j} \\
&\beta_{1j}
\end{aligned}
\end{array}
\right)
&\sim N \left(
\left(
\begin{array}{c}
\begin{aligned}
&\mu_{\alpha_{j}} \\
&\mu_{\beta_{1j}}
\end{aligned}
\end{array}
\right)
,
\left(
\begin{array}{cc}
\sigma^2_{\alpha_{j}} & \rho_{\alpha_{j}\beta_{1j}} \\
\rho_{\beta_{1j}\alpha_{j}} & \sigma^2_{\beta_{1j}}
\end{array}
\right)
\right)
\text{, for Subject j = 1,} \dots \text{,J}
\end{aligned}
\]
