Power/Sample-size GUI for balanced ANOVA models
Creating a GUI
Use the form above to enter the information needed to create a graphical
user interface (GUI) for the ANOVA model of your choice. The GUI is useful
for examining power, detectable effect size, or sample size for the F
tests. The required information is:
-
Title: Enter any title you want; it will be displayed at the top
of the GUI window.
-
Model: Enter the ANOVA model. Terms are delimited by a "+" sign
(not optional!); "*" denotes interactions, "|" denotes expansion
into all interactions, and "()" denotes nesting. Some examples appear later.
-
Levels: Enter factor levels in the form
fac1 #1 fac2 #2 ...
, in any order you like. Any factor not listed defaults to 2 levels.
-
Random: Enter the names of the factors that have random levels,
in the form fac1 fac2 ... . Anything not listed is taken as fixed.
-
Reps: This window must contain a whole number. If that number
is 2 or more, a "within cells" error is added to the model; if it is 1
or less, the design is considered unreplicated and there will be a "residual"
term instead.
When everything is entered, click on the Create GUI button; a new
window then pops up with the user interface for the model you specified.
It may take awhile for the window to appear (especially the first time).
You can revise the information (or not) and open as many additional GUIs
as you like. Note: In the Levels and Random windows, only
factors
should be listed. Give only the root name of the factor.
This dialog is not very robust. If you misspell a factor name, fail
to put "+" signs between the terms, etc., the model structure will not
be built. You may refer to the Java console (on Netscape, use "Options/Show
Java console") to see error messages. These messages are Java-generated
but they often display the string that it found troublesome.
Examples
Randomized complete-blocks design
Suppose we have n subjects (we'll start with n=10) and each subject is
to be tested once with each of 4 different treatments (in separate random
order for each subject). Then use:
Title: RCB design
Model: SUBJ + treat
Levels: SUBJ 10 treat 4
Random: SUBJ
Reps: 1
Giving reps = 1 sets it up so that an unreplicated design is assumed.
Two-way ANOVA
Here we'll leave the Levels and Random fields blank; we can always change
them in the GUI. A standard 2-way model with interaction is used.
Model: Temp | Speed (expands to TEMP + SPEED + TEMP*Speed)
Reps: 5 (or any number >= 2)
Nested factorial
Subjects are divided into 4 groups, each identified with a particular drug.
Subjects are tested with 3 different dosages of the assigned drug.
Model: drug + SUBJ(drug) + dose + drug*dose
Levels: Drug 4 Dose 3 Subj 6
Random: subj
Note: Case is ignored, so "Subj" and "subj" get identified with "SUBJ".
The model statement dictates what gets displayed. Note in the Levels and
Random fields, only "subj" is given, not "subj(drug)".
Using the GUI
The ANOVA power GUI contains familiar elements such as radio buttons and
input windows, as well as several unfamiliar bar-graph-like elements that
work a little bit like scrollbars.
To use the bar-graph interface
lick the mouse at any point along the centerline of a bar, and the value
of that parameter will be changed to that number. (The exception are the
bars that display power; inputs on those are disabled.) Alternatively,
you may enter a value in the associated input window and hit the Enter
(or Return) key. You may also drag the end of a bar with the mouse. (This
can keep your computer pretty busy calculating noncentral F probabilities;
so drag cautiously.)
Below each set of bars is a numerical scale. If the range of that scale
is not to your liking, click at any point along that scale and drag to
a new position for that scale value. For example, to double the range of
the scale, you might click at "2" and drag left to "1" before letting go.
Scales are always updated to make room for all the values in a set.
Solving power/sample-size problems
The top section of the window is used for changing the sample size (or
number of levels) and selecting which factors are fixed and random. The
bottom section has bar graphs for effect size and power. There is also
an "alpha" window for setting the significance level (for all of the F
tests). To obtain sample size, input the effect sizes (see below for details)
and try different sample sizes until the desired powers are achieved. There
are potentially several tests, with different powers. Keep in mind that,
for equal effect sizes, the biggest player is the number of observations
at each level (or combination of levels) of a term. So, for example, tests
of interactions are less powerful than tests of main effects, all other
things being equal. Often, sample size is limited by time or budget. Then
you would probably enter the budgeted sample size(s), and either observe
the power, or vary effect size to see what can be detected with a reasonable
power.
Effect size
Effect size is quantified by the standard deviation (SD) of the associated
effects in the ANOVA model. For a random effect, the effect size, squared,
is thus the variance component for that term. Note in particular that the
effect size for "Within" or "RESIDUAL" is the error SD. Typically, one
uses existing data or does a pilot study ahead of time to estimate these
standard deviations.
For a fixed effect (meaning that all factors involved have fixed
levels), there is a model term of the form tau_{i,j,...}. Let Q denote
the sum of the squares of all of these taus; then the effect size is sqrt(Q/d),
where d is the degrees of freedom associated with the model term (not
the error df!).
More development is afoot for providing more flexibility in the ways
that effect sizes can be specified. (Suggestions are welcome!)
Technicalities
All tests are based on the "unrestricted model" for the analysis of variance,
whereby it is assumed that all random or mixed terms are iid normal random
variables. This is the same model used by SAS in determining expected mean
squares. There are other possible models, the most popular being the "restricted
model" where a mixed effect is constrained to sum to zero over any subscript
associated with levels of a fixed factor.
Sometimes, the correct error term (denominator) for an F ratio cannot
be found, in which case an appropriate linear combination of mean squares
is constructed, and Satterthwaite approximation is used to obtain the degrees
of freedom. Such tests are only approximate.
It is extremely important to remember that the effect sizes of random
and mixed terms can affect the powers of other tests! That is because they
are used as values of the effect standard deviations that may appear in
the error terms of other tests. To dramatize this, try a model like A
| B | C | D and watch what happens to the power when you change one
of the factors to random.
Disclaimer
Please understand that, though I have made every effort to write correct
code, there may still be errors. I do not guarantee the correctness of
any results you obtain and do not take responsibility for any losses incurred
as a result of those errors.
Russell V. Lenth
Department of Statistics and Actuarial Science
University of Iowa
Iowa City, IA 52242
Russell-Lenth@uiowa.edu