Recent years have seen an increased emphasis on power analysis in published work in subject-matter journals, as well as in requirements for research-grant proposals. Yet this is an area that is not well covered in statistics curricula. It is also a technically complicated area. Even in standard normal-theory analyses, power analysis involves non-central distributions that are typically not implemented in statistical software. Understanding the parameterizations of these distributions is beyond the reach of many practitioners.
The challenge we face is that it is the practitioners who must either do the power analysis, or be very closely involved in it. That is because power analysis is meaningless without a clear reference to the scientific goals of the study design. Central to power analysis is the concept of effect size, which translates to a parameter value under the alternative hypothesis at which the power of the test is to be studied. It is often challenging to identify a target effect size of scientific importance, and it is often confusing to researchers to convert this value, once determined, into a non-centrality parameter for use in power analysis.
Two books, [Cohen(1988)] and [Kraemer and Thiemann(1987)], have become more-or-less standard references for practitioners. These references rely largely on approximations to the non-central distributions (which in most cases is probably just fine) and---more troubling---on simplifications for specifying the target effect size. Cohen, for example, uses data from the existing social-science literature to establish ``small,'' ``medium,'' and ``large'' effect sizes for various classes of tests. These are defined as multiples of the error standard deviation. To use Cohen's effect sizes, we need no pilot data. That's certainly simple and convenient, but that also means that we don't consider any goals connected with the actual scale of the data. It also ignores such important and useful design strategies as blocking and stratification that can significantly reduce the error variance. We prefer to concentrate on situations where the target effect size is based on the scale of the data. This makes the problems harder (in that pilot data or past experience is necessary), but more realistic.
There is of course a growing amount of computing support for power analysis. [Goldstein(1989)] reviews several software packages available in the late 80s, and the [Thomas and Krebs(1997)] brings us up to the present. Thomas and Krebs also maintain a web site at http://www.interchg.ubc.ca/cacb/power/ that is a useful resource for seeking out suitable power-analysis software, commercial or otherwise.
The purpose of this article is to explore some ways of making power analysis more accessible to the investigators who must do it. These days, this usually indicates a need for a graphical user interface (GUI), especially where inexpert users are concerned. We require the interface to be straightforward, the technicalities of non-centrality parameters to be relatively invisible, and the goals of the experiment to be highly visible. Meanwhile, we also need to accommodate the complexities of many real studies, for example multifactor analysis of variance.
The two approaches considered here are add-in functions for Microsoft Excel, and Java applications. The former approach is useful because of the wide availability of Excel and because spreadsheets are a good fit to the ``what if?'' character of power analysis. The Java approach is useful for creating new interfaces that would be hard to do in Excel. Such applications are portable and can be made accessible to distant consulting clients through the Internet.
Interested users may connect to the author's web site at http://www.stat.uiowa.edu/~rlenth/Power/ to download the Excel module or to try out some of the Java applets.