EDM 1 - 2 - 102 :: Part - 2 :: Design of Experiments and Research Method Concepts 2026-07-11 00:16 UTC

E. Balanced Incomplete Block Designs

In section* B, we introduced incomplete block designs, where not all treatments appear in each block. A particularly important class of such designs is the Balanced Incomplete Block Design (BIBD). In a BIBD, as defined earlier:

• There are treatments and blocks.

• Each block contains treatments ( , so blocks are incomplete with respect to the full set of treatments).

• Each treatment appears in exactly blocks.

• Every pair of distinct treatments appears together in exactly blocks.

The parameters of a BIBD satisfy two fundamental equations:

(because counting the total number of treatment placements in all blocks can be done by summing by blocks or by treatments), and

The second equation comes from considering a specific treatment : it appears in blocks, and in each such block it co-occurs with other treatments, so the total number of pairs involving treatment is . On the other hand, if every other treatment pairs with in exactly blocks, and there are other treatments, this count is also . Setting these equal gives the equation. From these, one can derive as another relation.

A BIBD is called symmetric if (the number of blocks equals the number of treatments). In a symmetric BIBD, . One example of a symmetric BIBD is when , , (projective plane design). For instance, for , and we have the Fano plane design.

Statistical analysis of a BIBD: We use the same additive model for observations of treatment in block . Because the design is balanced, the least squares estimates of treatment effects have a neat form: each treatment mean is based on observations, and each difference between treatments can be estimated with the same variance. In fact, all pairwise treatment comparisons have equal precision under a BIBD. The information matrix for treatments (after eliminating and block effects) is proportional to , where is the identity and is the matrix of all ones. This structure (with two eigenvalues: one corresponding to the eigenvector of all ones, and one for the orthogonal directions) indicates that there are only two distinct variances for estimates: one for differences and one for means vs overall mean. Typically, one finds the variance of the estimated difference between any two treatment effects is

assuming a fixed-effects ANOVA model with residual variance . Thus, all treatment comparisons share the common standard error .

The ANOVA for a BIBD will partition variability into Treatments, Blocks, and Error. The treatment sum of squares will have degrees of freedom and the block sum of squares degrees. We can test whether treatments differ using as usual. The balance ensures that this F-test is optimal (uniformly most powerful for detecting any contrasts among treatments).

Optimality of BIBD: BIBDs are in fact variance-balanced designs and often A-optimal or D-optimal within the class of designs with given . Fisher’s information matrix for (treatment effects) in a fixed-effects model is proportional to , where gives block sises. For a BIBD, this information matrix has a very simple eigenstructure, which often maximises certain optimality criteria (A-optimality: minimising average variance of estimates, D-optimality: maximising determinant of information matrix, etc.) among incomplete block designs.

Existence and construction: Not every parameter tuple satisfying the basic equations corresponds to an actual BIBD. There are combinatorial conditions (such as Fisher’s inequality which states in any non-trivial BIBD, meaning the number of blocks is at least the number of treatments). Construction methods for BIBDs include:

• Direct methods: Finite geometries, symmetric Latin squares, and other combinatorial configurations have been used to directly construct BIBDs for certain parameter sets. For example, the projective plane of order gives a symmetric BIBD with as mentioned.

• Recursive methods: One can often build larger BIBDs from smaller ones (e.g., using multiplication theorems or developing methods). For instance, if a BIBD with parameters exists, sometimes one can construct a BIBD for for some multiples or transformations of these parameters (like adding a new treatment and appropriate blocks in a symmetric design).

• Computer search: For arbitrary parameters that satisfy the necessary equations, sometimes one must resort to computational search or heuristic algorithms to find a BIBD or prove none exists.

Many sets of parameters remain unknown in terms of existence of a BIBD, which is a topic in combinatorial design theory. However, for small or special forms, tables of known BIBDs are available in the literature \cite{8}.

BIBDs provide a balance that makes analysis straightforward and optimal in many senses. When a BIBD exists for given and block sise constraints, it is often the design of choice for comparing treatments in blocks of sise .

F. Factorial Experiments

A factorial experiment is one in which multiple factors (independent variables) are investigated simultaneously. Each factor has two or more levels (settings). A treatment in a factorial experiment is a combination of levels of all factors. The factorial approach allows one to study not only the individual effects of each factor (main effects) but also their joint effects (interactions).

Suppose there are factors . Let factor have levels ( ). The total number of treatment combinations (distinct factor level combinations) is

If the experiment includes all possible combinations of levels of the factors, it is called a full factorial experiment at those levels. Often we denote a specific treatment combination by the tuple meaning factor is at level , factor at level , etc.

As an example, consider factors: Factor with levels and Factor with levels. The full factorial design consists of all combinations with and . If we have replicates of each combination, the total number of runs is . This is a two-factor factorial experiment.

Factorial designs can be symmetric if all factors have the same number of levels (e.g., design where each factor has 2 levels, or design, etc.), or asymmetric if factors have differing numbers of levels (e.g., a design for two factors with 2 and 3 levels respectively).

The advantage of factorial designs is that they allow the study of interaction effects ; how the effect of one factor may depend on the level of another factor. Factorial experiments are more efficient than studying each factor separately because the same experiment runs provide information on all factors.

Linear Model for a Factorial: In a two-factor example with factors and , a common fixed-effects model is:

where is the effect of the -th level of factor , is the effect of the -th level of factor , is the interaction effect for the combination of and , and is the random error for the -th replicate of that combination. Constraints like , , and are applied to make the parameters identifiable. For factors, the model extends similarly, with main effect terms for each factor and interaction terms for every subset of factors.

ANOVA for Factorial Designs: The analysis of a factorial experiment partitions the variability into components due to each main effect and each interaction. For a two-factor design:

• Factor has degrees of freedom (df).
• Factor has df.
• The interaction has df.
• The residual error has df (if replicates).

We can arrange the ANOVA table as:

\begin{center} \begin{tabular}{|l|c|c|c|} \hline Source & Sum of Squares & df & Mean Square \hline Factor & & & Factor & & & Interaction & & & Error & & & \hline Total & & & \hline \end{tabular} \end{center}

The sums of squares can be computed by formulas using cell means (mean for combination ) and marginal means. For instance:

where is the grand mean of all observations.

We then form F-tests:

to test the null hypotheses of no factor effect, no factor effect, and no interaction respectively. Each follows an -distribution under its respective null hypothesis.

One generally tests the interaction first. If interaction is significant, interpretation of main effects needs caution because the effect of one factor depends on the level of the other. If interaction is not significant, one may drop the interaction term and test main effects on a reduced model (pooling the interaction SS into error).

Factorial experiments can involve more than two factors. For example, a factorial (three factors, each at 2 levels) has main effects for factors , , (each 1 df), two-way interactions , , (each 1 df), and a three-way interaction (1 df). The analysis follows similarly with appropriate partitioning of sum of squares.

Factorial designs are very efficient. For example, comparing a factorial (8 treatment combinations) to doing three separate two-level experiments for each factor (which would require combinations but would not reveal interactions): the factorial uses a bit more runs but yields interaction information. Moreover, factorial designs permit estimation of interactions that separate one-factor-at-a-time experiments cannot detect.

Orthogonality and Alias Structure: In a full factorial with equal replications, the design matrix can be made orthogonal with respect to main effects and interactions. For instance, in a factorial (with factors coded as and ), the columns corresponding to main effects and interactions (using products of factor-level code for interactions) form an orthogonal matrix (each column has mean 0 and is uncorrelated with others). This simplifies analysis as each effect’s estimate is proportional to a contrast of the data.

When factors have different numbers of levels (asymmetric factorial), orthogonality can still be achieved if the design is balanced (equal replications of each combination), although the coding of interactions is a bit more complex. A factorial experiment is said to be balanced if each level of any factor occurs equally often in combination with each level of every other factor. Full factorial designs with equal replication are balanced by construction.

Factorial designs allow researchers to explore a large experimental domain efficiently. For instance, a design covers combinations which grows exponentially with , but often is not too large or one uses fractional factorials (next section*) to manage runs.

G. Estimation in a Random-Effects Model

Up to now, we have primarily discussed fixed-effects analyses. We now turn to estimation in a model that includes random effects, i.e. a model of the mixed or random-effects class as introduced in section* A. A typical scenario is a randomised block design where blocks are considered random effects. Another example is a multi-stage experimental design where one factor is a random sample from a population of factor levels.

Consider the block design model:

but now assume the block effects are random effects. Specifically, assume for , and the random errors , independent of the . The treatment effects could be fixed (common in mixed model: treatments fixed, blocks random), or possibly also random if treatments are also a random sample from a population (less common in practice unless one is interested in variance components among treatments).

The presence of random effects introduces parameters called variance components: here is the variance component for blocks, and for residual error. Estimation in random/mixed models typically involves estimating these variance components as well as fixed effects.

Two common methods for variance component estimation are:

• Method of Moments (ANOVA Estimators): Use the expected mean squares from ANOVA to solve for variance components. For the random block model, the ANOVA table would have:

where is the number of observations per block (assuming equal block sises for simplicity). From these, an unbiased estimator of is

and . This is the classical ANOVA estimation of variance components.

• Maximum Likelihood (ML) or Restricted Maximum Likelihood (REML): One can write down the likelihood of the data under the normal mixed model and solve for the variance components that maximise it. REML is often preferred as it produces unbiased estimates of variance components by basing on residual likelihood. In our simple model, the REML equations lead to results equivalent to the ANOVA method for balanced designs.

Once and are estimated, we can estimate the fixed effects (e.g., treatment means) typically by generalised least squares or by using the so-called Best Linear Unbiased Estimators (BLUE) for fixed effects, and Best Linear Unbiased Predictors (BLUP) for random effects. In the mixed model, the BLUE for a treatment effect is essentially the difference between that treatment’s average and the grand mean, adjusted by a shrinkage factor if the design is unbalanced or if treatments were also random. For balanced designs, the estimated treatment means are just the sample means (same as in fixed effects model).

An important concept in random-effects models is the intraclass correlation or variance ratio. In the random block model,

is the correlation between two observations in the same block (since they share the same block effect). This indicates how much of the variability is attributable to block-to-block variation. A high means blocks are very different (and blocking is very important), whereas near 0 means blocks contribute little variability (and the design effectively reduces to a completely randomised design).

To illustrate, in a BIBD treated as a mixed model (blocks random, treatments fixed), one can derive the minimal sufficient statistics for estimating and . It turns out that the sum of squares between blocks and within blocks form such sufficient statistics. For example, one can show for a BIBD:

if is the number of blocks in which a pair of treatments co-occur (appearing from expected mean square of block effect in BIBD). This formula generalises the simpler one given above for equal .

In mixed models, after estimating variance components, we often are interested in predicting the random effects (for example, estimating the effect of a particular block). The BLUP for a block effect is

which is a weighted version of the block mean deviation, shrunk towards 0 by the factor . If (block sise) is large or is large relative to , this shrinkage is small (block effect estimate close to block mean); if blocks are small or block variance is low, the estimate is strongly pulled toward zero (because with little evidence or little true variability, the best guess for a block effect is near zero).

Random-effects models are analyzed in the framework of variance component analysis. The primary inference is often on variance components themselves (e.g., testing if to see if blocks matter) or on fixed treatment effects while accounting for random variation from blocks. Tests for fixed effects in a mixed model can still often be done with F-tests, but one must use the appropriate denominator mean square (which may be a linear combination of mean squares if the design is unbalanced). In balanced designs, usually one of the mean squares (like MS ) serves as error term for treatment F-tests even with random blocks, because and under (treatments no effect). In the random block case, the F for treatments uses MS same as fixed model. If one were to test the significance of the random block variance , a likelihood ratio test or a quasi F-test can be used (but testing a variance component is tricky since is a boundary of parameter space; in balanced designs, one can use and consult certain reference distributions or approximate it by ).

In summary, estimation in random-effects models involves extracting information about variance components from the data. Balanced experimental designs are very helpful here as they give orthogonal estimates of variance components. The block design with random blocks is a prototypical case that illustrates the methodology: one uses differences between block means and within-block variability to estimate and . Once variance components are known, one can better estimate fixed effects (with potentially improved precision by appropriate weighting) and make predictions for random effects if needed.

H. Hypothesis Testing in Random-Effects Models

When some factors are random, the nature of hypotheses and tests changes slightly compared to the purely fixed-effects case. In a random-effects or mixed model, one often is interested in:

• Testing hypotheses about fixed effects (e.g., treatment differences) while accounting for random effects.

• Testing whether variance components are zero (i.e., whether a random effect is actually needed or has any variability).

For fixed effects in a mixed model (like treatment effects in a blocks-as-random design), we can largely proceed as in section* D, but we must use the correct error term. The presence of random effects can alter the distribution of certain sums of squares under . In balanced designs, there is a traditional approach using expected mean squares to determine the appropriate -ratio.

Consider again a random block model (treatments fixed, blocks random). To test : no treatment differences ( ), the appropriate -statistic is

just as in the fixed model. Here estimates and has expectation under any alternatives. Under , as well, so under follows . Thus, we test treatments as usual.

Now suppose we want to test if the block variance is zero: (no block-to-block variability). This is testing a random effect. Classical -tests are not exact here because under , the distribution of is not exactly (since under blocks contribute nothing and becomes a ratio of two that are not independent, one having fewer degrees of freedom). However, for moderate block numbers, one can treat

as an approximate with and degrees of freedom. Alternatively, one can do a likelihood ratio test comparing the model with estimated to a model with (though the latter is on the boundary, the LRT null distribution is a mixture of distributions in such cases).

In general, for random effects, one often uses either:

• Chi-square tests (Likelihood ratio tests): Compare the log-likelihood of the model with the random effect to one without it. The difference in is asymptotically where is the difference in number of parameters (here for one variance component), though with a mixture or half- adjustment for variance component at boundary.

• F-tests via Mean Squares (Approximate): Using ANOVA mean squares and expected mean squares to construct -ratios. In balanced designs, these are often given by textbook formulas. For instance, in a two-factor design with one factor random and one fixed (a mixed two-way ANOVA), one uses a specific mean square for the denominator when testing the fixed factor (the denominator mean square would include the interaction which contains the random factor’s variance).

To illustrate the latter: Suppose factor is fixed with levels, factor is random with levels (and replicates per combination). Then:

• To test (fixed): use , since (interaction mean square) contains the random variation of and and serves as error term for . (Because and , so under , both have same expectation.)

• To test (random): test by , approximately, or via LRT.

• To test interaction (if considered random): .

This is a standard mixed-model ANOVA approach \cite{8}.

In summary, hypothesis testing in mixed models requires choosing correct error terms. A rule of thumb in balanced designs (sometimes called the expected mean square rule) is: for testing a fixed effect, use an error term (denominator mean square) that contains the random effect components that the numerator contains except the effect of interest. For testing a random effect, use an error term from a source that would disappear if that random effect variance were zero (often the interaction or error term).

In modern practice, one can also directly use computational methods (REML and approximate Wald or Kenward-Roger adjusted F-tests) to test fixed effects in mixed models, and likelihood ratio tests for random effects. These methods handle unbalanced data more generally. But in the context of classical experimental design theory, many designs were balanced to allow straightforward F-tests as described.

I. Fractional Factorial Designs

When the number of factors or levels grows, a full factorial experiment can require a very large number of runs. Fractional factorial designs address this by running only a carefully chosen subset (fraction) of the full factorial combinations, while still allowing estimation of the most important effects under certain assumptions.

For instance, a full factorial has runs. A fractional factorial is a design with runs, which is of the full set. It is called a fraction of the design. The idea is to sacrifice the ability to estimate some higher-order interactions (which are assumed negligible) in order to reduce the number of experiments.

In fractional factorial designs, some effects are aliased with each other. Aliasing means that a certain linear combination of effects is what is actually estimated by a contrast, rather than a single effect. In a design, independent effects (typically high-order interactions) will be aliased with some set of other effects. For example, in a run fractional design for 3 factors ( ), we might choose the design generators such that the fraction runs correspond to those satisfying (i.e., the product of the signs for factors is positive). This generator means we have aliased the interaction with the grand mean (i.e., we restrict ourselves to one subset of combinations). The result is that in the analysis, the main effect of factor is aliased with the interaction (because their columns in design matrix become identical under the fraction constraint), and the main effect of is aliased with , etc. We cannot distinguish between an main effect and a interaction effect; they are confounded.

The concept of resolution is used to characterise fractional factorial designs:

• Resolution means that no effect involving or fewer factors is aliased with any other effect involving or fewer factors \cite{8}. For example, a Resolution III design means no main effect is aliased with any other main effect, but main effects may be aliased with two-factor interactions (3 = 2+1, indicating main effect can be confounded with 2-factor interaction). Resolution IV means no main effect is aliased with any 2-factor interaction, but 2-factor interactions may be aliased with each other. Resolution V means no main or 2-factor is aliased with any main or 2-factor; they may be aliased only with 3-factor interactions, etc.

A design of resolution is often denoted by Roman numeral, e.g., design. Generally, higher resolution is better because it means important lower-order effects (which we typically care about) are not confounded with each other.

As an example: A half-fraction of a (four factors) can be constructed as a Resolution IV design. Say factors are . We choose a generator (meaning the design points satisfy , effectively as generator). This yields aliases: is aliased with (a three-factor interaction), with , with , with (so all main effects aliased with 3-factor interactions, which is less concerning if 3-factor interactions are assumed negligible). Also, any two-factor interaction will be aliased with another two-factor: e.g., is aliased with , with , and with . So it’s Resolution IV (main effects clear of two-factors, but two-factors are confounded with two-factors). If we believe 3-factor interactions are zero, the main effects are estimable without bias; if we also believe two-factor interactions are negligible relative to main, then even two-factor interactions estimates may be interpretable after the fact (but strictly, they are aliased pairs).

Designs can be constructed using orthogonal arrays as a generalisation. An orthogonal array is an array of runs, factors (columns), each factor has levels (assume is same for simplicity), with the property that in any subset of columns, every level combination appears equally often. This is a generalisation of the idea of a fractional factorial where the design is balanced for any -factor interactions. The strength of an corresponds to a design in which all interactions among any factors are not aliased with each other (they are balanced). A full factorial in factors of levels is of strength . A fractional factorial achieving strength (meaning resolution if interpreted in two-level designs) would be an .

For two-level designs, an has strength 2, which corresponds to resolution III if it’s a fractional factorial where main effects are not aliased with each other (and balanced). Strength 3 corresponds to resolution IV (no main with 2-factor aliasing), etc.

Chakravarti (1956) introduced the idea of balanced fractional factorials and orthogonal arrays in the context of asymmetric factorials \cite{8}. An orthogonal fractional factorial design is one in which the chosen fraction maintains orthogonality (balance) for certain subsets of effects (up to a certain order). These designs allow estimation of those effects as if it were a full factorial (just with fewer total runs).

In practice, when running a fractional factorial, the experimenter typically assumes that higher-order interactions (three-factor and above, often) are negligible. Under this assumption, the aliased effects involving higher-order terms can be ignored or assumed zero, thereby allowing clear interpretation of lower-order effects. This is known as the sparsity-of-effects principle: most of the time, systems are dominated by main effects and possibly a few two-factor interactions; higher interactions are rare.

If after running a fractional factorial it turns out some aliased effects are both potentially large (e.g., a main effect and a two-factor interaction aliased together seem to produce a significant contrast), further runs can be done to de-alias them. This is the idea of design augmentation: one can run the complementary fraction or certain additional runs to distinguish confounded effects.

To summarise fractional factorials:

• They are highly economical designs when many factors are involved.

• They introduce confounding (aliasing) of effects, which must be managed by careful planning and making reasonable assumptions.

• They are often used in screening experiments where the goal is to identify the few factors that have significant effects among many candidates. One might start with a resolution III design (which confounds main effects with two-factor interactions) to screen factors, assuming two-factor interactions are negligible initially; then follow up with higher resolution designs on the surviving factors to check interactions.

Modern industrial experimentation heavily uses fractional factorials, often with factors at two levels (also called screening designs). These are a part of the Six Sigma toolkit and other quality improvement strategies, where dozens of factors might be initially considered.