EDM 1 - 2 - 102 :: Part - 1 :: Design of Experiments and Research Method Concepts 2026-07-11 00:08 UTC

This exposition is a near extractum~\footnote{I copied the original! some one can personally kick me out of a good university} and presents principles and methodologies for the design of experiments; and explores classical foundations, fixed and random effects models, block and factorial designs, and modern techniques including response surface methodology and orthogonal arrays. Emphasis is placed on the statistical modelling, estimation, and hypothesis testing procedures that validate experimental conclusions; attention is given to optimality criteria and design algebra. The narrative integrates design theory with broader research methodology; highlighting the roles of randomisation, replication, and control in achieving scientific validity.

Applications across agriculture, industry, and the sciences are illustrated through contestable mathematical treatment, because I guessed, now you will not know where and how I guessed. A whole engineering and science course in a few pages, what is missed here for me will be contextual disciplinary glossaries used according to the nature of work. Given any application, using the available glossary in an intelligent manner with common sense, utilising the discussed concepts, I must argue will result in simplicity and reproducibility. The only difference here is I have offered some views on research methods(I do not know), which I have been thinking about for a very long time.

A. General Remarks

The design of experiments is a part of the broader process of statistical planning required to collect data in a manner suitable for valid statistical inference. In various fields of research and application, careful experimental design ensures that data are collected efficiently and that subsequent statistical analysis (such as hypothesis testing or estimation) will be meaningful. The main purposes of the design of experiments are:

• To specify an appropriate statistical model for the experiment, along with the associated linear model for the data;

• To plan the experiment so that it yields results with maximum precision for a given amount of resources (thus achieving efficiency and economy);

• To provide a theoretical foundation justifying the statistical analysis of the experimental results (for example, validity of tests of significance under the chosen design and model).

Historically, the field was greatly developed by Sir R. A. Fisher, whose contributions established fundamental principles for experimental design. Fisher introduced three key principles for designing experiments:

• Replication: Each treatment (or condition being studied) should be applied to multiple experimental units. Replication provides a way to estimate experimental error and increases the precision of treatment effect estimates by reducing the impact of random variation.

• Randomisation: Treatments should be assigned to experimental units at random. Randomisation protects against systematic biases and ensures that error terms have identical distributions, allowing the use of probability theory to justify significance tests.

• Local Control (Blocking): If there are sources of variability (nuisance factors) among experimental units, these should be controlled by grouping similar units into blocks. Within each block, treatments are assigned at random. Blocking reduces unexplained variability and thus improves the accuracy of comparisons among treatments.

Using these principles, I seek to reduce the influence of nuisance factors and obtain more reliable conclusions. In practice, there is often a trade-off between feasibility and strict adherence to these principles; a good design balances practical constraints with statistical benefits.

In statistical terms, an experiment is typically described by a linear statistical model. We observe an -dimensional data vector (with observations or experimental units total). The general linear model can be written as

where is an design matrix of fixed known values (determined by the assignment of treatments to units), is a -dimensional vector of unknown parameters (often called effects or effect parameters), and is an -dimensional random error vector. Each component of corresponds to a fixed effect such as a treatment mean, a block effect, or other factor effects in the model. The components of represent random error terms for each observation (experimental unit). It is typically assumed that (the errors have zero mean) and (all errors have a common variance and are uncorrelated). Often are further assumed to be normally distributed. Under these assumptions, ordinary least squares will yield unbiased estimates and the usual analysis of variance (ANOVA) and hypothesis testing procedures are applicable.

There are three broad classes of models in experimental design:

• Fixed-Effects Models: All factors (effects) in are treated as fixed, unknown constants. For example, if includes an overall mean and treatment effects , a fixed-effects model treats each as a fixed but unknown parameter to be estimated. Conclusions apply only to the specific treatments tested.

• Random-Effects Models: Here some factors are random variables rather than fixed constants. A factor is considered random if the levels used in the experiment are randomly sampled from a larger population of possible levels. In such a model, the corresponding components of are random with specified distributions (typically mean zero and some variance). For example, if blocks are considered a random sample from a population of blocks, block effects can be modeled as random variables.

• Mixed-Effects Models: These contain both fixed and random effects. A common case in experimental design is a mixed model with fixed treatment effects and random block effects. In a mixed model, we can partition as where contains fixed effects and contains random effects. The linear model can be written

where is a design matrix for the random effects. For instance, might assign treatment means and might assign block effects to observations. We assume and typically for some . We also assume any random effects in are independent of the error . Under a mixed model, some of the variability among observations is attributed to random effect terms rather than residual error.

For statistical inference, we usually impose distributional assumptions on and any random effects. A common set of assumptions (normal theory) is:

• (a) for all errors , and for all .

• (b) The errors are independent (or at least uncorrelated).

• © The errors are normally distributed (i.e. ).

• (d) For any random effects in , , , and has a normal distribution.

• (e) Random effects are independent of the error terms .

Under these conditions, the standard results of the general linear model and normal theory apply. In particular, least squares or maximum likelihood methods can be used for estimation, and F-tests or likelihood ratio tests can be used for hypothesis testing.

Many problems in experimental design can be formulated as questions about the linear model (1) under one of the above classes of models. The two major statistical issues are:

• Parameter estimation and hypothesis testing for effects of interest (treatments, interactions, etc.), given the design matrix (and for mixed models).

• Design optimality and selection of a good design matrix (and ). This involves choosing the arrangement of treatments on experimental units to meet certain optimality criteria (such as minimising variance of estimates, ensuring identifiability of effects, etc.).

We also seek theoretical justifications for our methods. For example, demonstrating that under randomisation, the test of a null hypothesis (like “no treatment effect”) is valid and the significance level is controlled, requires a combinatorial or randomisation-based argument.

In summary, the design of experiments addresses how to arrange experimental runs (trials) in such a way that the resulting data can be analyzed to yield clear and valid conclusions. In the following section*s (B through M), we discuss specific types of experimental designs, the linear models associated with them, and methods for estimation and hypothesis testing in each context.

B. Block Design

Many experiments involve variability among experimental units that is not of primary interest. A block design is used to control (or “block”) such nuisance variability. In a block design, the experimental units are divided into groups called blocks, such that units within the same block are relatively homogeneous or similar, and differences between blocks account for a major source of variability. By restricting comparisons of treatments to within blocks, we achieve greater precision.

Formally, suppose there are treatments (treatment levels) to be tested and blocks of experimental units. Let denote the number of units in block that receive treatment . The matrix is called the incidence matrix of the block design. The design is said to be uniform if each experimental unit receives exactly one treatment and each unit belongs to exactly one block (which is the usual case). We define:

• as the total number of replicates (applications) of treatment across all blocks.

• as the number of units (or treatment assignments) in block (sometimes called block sise).

If each treatment appears equally often ( for all ) and each block has the same sise ( for all ), the design is said to be uniformly replicated and equal block sise. A special case is the randomised complete block design (RCBD), in which each block contains exactly one replicate of each treatment. In an RCBD, (each block contains all treatments once) and for all (each treatment appears in every block). RCBDs satisfy Fisher’s three principles: each treatment is replicated times, assignment within each block is random, and blocking provides local control of variability.

If a block design does not contain all treatments in each block, it is called an

Incomplete Block Design In an incomplete block design, ; that is, each block contains only a subset of the treatments. Despite not having every treatment present in every block, comparisons between all treatments are still possible via the overlap of treatments across different blocks. A necessary condition for this is that the design be connected; that is, for any pair of treatments, there exists a sequence of blocks in which each consecutive pair shares a common treatment, allowing one to conceptually “travel” from one treatment to another. Formally, a block design is connected if the incidence matrix has rank ; equivalently, no subset of treatments is isolated in a disjoint subset of blocks. In a connected design, all treatment contrasts are estimable. We typically assume that the design is connected; otherwise, certain treatment comparisons would not be statistically meaningful.

Statistical Model for a Block Design The standard statistical model used for block designs is a two-way additive ANOVA model given by:

where denotes the observation from the unit receiving treatment in block ; is the overall mean; is the effect of treatment ; is the effect of block ; and is the random error term associated with that observation. The treatment effects represent deviations from the mean due to treatments, and the block effects represent systematic differences across blocks. For a fixed-effects analysis, and are treated as fixed but unknown parameters, usually constrained by identifiability conditions such as and . Alternatively, for a random-effects analysis (see Section \textasteriskcentered G), the block effects may be treated as random variables drawn from a common distribution.

Under the fixed-effects model, the parameters of interest are the treatment effects . The block effects are nuisance parameters that help absorb variability. The presence of block effects generally reduces the residual error variance, leading to more powerful tests for differences among treatments.

Randomisation in a block design is performed by randomly assigning treatments to units within each block. In a randomised block experiment, for each block we randomly permute the treatments (if complete) or randomly select and assign treatments (if incomplete) to the units in that block. This ensures that within each block, every treatment assignment is equally likely, which validates the use of the usual analysis assuming independent, identically distributed errors within blocks.

We distinguish a few special types of block designs:

• Complete Block Design: Every block contains all treatments (this is the RCBD case). The incidence matrix has all entries and each column (block) has exactly one in each row (treatment), meaning for each treatment in block . Thus and . The RCBD is connected (indeed, each treatment can be compared within any block).

• Balanced Incomplete Block Design (BIBD): This is a specific type of incomplete block design where each treatment appears in exactly blocks, each block contains exactly treatments, and every pair of distinct treatments co-occur in exactly blocks (for some ). Such a design is balanced in the sense that all treatments are treated equally and all pairs of treatments have equal opportunities to be compared. The parameters of a BIBD satisfy the combinatorial relationships:

where is number of treatments, number of blocks, replicates per treatment, treatments per block, and the pairwise concurrence. The second equation comes from counting, in two ways, the number of times a given treatment pairs with any other treatment in blocks. A BIBD is necessarily connected. An example of a BIBD is given by a projective plane arrangement when for some integer : there exists a symmetric BIBD where , , and . (Fisher’s famous “Latin square of order ” construction for prime power yields such a design.)

In any block design, each observation can be thought of as:

if treatment is applied in block (and if treatment is not in block , no observation exists). We can represent this using the incidence matrix : one way to write the model is

where is an matrix indicating treatments (it has if observation received treatment , else ) and is an matrix indicating blocks ( if observation is in block ). The incidence matrix summarises how treatments and blocks intersect.

Analysis of a Block Design (Fixed Effects): If all treatment and block effects are fixed, the least-squares estimates of the effects can be obtained by solving the normal equations. In matrix terms, we have the model where and . The normal equations are

Because of the linear constraints (e.g. , ) or the singularity of (due to overparameterisation if no constraints are set), one typically uses generalised inverses to solve for . The least squares estimator of is (the grand mean of all observations). The estimator of each treatment effect is , where is the average of observations under treatment . Similarly, the block effect estimator (with the average in block ). These estimates satisfy and by construction.

To test hypotheses about treatment effects in a block design, we use the analysis of variance (ANOVA) approach. The total variability in the data can be decomposed into:

with corresponding degrees of freedom:

(for a complete block design; for incomplete blocks, error df = after accounting for both treatments and blocks). In an RCBD, one commonly computes:

and

where sums are taken only over observed . The mean squares are , , and . To test whether there is any treatment effect, we test (all treatments have the same mean). Under , and assuming the usual normal error model, the appropriate test statistic is

which follows an -distribution with and degrees of freedom if is true. If is significantly large, we reject and conclude that at least one treatment mean differs from the others.

Blocking generally removes some variability from the error term. We can also test for the presence of block effects (though usually blocks are a nuisance factor, sometimes one might check if blocking was effective). The hypothesis can be tested with

with and degrees of freedom. A significant result would indicate differences between blocks (justifying their inclusion).

In summary, a block design allows more precise estimation of treatment differences by accounting for block-to-block variation. If blocks are chosen well, the error mean square will be substantially smaller than it would be in an unblocked design (i.e. a completely randomised design where all units are treated as a single block), thus yielding more powerful tests for treatment differences.

C. Estimation under the Fixed-Effects Model

We now consider the general problem of estimating the parameters in the linear model (1) under a fixed-effects assumption. This means we treat all effects of interest as fixed, unknown quantities. We assume that the conditions (a) and (b) from section* A hold (zero-mean errors with common variance, and uncorrelated errors; normality © is not required for unbiased estimation, but will be used later for tests).

The primary method of estimation is the method of least squares. The least squares estimator (LSE) of minimises the sum of squared residuals:

leading to the normal equations as mentioned. If is nonsingular (full column rank, which requires and the design is such that all parameters are identifiable), then the solution is

If is singular (which happens when the parameters are not all identifiable due to linear dependencies in columns of , as in effects models with sum-to-zero constraints), one can still obtain a solution using a generalised inverse. The solution for is then not unique, but any solution that satisfies provides estimates of all estimable functions of . An estimable function is typically a linear combination of that does not depend on the specific parametrisation (for example, differences between treatment means are estimable even if individually are not unique due to a sum-zero constraint).

For practical computations and theoretical analysis, it is convenient to use the projection matrix , where is a Moore-Penrose inverse of . is the hat matrix (or projector) onto the column space of . Then is the vector of fitted values (projection of onto ). The vector of residuals is . The variance-covariance matrix of is (when is invertible; for a generalised inverse solution, is well-defined for any estimable linear combination ).

In many experimental designs, the matrix has a special structure that simplifies calculations. For example, in a balanced design (like a BIBD or factorial with equal replication), is often block-diagonal or has a simple eigenstructure that yields closed-form solutions for estimates of certain contrasts. Orthogonality is an important concept: a design is said to be orthogonal if the columns of corresponding to different factors are orthogonal in the sense that is diagonal or block-diagonal along factor partitions. Orthogonality implies that the estimates of one factor’s effects do not depend on the levels of other factors (no correlation between effect estimates). Many standard designs (RCBD, factorial designs with equal replications) are orthogonal designs.

As an example, in the one-way layout (a single-factor completely randomised design with say treatments and total observations), the design matrix can be partitioned as where is the column for the overall mean and indicates treatment. Then

which (under constraint ) leads to , as we already described. This design is orthogonal: is orthogonal to each treatment contrast in the sense that and for (with appropriate coding, e.g. using constraints, or if using cell-means coding then columns are not strictly orthogonal but normal equations separate nicely due to the sum-zero constraint).

In more complex designs, solving the normal equations directly can be tedious. However, the principle remains: we find linear combinations of the observations that provide unbiased estimates of treatment effects. For example, in a two-factor factorial (see section* F), the orthogonality of main effects and interactions in a balanced factorial simplifies the calculation of effect estimates via contrasts of cell means.

In summary, under the fixed-effects model, estimation is usually done via least squares. The result is a set of linear estimators for the effects. These estimators are unbiased (under assumptions (a) and (b)) and have minimum variance among linear unbiased estimators (by the Gauss-Markov theorem). If in addition the errors are normal (assumption ©), then the LSEs are also the maximum likelihood estimators and have a multivariate normal distribution.

D. Test of a Hypothesis under the Fixed-Effects Model

Having obtained estimates of effects, we often need to test hypotheses about them. The typical hypotheses in experimental design are that certain treatment effects or interactions are zero (no effect).

Consider a general linear hypothesis

where is a matrix specifying a linear combination of parameters (e.g., differences between certain treatment means). Under the fixed-effects linear model with normal errors, will have a multivariate normal distribution with mean and covariance . The hypothesis is equivalent to . The classical approach to testing is to use the F-test based on sums of squares (the general -test in linear models, sometimes called the -test for contrasts).

In the context of ANOVA, often states that all effects of a certain type are zero. For instance, in a one-way layout (completely randomised design), means no treatment differences. In a two-way layout with no interaction, one might test for no row factor effect or no column factor effect.

The general theory of the F-test says: if has rank (meaning there are independent linear constraints), then one can construct a sum of squares due to , , which has expectation when is true (and larger when is false). The mean square for the hypothesis is . This is typically compared to the residual (error) mean square from the model fit, , by forming

Under , follows an -distribution with and degrees of freedom (where is number of parameters in ). A significant indicates rejecting .

For example, in the block design of section* B (RCBD with fixed effects), to test of no treatment differences ( ), we have:

This sum of squares has degrees of freedom. The error sum of squares has degrees of freedom (for RCBD, , so error df = ). The -statistic

is used to test . This is exactly the described in section* B for treatments.

Another example: In a two-factor factorial (say factors with levels and with levels, in a completely randomised design with replicates of each combination), we might test : no interaction between and . The hypothesis matrix would pick out all interaction effects (or equivalently test that cell means equal sums of main effects). The sum of squares for interaction can be computed by subtracting main effect fits from the cell means. One would form with and degrees of freedom to test for interaction.

Multiple Comparisons: If is rejected (e.g., treatments are not all equal), one may proceed to identify which treatments differ. This often involves multiple comparison procedures (such as Tukey’s HSD, Bonferroni-corrected t-tests, or Scheffé’s method). Scheffé’s method in particular provides simultaneous confidence intervals for all contrasts of treatment means and can be used to test any linear contrast with a controlled family-wise error rate \cite{4}. Multiple comparison methods ensure that the overall error rate is controlled when making several comparisons between treatment means.

In summary, under a fixed-effects model with normal errors, hypothesis testing is carried out by constructing sums of squares for sources of variation (treatments, blocks, interactions, etc.) and comparing mean squares via F-tests. If the design is balanced and orthogonal, these sums of squares cleanly partition the total variability and the F-tests for different effects are independent. In unbalanced designs or non-orthogonal situations, one must be careful in defining hypotheses (e.g., use of Type I, II, III sums of squares in software to handle different marginal hypotheses).