How Experimental Design Optimizes Assay Automation
How Experimental Design
Optimizes Assay Automation
by
Thomas Erbach & Lisa Fan, Beckman Coulter, Inc.
Shari Kraber, Stat-Ease, Inc.
An innovative blend of hardware, software and
the right training in statistical know-how
supercharges research automation.
Optimizing biological assay conditions is a challenging process that scientists face every day.
The requirement is to develop high quality, robust assays that work across a range of biological
conditions. The demand is to do this within a short development timeframe. In addition, automated
systems are often required to accommodate large numbers of samples.
Setting up a model to systematically study all key experimental parameters, each across a
defined range, is a challenge. Traditionally, the one-factor-at-a-time (OFAT) approach is used to
study individual experimental conditions. This format is time consuming and tedious. More
importantly, while measuring how changing a single factor will affect the assay, it leaves the
experimenter blind to interactions that may exist between experimental factors. The information lost
in OFAT designs may significantly reduce assay quality and robustness.
In light of this, scientists are adopting design of experiments (DOE) methodology. Using
mathematical models, they evaluate experimental parameters in order to optimize assay conditions.
Designs are created that evaluate both main effects and interactions between experimental factors
such as sample concentration, reagent types, and incubation time. These experimental parameters
represent only a few of the many influencing assay factors that can be studied using DOE.
Integrating DOE and automated liquid handling technology offers scientists the ability to
design experiments that test a wider range of assay conditions. This allows for a clear understanding
of the assay and its components so that improvements can be implemented and the assay optimized.
Statistical software packages that create experimental designs are readily available. However,
translating statistical designs to run on automated liquid handling systems is a complex endeavor. To
simplify the task, Beckman Coulter has developed a software package to help automate DOE
protocols on the Biomek® FX Laboratory Automation Workstation. SAGIANTM Automated Assay
Optimization (AAO) software allows scientists to import designed experiments that are translated
into corresponding Biomek FX methods.
Designed experiments are created in statistical software packages such as Design-Expert®
software, developed by Stat-Ease, Inc. (Minneapolis, MN). Design-Expert software offers scientists a
straightforward approach to creating a variety of design models such as two-level factorials, D-
optimal designs, split-plot designs, and others. Beckman Coulter’s AAO software guides the user
through a wizard-like interface to specify experimental parameters such as labware, deck
configurations and pipetting options to generate Biomek FX methods. Experimental results can be
deconvoluted back into the original design format and transferred into Design-Expert software for
analysis. Design-Expert provides in-depth analysis with interactive graphs to assist in interpreting
assay results. Use of the AAO software enables the end-user to determine which key assay factors
provide the maximal signal and enable the design of a robust assay.
-
1
-
Here is a brief overview of a typical process (Figure 1):
1) Researchers DESIGN the Assay experiment with Design-Expert.
2) Researchers IMPORT the Assay Design into AAO software, and then use it, along with the
Biomek FX to:
• Generate appropriately randomized plate maps
• Configure and create testing methods
• Run assays within the Biomek FX
• Collect response data
• Export the data to Design-Expert DOE software.
3) Researchers ANALYZE the statistical results with
Design-Expert.
Use of this process enables the end-user to screen up to 1,000 individual variations of assay
conditions. The use of Biomek FX automation eliminates months of tedious manual lab work while
the AAO software reduces programming time from weeks to hours. In addition, Design-Expert
software ensures that the maximum amount of information is obtained from the experimental runs.
TRAINING – AN ESSENTIAL ELEMENT FOR ULTIMATE SUCCESS
When AAO software was first introduced, Beckman Coulter trainers became aware that
many researchers were not well grounded in DOE. This knowledge is vital for making the most of
assay optimization. Needing an effective solution, Beckman Coulter called upon Stat-Ease experts to
supplement the statistical aspects of their curriculum. Stat-Ease was selected as a training partner
because their Design-Expert user interface—with its logical layout, sequence, and straightforward
presentation of results—makes design decisions clear and conclusions succinct. Just as important to
Beckman Coulter was Stat-Ease’s commitment to quality training and customer satisfaction. Today,
customers can take advantage of a dedicated 3-1/2 day course at Beckman Coulter’s facilities. This
unique class is designed to educate scientists on the process of using DOE with AAO software.
AAO
DESIGN
ANALYSIS
Software
Design-Expert®
Design-Expert®
Design your
Import DOE design into
Transfer data into
experiment
AAO to create a
DOE for analysis
corresponding Biomek FX
method
Figure 1: Diagram of information flow.
-
2
-
To maximize the return on time invested in the Biomek AAO training sessions, we
recommend that attendees take a refresher course in basic statistics, such as analysis of
variance (ANOVA). Stat-Ease offers a web-based “PreDOE” refresher that will quickly
bring everyone up to speed.
Let’s take a brief look at a collaborative AAO/DOE example that Stat-Ease principal and
statistical consultant Pat Whitcomb provides Biomek FX users during training. This simple
hypothetical case illustrates the process and flow of information when using DOE with AAO
software.
An assay-development group wants to use DOE to study a mouse-cell fluorescent-assay
system. The objective of the study is to find the maximum signal by evaluating three factors that they
suspect may affect it. The three factors selected are:
-Number of cells (5,000 vs. 10,000)
-Amount of stimulant (5 vs. 10 microliters)
-Amount of ligand (5 vs. 10 microliters)
The general procedure is as follows: Cells are pipetted into a 96-well plate where stimulant is
added to induce the mouse cells to express a biomarker. The wells are incubated for two hours and a
fluorescently tagged ligand is added to bind to the biomarker. Finally, the liquid in each of the wells
in the 96-well plate is adjusted to the same level with media and then the plate is read using a
fluorescence plate reader.
The design is first set up in Design-Expert software. All combinations of the two levels of
three factors create eight possible combinations. Given the substantial amount of variation in the
process, replications of the design will be required to determine if any of the factors substantially
affect the response. The DOE software can help determine the appropriate number of replicates; in
this case, five replicates are sufficient (Figure 2). The design is then imported into the AAO software.
-
3
-
Figure 2 - Design-Expert software is used to create the design of experiments (DOE) file.
At this point, the researcher uses AAO to create the method (procedure) that the Biomek FX
will follow (Figure 3).
Figure 3 - AAO software introductory screen. Users import their Design-Expert file into AAO
software and generate Biomek FX methods.
In AAO software, the user specifies the plate type (Figure 4, 96-well), the buffer
information, the pipetting procedures, the incubation period, and the order in which the procedure
will take place. AAO provides the brains and Biomek FX provides the muscle to eliminate tedious
manual pipetting and automate complicated liquid handling operations.
-
4
-
Figure 4 - Experimental plate map shows how the design is randomized in a 96-well plate format.
The method is generated (Figure 5) and executed on the Biomek FX and the final readings
are obtained from a standard fluorescence plate reader. This data is transferred back to
Design-Expert software for the researcher to analyze.
-
5
-
Figure 5 – Screenshots of the generated Biomek FX method. The transfer step highlighted indicates
how experimental factors are pipetted into the experimental plate.
The analysis of a two-level factorial design such as this involves the use of a tool called a half-
normal plot of effects (Figure 6). Any statistically significant factors are easily revealed as outliers on
this plot. This case study revealed a statistically significant interaction between the amount of
stimulant (B) and the amount of ligand (C). After generating the appropriate analysis of variance as
confirmation of the effects plot, model graphs were generated to show the final results (Figure 7).
This interaction graph reveals that the experimenter should avoid the combination “B-, C-“which
causes a significant reduction in fluorescence. The other combinations are acceptable.
-
6
-
DESIGN-EXPERT Plot
Fluorescence Units
Half Normal plot
A: Cell Number
99
B: Stimulant
B
C: Ligand
97
BC
i
li
ty
95
b
a
b
90
C
ro
p
85
%
l
80
a
rm
70
No
60
l
f
a
H
40
20
0
0.00
138.60
277.20
415.80
554.40
|Effect|
Figure 6: Half-normal plot of effects used to select significant effects – the “outliers” on the graph.
These three combinations of ligand
and stimulant yield a relatively
high fluorescence response.
Figure 7: Interaction graph shows the combinations of stimulant and ligand that yield the best
fluorescent values.
-
7
-
There is no doubt that, given today’s accessibility to high speed computing, we will see ever-
increasing application of DOE to all aspects of product development and quality control. The
development of effective software combined with timely, targeted training simplifies and automates
efficient DOE studies and enables more cost-effective discovery. The tiresome and frustrating one-
factor-at-a-time process is yielding to the more informative, efficient and effective world of statistical
design of experiments.
ABOUT THE AUTHORS:
Thomas Erbach is Field Development Manager; Lisa Fan is a Senior Applications Scientist.
Both are with Beckman Coulter’s Biomedical Research Division, Customer Technical Support.
Shari Kraber is a Statistical Consultant with Stat-Ease, Inc.
CONTACT INFORMATION:
Beckman Coulter, Inc.
Biomedical Research Division
4300 N. Harbor Boulevard, Box 3100
Fullerton, California 92834-3100
Phone: 800.742.2345 Fax: 800.643.4366
Website: www.beckmancoulter.com
Design-Expert® software
Stat-Ease,
Inc.
2021 East Hennepin Avenue, Suite 480
Minneapolis, MN 55413
Phone: 612.378.9449 Fax: 612.378.2152
E-mail:
info@statease.com
Website: www.statease.com
-
8
-
