: Each study adds to the storehouse of know=
ledge=20
about distance learning. This data is of value to instructional desi=
gners,=20
instructors, administrators, and other researchers. Some studies con=
firm=20
what we already know =96 or thought we knew; some findings challenge=
our=20
previous positions, others provide reinforcement.
<=
/P>
Dr. Liu provides additional building blocks=
that=20
support areas of significant difference and areas with no significan=
t=20
difference. The latter is as important as the former, because it giv=
es=20
assurance that little or nothing is lost if we implement learning pr=
ograms=20
online using appropriate pedagogy and technology. The flip side of t=
he=20
coin is ability to serve many learners who could not otherwise=20
participate, and know that losses are not significant and significan=
t=20
gains can be achieved.
Effects of Online Instruction vs=
.=20
Traditional Instruction
on Students=92 Learning
Yuliang Liu
Abstract
This=20
quasi-experimental study was designed to compare the effects of onli=
ne vs.=20
traditional instruction on students=92 learning in two different sec=
tions=20
(online vs. traditional section) of a graduate course for K-12 schoo=
l=20
teachers on Research Methods in Education in the summer of 20=
03.=20
The experimental group involved twenty-two graduate students who rec=
eived=20
online instruction on WebCT; the control group involved twenty-one=20
students who received traditional instruction. Participants in both =
groups=20
completed the same chapter quizzes and a final test, as well as othe=
r=20
essay writings, peer critiques, and group projects during the 10-wee=
k=20
summer semester. Results indicated the experimental group significan=
tly=20
outperformed the control group in most quizzes and the final test.=20
Keywords: online instruction, learni=
ng=20
outcomes, significant difference, no significant difference. =
Introduction
Distance=20
education has grown fast in recent years. In the 2000-2001 academic =
year,=20
56% of all 2-year and 4-year institutions in the United States offer=
ed=20
distance education courses for various learners. In addition, 12% of=
all=20
institutions planned to offer distance education courses in the next=
3=20
years (Waits & Lewis, 2003). Currently, online instruction is a=
=20
primary method for distance education. With online instruction, the=
=20
student is separated from the teacher and connected through the use =
of a=20
computer and the Internet. More and more institutions are offering o=
nline=20
courses and/or programs to their students in order to meet various=20
learners=92 needs. Online learning and instruction, as an integral p=
art of=20
the teaching and learning process in higher education, is growing as=
fast=20
as the technology itself. On the other hand, traditional classroom=20
instruction is face-to-face instruction, typically conducted in a=20
classroom setting in a lecture/discussion/note taking mode. =20
Recent=20
research has indicated that online education has positively influenc=
ed=20
many aspects of education both directly and indirectly (CEO Forum, 2=
000;=20
Phipps & Merisotis, 1999). Until recently, the viability of onli=
ne=20
learning was not proven. On one hand, Clark (1983, 1994) maintained =
that=20
media do not influence learning in any condition. On the other hand,=
Kozma=20
(1994) debated that educational technologies influence learning by=20
interacting with an individual=92s cognitive and social processes in=
=20
constructing knowledge. These earlier debates are still relevant sin=
ce=20
newly emerging technologies respond to the earlier criticisms and en=
able=20
learners to use them more efficiently.
According=20
to Phipps and Merisotis (1999) and Russell (1999), there have been t=
wo=20
lines of research comparing students=92 end-of-semester grades or le=
arning=20
outcomes for online and traditional sections. The first line of rese=
arch=20
focused on the =93significant phenomenon=94 and cited significant in=
creases in=20
learning outcomes for online learners over their traditional counter=
parts.=20
The most widely cited literature in this line is McCollum=92s (1997)=
report.=20
McCollum cited a sociology professor who divided his statistics clas=
s into=20
two groups: one in online format and one in face-to-face (FtF) forma=
t.=20
According to McCollum, online students had more collaboration and th=
eir=20
performance outscored their traditional counterparts by an average o=
f 20=20
percent.
Later=20
studies also supported the =93significant phenomenon=94. Day, Raven,=
and=20
Newman (1998) compared and studied the effects of web-based vs.=20
traditional instruction on students=92 achievement in undergraduate=
=20
technical writing in an agricommunication course. They found that on=
line=20
students attained significantly higher achievement scores in the maj=
or=20
class project and essay assignments than those in the traditional co=
urse.=20
In addition, Day, Raven, and Newman found that online students obtai=
ned a=20
higher mean gain in attitudes toward writing.
Nesler,=20
Hanner, Melburg, and McGowan (2001) studied a large sample from 30=20
institutions and found that nursing students in distance programs ha=
d=20
higher scores in professional socialization outcomes than their=20
campus-based counterparts. Al-Jarf and Sado (2002) investigated two =
groups=20
of freshman students in their first ESL writing course and found the=
=20
experimental group (web-based instruction) made more gains in writin=
g,=20
became more efficient, made fewer errors, and communicated more easi=
ly and=20
fluently, compared with the traditional classroom control group.=20
The second=20
line of research supported the =93no significant phenomenon.=94 Thes=
e studies=20
cited no differences in learning outcomes between online and traditi=
onal=20
groups. Navarro and Shoemaker (1999) found that about 90% online lea=
rners=20
in a graduate MBA class believed that they learned as much as or mor=
e than=20
they would have in a traditional classroom. Schulman and Sims (1999)=
did=20
not find any significant differences on the posttest scores between =
the=20
online and traditional students in an undergraduate course. Jones (1=
999)=20
conducted a comparison study of an all web-based class to a traditio=
nal=20
class and also found no significant differences in GPA between onlin=
e and=20
traditional learners.
More=20
recently several other studies have found no differences in learning=
=20
outcomes in various courses between online and traditional learner.=
=20
Johnson, Aragon, Shaik, and Palma-Rivas (2000) compared a graduate o=
nline=20
course with an equivalent course taught in a traditional format on o=
utcome=20
measures such as course grades and student self-assessment of their=
=20
performance in the course. They found no significant differenc=
es=20
between the online and traditional student groups. Less than signifi=
cant,=20
traditional students had slightly more positive perceptions about th=
e=20
instructor and overall course quality.
Ryan=20
(2000) compared online and traditional student performance in=20
construction, equipment and methods classes and found no significant=
=20
differences in performance between the two groups. Student evaluatio=
ns of=20
the course were also similar. Similar results of no significant=20
differences in performance were also found by Gagne and Shepherd (20=
01) in=20
their graduate accounting class, as well as by Johnson (2002) in an=
=20
introductory biology class.
Review of=20
the above studies indicates most studies in this area found no signi=
ficant=20
differences in learning outcomes between online and traditional cour=
ses in=20
various subjects. Fewer studies have been conducted at the graduate=
=20
level.. This exploratory study was designed to investigate whether o=
nline=20
instruction affects learners=92 learning during a semester-long grad=
uate=20
course in teacher education. Learners=92 progress in the online and=
=20
traditional sections is assessed by chapter quizzes and final grades=
, as=20
well as essay writings, peer critiques, and group projects. A pre-co=
urse=20
assessment was conducted and analyzed to ensure that both sections w=
ere=20
equivalent. Based on the above literature review, the major research=
=20
hypothesis in this study was:
Research=20
hypothesis:
There was no significant difference in learn=
ers=92=20
learning performance, as measured
by chapter quizzes and final g=
rades,=20
between the online section and the traditional section, at the compl=
etion=20
of a semester-long graduate course.
Method
Participants
All=20
students who self-selected to enroll in EDUC501 (Research methods in=
=20
Education) in both online and traditional sections for 10 weeks in t=
he=20
summer semester of 2003 were solicited in the first week for partici=
pation=20
in this study. EDUC501 is a required core course in education at the=
=20
master=92s level at a midwestern state university in the United Stat=
es.=20
Students in this course were from different graduate programs in=20
education. Twenty-four students enrolled in the online section, but =
two of=20
them withdrew within the first two weeks due to time commitment and=
=20
unexpected family issues. Thus twenty-two students in the online sec=
tion=20
were included for final analysis and twenty-one students enrolled in=
the=20
traditional section. Thus, a totals of 43 participants in both secti=
ons=20
were recruited to participate in the study. Participants in both sec=
tions=20
were asked to complete consent forms and demographic surveys in the =
first=20
week. A pretest of course content in both sections was administered.=
A=20
preliminary analysis of the pretest revealed that the control group =
scored=20
a little higher than the experimental group. No significant differen=
ces=20
were detected in pretest performance between online and traditional=
=20
sections.
Instruments
Formative=20
and summative assessments of participant learning were conducted in =
two=20
major domains: knowledge and application. Knowledge assessment focus=
ed on=20
individual learning and included seven chapter quizzes and one final=
test.=20
Application assessment focused on collaborative learning and include=
d a=20
combination of essay writings, peer critiques, and a group research=
=20
project. The application assessment is consistent with Wade=92s (199=
9)=20
perspective. That is, writing is a unique indicator of student=92s l=
earning=20
including communication between student and student, as well as betw=
een=20
student and teacher. The final grades of students were assigned base=
d on=20
these two major assessments. Both sections had the same quizzes, ess=
ay=20
writings, and group research paper every week. Each chapter quiz was=
=20
administered as an individual open-book test, but without peer discu=
ssion=20
in both sections. Each quiz had 25 objective multiple-choice items=20
regarding each chapter to be completed within 40 minutes. The quizze=
s in=20
the online section were only available during a specific week and we=
re=20
graded instantly after the completion. Online learners were delighte=
d to=20
have immediate quiz results and feedback; quiz results and feedback =
in the=20
traditional section were reported back to the class in the following=
week.=20
Experimental Design
This study=20
used a non-equivalent control group design. In both the experimental=
group=20
(online via WebCT) and control group (traditional classroom), the=20
dependent variables of learning performance were pretested and postt=
ested.=20
The independent variable was online vs. traditional instruction in a=
=20
graduate course. Based on recommendations from the Institute for Hig=
her=20
Education Policy (2000) and Kearsley (2000), a hybrid of instruction=
al=20
techniques was employed in the online section. Specifically, several=
major=20
features of WebCT were used throughout the semester such as weekly o=
nline=20
writing, peer critiquing, bulletin board discussion, online testing,=
and=20
e-mail. Constructivist learning theory was the major theoretical=20
foundations for online instruction in this course. Instructional des=
ign=20
was based on the ADDIE model (Analysis, Design, Development,=20
Implementation, and Evaluation) proposed by Dick, Carey, and Carey (=
2001).=20
For additional information on design, development, and instructional=
=20
strategies used in this course, see other recent publications =
by the=20
author (Liu, 2003a; 2003b).
To reduce=20
learner anxiety and maximize learning, one FtF orientation was condu=
cted=20
in the first week for the online section. The traditional section me=
t once=20
a week for 3 hours and was primarily taught FtF throughout the semes=
ter.=20
Both sections were taught simultaneously by the lead investigator in=
the=20
summer semester of 2003. In order to make both sections as equivalen=
t as=20
possible, the instructional objectives, content, requirements,=20
assignments, and assessments in both sections were the same. =
Procedure
The=20
pretest was administered in paper-and-pencil format to both sections=
in=20
the first week to determine initial learning and performance. Next,=
=20
participants in the online section were introduced to the online Web=
CT=20
environment from the second through the final week. Ongoing posttest=
s,=20
including chapter quizzes and final test, were administered online f=
or the=20
online section and administered in paper-and-pencil format for the=20
traditional classroom.
Results and Discussion
Pretests=20
and posttests of learning performance in both sections were coded an=
d=20
analyzed using SPSS 12. Descriptive statistics of all quizzes and te=
sts in=20
online and traditional sections are presented in Table 1. Results of=
=20
participants=92 seven chapter quizzes and one final test in both sec=
tions=20
were analyzed using independent samples t test and presented =
in=20
Table 2.
Table 1
Descriptive Stat=
istics=20
of all Quizzes and Tests
in Online and Traditional Sections
Groups
N
Mean
Standard Deviation
Standard Error Mean
ch1 quiz
experimental group
22
96.82
5.68
1.21
control group
21
83.10
10.18
2.22
ch2 quiz
experimental group
22
92.73
4.81
1.03
control group
21
88.33
8.56
1.87
ch3 quiz
experimental group
22
91.36
7.27
1.55
control group
21
86.67
5.99
1.31
ch4 quiz
experimental group
22
90.23
7.48
1.59
control group
21
83.10
7.33
1.60
ch5 quiz
experimental group
22
85.23
9.19
1.96
control group
21
81.19
8.79
1.92
ch6 quiz
experimental group
22
89.77
8.66
1.85
control group
21
86.90
4.60
1.01
ch13 quiz
experimental group
22
90.23
8.38
1.79
control group
21
84.76
8.87
1.94
Pre-assessment
experimental group
22
41.4545
12.07
2.57
control group
21
44.9524
8.66
1.89
Final test
experimental group
22
87.6364
7.24
1.54
control group
21
77.7143
9.68
2.11
Final grade
experimental group
22
4.0000
.00
.00
control group
21
3.8095
.40
.09
Table 2
Results of t<=
/U> Test=20
in Various Assessments
between the Experimental and Control=20
Groups
t-test for Equality of=
=20
Means
t
df
Sig.=20
(2-tailed)
Mean=20
Difference
Std. Error=20
Difference
95% Confidence Interval=
of the=20
Difference
Lower
=
Upper
=
ch1 quiz
Equal variances=20
assumed
5.491
41
.000
13.72
2.50
8.68
18.77
Equal variances not=20
assumed
5.423
31.033
.000
13.72
2.53
8.56
18.88
ch2 quiz
Equal variances=20
assumed
2.087
41
.043
4.39
2.11
.142
8.65
Equal variances not=20
assumed
2.061
31.178
.048
4.39
2.13
.05
8.74
ch3 quiz
Equal variances=20
assumed
2.307
41
.026
4.70
2.04
.59
8.81
Equal variances not=20
assumed
2.318
40.159
.026
4.70
2.03
.60
8.79
ch4 quiz
Equal variances=20
assumed
3.157
41
.003
7.13
2.26
2.57
11.69
Equal variances not=20
assumed
3.159
40.969
.003
7.13
2.26
2.57
11.69
ch5 quiz
Equal variances=20
assumed
1.471
41
.149
4.04
2.75
-1.51
9.58
Equal variances not=20
assumed
1.472