Implementing a Large On-Campus ALN: Faculty Perspective

Edwin Kashy, Michael Thoennessen, Guy Albertelli II, Yihjia Tsai ¹
Michigan State University

¹ Permanent Address: Department of Computer Science, Tamkang University, Taiwan

Abstract
This case study describes the implementation and continued operation of a large on-campus ALN for a 500-student course in introductory physics. The ALN was used to modify and complement the original course and thus represents an evolution rather than a revolution. A highly positive impact on student success rates was achieved and continues. Factors that increased faculty satisfaction and instances of dissatisfaction are presented. The potential increase in the latter with technology is of some concern.

I. INTRODUCTION

Approximately 500 science and engineering students enter the calculus-based Introductory Physics two-semester course, PHY183-184. For most of the students in the class, this course is a requirement each semester. The course is part of the established curriculum. Its goals and standards are well defined by the textbooks commonly used across various institutions. In its present form, it is an attempt to combine the best features of the face-to-face interactions and lectures with the use of network tools for anytime/anywhere interaction. The goal has been to establish and maintain high standards while providing students with the means and opportunity to succeed. This ALN was initiated in fall 1995 and has been continued since that time [1].

II. RATIONALE

In fall 1995, we implemented our first ALN with support of the Sloan Foundation. This was in large part as a consequence of discussion with colleagues at UIUC who had embraced the ALN concept and already had encouraging initial results [2]. Our ALN followed three years of computerized assignments in introductory physics and chemistry courses using the Internet. This showed the added dimension that networked technology could add to a course. The prospect of improved student performance and satisfaction was also a driving factor, especially as we had observed that students were spending considerably more time in the course. For the first time, we were seeing student effort at a level that justified our own instructing efforts. This was indeed a source of satisfaction!

Other members of the department taught this course during the past two years (Table 1). While using the ALN was not required, they were encouraged to do so to take advantage of the savings that were generated by the reduced teaching staff used in the ALN, i.e., about two-thirds of that in the traditional course (The reduction of teaching staff results from the automated grading with CAPA. Teaching assistants (TA) for grading are not needed anymore. Instead a smaller number of TAs is used for face-to-face and ALN help. The savings are proportional to the class size). The comments of these instructors are included in this case study.

Table 1. Information About the Four Faculty Members of this Study

III. BACKGROUND INFORMATION FOR THE COURSE

This course was the first to use essentially all the features of an on-campus ALN. Being a required course for a large number of students, planned and actual enrollment was the same, 480 students.

A coordinator provided the technological support for the 15-20 instructors using the CAPA system each semester at Michigan State University (MSU). This included obtaining class lists electronically and providing them to the instructor in the proper format for immediate use, setting up the class directories, and testing that the system was operational for student login via the Web and/or Telnet. The coordinator also set up the Internet discussion forum for students, initially with commercial software and later as part of CAPA, and assisted in responding to students who encountered difficulties related to the technology.

Instructor training was a key part of the technological support. At MSU, supported by our Sloan Foundation grant, we have opted for one-on-one training and it has worked very well so far. The coordinator introduces the instructor to the system and works with the instructor on the technical aspects of content preparation and operation. The learning curve is thus very steep and de-facto adapted to the instructor's technological skill levels. Time to become proficient enough to run a course has varied from less than three hours to three days. For most of that time, however, the instructor and coordinator are working on their own in the same office, with any difficulty encountered being addressed immediately. While this may appear to be less efficient than training in a class or workshop, our success rate is total; once proficient, most instructors continue to use the technology.
 
Note that E. Kashy and M. Thoennessen are part of the development team of CAPA as an ALN tool. They had relatively small technology support requirements for ongoing course tasks, but had support in testing and implementing new or upgraded features.

A. Technology and Infrastructure
The principal ALN tool used was CAPA, a computer system developed at MSU over the past seven years. In the initial experience with its use, a 90-student science class, student reaction was very positive and has since been replicated in numerous other disciplines and at other institutions [3-9]. This integrated software system has been used to

• Prepare, deliver, and grade personalized homework, quizzes and examinations.
• Provide feedback to students and instructors.
• Communicate with students in a class and provide a discussion forum for students.
• Provide links for student help via the Internet.
• Facilitate course management - Table 2 summarizes some of the CAPA features for the instructor.
• The system was originally developed for Physics and Chemistry courses [3,5] and was used to develop a broad variety of conceptual questions adapted to the technology. Students are given several tries to get correct solutions and are given full credit when they succeed before the due date.

Table 2. The Three CAPA Modules Available for the Instructor

While hints are available when an incorrect answer is given, the system does not attempt to give the students feedback on the particular error made (other than a formatting error). We believe that a key aspect of problem solving lies in the ability to both detect and correct one's mistake.

The strong emphasis on conceptual problems within CAPA makes it is a useful tool for many fields [7,9,10]. Figure 1 shows an example of a conceptual problem. It clearly goes beyond a traditional one-out-of-five multiple-choice question. To solve this problem, students need a reasonable understanding of Archimede's Principle.

Figure 1. Example of a Conceptual Question in CAPAThe hint will read: "Think Archimede's Principle. How does the volume of fluid displaced by a body that `floats' differ from that for a body that sinks?"

An additional strength of CAPA is the variations of a problem among students. A simple example is shown in Figure 2. Two versions of the same problem are shown for different students. This encourages collaboration without simply copying the solutions. Again, each individual student has to understand his own problem. It is also possible to give the students different selections; however, one should be careful that all the same concepts are presented to all students.

Figure 2. Two Different Versions of the Same Problem
The selections are automatically randomized.

Conceptual problems are an important part for understanding. There is a large correlation between understanding the concepts and the ability to solve numerical questions as shown in Figure 3.

 

Figure 3. Correlation Between Conceptual and Numerical Problems

The most recent addition is a direct link to a discussion forum implemented in CAPA when accessing from the Web. This allows each student to participate directly in a discussion with other students and/or the teaching assistant (TA) while working on a specific problem. This feature simplifies the use of discussion groups significantly because the additional log-in and password required for an external discussion forum is not necessary. In this moderated discussion forum, the TAs make sure that the postings are hints and not just the posting of formulas. This control mechanism does not exist for Websites initiated by students that have recently appeared. On these sites solutions are posted and even problems having typical randomized variables can have their solutions posted. Fortunately, these student endeavors can be overcome. Figure 4 shows an example we have used. As can be seen it is virtually impossible to communicate the solution of this problem over the Web without discussing the physics involved (Kirchoff's Rules). That is exactly the purpose of collaboration¾explaining and understanding the problems. It becomes easier to learn how to do the problem than to subvert the system.

Figure 4. Additional Randomization of Labels with CAPA

B. Content Delivery
Table 3 shows the various components of the course. As in most physics courses, demonstrations are a large component of lecture time. Traditional lecture time is, however, significantly reduced, allowing for large segments of time on interactive lecture exercises and unannounced short quizzes. These quizzes have had a positive impact in improving class attendance even though they serve to assign only a small proportion of the student's grade (5%). The quizzes are also useful in identifying misconceptions very early.

Homework assignments are personalized. The conceptual problems are designed to encourage collaboration among students as they can benefit from the additional practice of working on somewhat different versions of the same problems. Numerical story problems have variables that inhibit rote copying among students. Recently we have added new techniques that have strongly inhibited the sharing of formulas where one can just plug in one's variables and get the correct answer without really understanding the problems. Each weekly assignment has a firm due date. This insures students do not fall behind. Students are given full credit if they get the correct answers before the due date. If a solution entered is incorrect, the student may work to find the errors. A number of attempts are allowed to obtain the solution. Thus, most students are able to receive very good grades on the assignments, and this has proved to be highly motivating. Note that these high grades do not lead to grade inflation, as a higher absolute scale is used to assign the course grade.

Table 3. Time (%) Spent on Various Aspects of the Course

Collaboration is also encouraged by a small component of teamwork in assignments. For example, groups of up to five students working together can submit a subjective essay discussing the observation of an experiment shown by video during class. This also has a big efficiency factor for the instructor grading these essays.

Assignments have a significant component of challenging problems. Students can obtain help at any time via the discussion forum established for the class. Posted questions are sure to be addressed within 24 hours or less by TAs or other students. In addition, help is available at scheduled periods throughout the week in a Physics Learning Center. There teaching assistants, who are assigned to the class and who have themselves solved their own personalized assignments, use the Socratic method in helping students with any difficulties. Students needing help can be there several hours while those who do not feel such a need are not required to be there at all. This face-to-face help is an important part of the class, especially for less prepared students. It is also helpful to the instructor as it provides complimentary feedback to the detailed on-line feedback available from the student performance as recorded by the system. The Learning Center is also the way we have countered the impersonal nature of instruction in this large course. It provides many opportunities for one-on-one interactions between students and teaching staff.

A concept test given during the first week and again near the end of the class helps compare the class to others at similar institutions and represents one of the measures of learning. Mid-term and final exams are the main assessment tools. In the case of the mid-terms, we allow students to earn partial credit by correcting any part of the exams on which they have not done well.

Following the proctored examinations, students can pick up a copy of a different version of the same exam. They then can solve the problems in that version within the next three days and enter answers via the Internet. They can consult with fellow students, the teaching staff, etc. to get help. By this process, the mid-term grade is the original grade plus 30% of the difference between the corrections and original exams, i.e., they recover "30 cents on the dollar" of every missed point. More than 99% of students avail themselves of this correction opportunity. It is a very popular option! For the instructor, it represents an effective and efficient way to encourage students to review all the material on the exam and improve their understanding. (A common comment by students finding out how to solve a problem missed on the exam is, "I should have gotten it! It wasn't that hard.")

The 99% of the students who work on the corrections actually solve all exam questions. In contrast, in traditional exams the students typically do not even look at their wrong answers and do not try to understand what they did wrong. This extra effort on the students' part comes at almost no extra effort by the instructor.

III. RESULTS

A. Effectiveness
Before we discuss faculty satisfaction, we need to mention briefly the effectiveness results obtained from our on-campus ALN approach. These can be found in the Journal of Engineering Education [1], where we demonstrated a strong, positive impact on student success in large classes while maintaining high standards. In retrospect, the improvement in student achievement observed should not have been surprising as our approach has been to retain the best established practices of more traditional teaching and to use ALN to fix the weak areas where it clearly can have a major impact.

The important aspects of using the technology which appear to have a significant positive impact on student achievement include

In addition, the efficiencies generated by the use of technology allow us to devote increased teaching staff time to Socratic one-on-one interactions with students.

B. Saving Paper
One issue in implementing the technology for large on-campus ALN course involves the printing of personalized assignments. Since the assigned work is on the Web, should students be given printed copies?  Our reason for printing and handing out assignments has been to promote an environment where students collaborate and discuss their work. Such collaboration has been shown to have positive impact on performance [13]. Would such collaborations be less likely if they had to work at individual computers? We have recently collected some data on the printing issue. In fall 1998, assignments for a large student course (not using CAPA) were provided on the Web. When polled, most of those students said they printed assignments from the Web all or most of the time.

We then polled the students in PHY183 (480 students using CAPA) during the fourth week of the semester to see if they would agree to have the following week's assignment not printed. The vote was overwhelmingly against, 420 to 6. In addition, 89 students commented by E-mail¾one was willing, 88 were not. The reasons given in those 88 E-mails are shown in Table 4.

Table 4. E-mail Responses of 88 Students

We believe that in our situation, the small effort by the instructor to prepare printed worksheets is well repaid by the large saving in student time and increased student interaction level. Note that because of a highly efficient format, our printing of the assignments prepared with Latex requires from four to nine times less paper than when students print from the Web.

C. Faculty Satisfaction
To put the faculty satisfaction issues in perspective, we have interviewed faculty, including some who have not used ALN in their disciplines and looked at previous studies of issues that affect faculty satisfaction [14-19]. Faculty satisfaction is complex. The principal factors, which emerge from the literature, from interviews with our colleagues in this study, and from our own experience, include collegiality, workload, and autonomy. An interesting observation concerns the role conflict that occurs at the intersection between faculty and administrative domains of responsibility. While it does not appear to affect general faculty satisfaction, it can be a source of disaffection and dissatisfaction.

Our experience has been that the implementation of ALN technology on a large scale in teaching has greatly increased the domain where administrative and academic responsibility and control intersect. Thus, it is not surprising that we have experienced numerous situations that engendered faculty dissatisfaction, ranging from not so important to instances that, in our perception, are critical factors in how we want to do our task. In this area of collective decision-making and responsibility, we have encountered a spectrum of administrative attitudes across the administrative ladder. In the four situations briefly described below, we should keep in mind that the descriptions are from the faculty and that the perception of the administrator(s) involved may be considerably at odds. The following paraphrase professors either using or using and developing ALN:

For the present survey we explicitly interviewed six physics professor who taught with CAPA recently. We also have close contact with most of the instructors using CAPA for about 20 courses in several disciplines on campus. The four cases above represent the major conflicts brought to our attention. Note that they are one-sided views in areas of shared responsibility and could be described in quite different terms from an administrative perspective. In the implementation of ALN technology, such areas have grown significantly, and thus, increase potential conflicts as faculty and administrators carry out their tasks.

With the increased use of technology in education, administrators have an increasing and important role in an area where they have limited knowledge and in which faculty also are not often experts. A quote (with a large dose of sarcasm) from a colleague asked to comment on sources of dissatisfaction in his work, "... too much administrative interference, decisions based on ignorance."  We do not want to leave the impression that conflicts dominate our interaction with administrators. On balance, they have been facilitators and helped to establish the conditions in which we have obtained highly positive and encouraging results [1].
The level of satisfaction with the on-campus ALN implementation is high across many disciplines and faculty who have implemented all or part of its functionality. This satisfaction comes in spite of the universal agreement among faculty that work is increased, especially initially. Technical support is rated good to "… wonderful." Positive interactions with satisfied students, by far the majority, is a big factor, as is the interactions with colleagues doing ALN with whom one can share a remarkable variety of wonderful stories. The on-campus aspect has strong appeal. One faculty who volunteered that he is now ".. a convert to this technology" added that he liked that he was "still teaching the normal way."  Another commented, "This was the first time I had the ability to really see how students were doing in such a large course and could review that information before meeting with them."

There has been a redistribution of responsibility in the large courses. There is some loss of faculty control as courses depend more on the support of system administrators and on the proper functioning of the technology infrastructure. More of the course administration and management has become part of the lecturer's work. Instructors now have detailed knowledge of student performance, and so do the students. There is far greater interaction with students, via E-mail in particular, which is yet another factor increasing the time spent on a course. Those students who are having a difficult time, or who believe that getting some work excused is equivalent to having done the work, take up a far greater proportion of the instructor's time. Some students seek assistance when it is clear that very little studying has occurred. This additional work is a source of dissatisfaction for many faculty. Others who perceive the added work are not interested in adopting the new technology.

For faculty excited by the new opportunities, there have been many rewards. These include

IV. SUMMARY

We believe that at MSU, ALN can and soon will be a significant part of the educational experience for a majority of students. This will be helped considerably if, as the number and variety of more sophisticated technical tools become available, faculty are assisted in becoming skilled in their use and the increased workload is kept in check. Broad implementation will also be assisted if faculty and administrators develop means to deal with the increasing number of conflict situations where their functions overlap.

ACKNOWLEDGEMENTS

We would like to thank Professors W. Lynch, B. Pope, T. Glasmacher and B. Sherrill for the participation and/or comments related to faculty satisfaction. We also appreciate the help of N. Davis during this project. Last but not least, support from the Sloan Foundation has played a key role: it represented an external evaluation of quality and provided resources that allowed us to quickly and broadly implement and experiment with the new network tools.

REFERENCES

ABOUT THE AUTHORS

Edwin Kashy is a University Distinguished Professor at Michigan State University in the Department of Physics and Astronomy. He earned his Ph.D. in Nuclear Physics from Rice University in 1959. He was an NSF post-doctoral fellow at MIT, and then served there as an instructor until 1962. He then joined the faculty as an assistant professor at Princeton University before joining the faculty at MSU as associate professor in 1964. His research areas have been in spectroscopy, Coulomb effects and temperature in atomic nuclei. Since 1992, he has been using technology in his classes and has led the teams at MSU who have developed the networked CAPA (Computer Assisted Personalized Approach) system. He has been assessing the impact of technology in teaching, currently with support from the Alfred P. Sloan and Andrew W. Mellon foundations. Dr Kashy's honors include the John Simon Guggenheim Fellowship, the Distinguished Faculty Award at MSU and the Excellence in Physics Teaching Award; he is also a past CASE Professor of the Year nominee. The work he has done with his colleague on the impact of technology in teaching has been recognized by the ASEE Benjamin J. Dasher Award (98) and the William Elgin Wickenden Award (99).

Contact: Department of Physics and Astronomy, College of Natural Science, Michigan State University, East Lansing, Michigan 48824; Telephone: 517-333-6318; Fax: 517-353-5967; E-mail: kashy@nscl.msu.edu.

Michael Thoennessen is a Professor of Physics at Michigan State University in the Dept. of Physics and Astronomy with an appointment at the National Superconducting Cyclotron Laboratory. He earned his Ph.D. in Experimental Nuclear Physics in 1988 at the State University of New York at Stony Brook. His main research is in nuclear physics where he studies nuclei far from stability. He has been using technology in teaching his classes since 1994, and has been a member of the CAPA development team. His honors include the Outstanding Mentor Award ('94), the T.H. Osgood Award for Teaching Excellence ('95), the Teacher Scholar Award for the College of Natural Science ('96), and the Physics department Outreach Award for his leadership of the Research Experience for Undergraduates program. He is also co-awardee of the ASEE Benjamin J. Dasher Award ('98) and the William Elgin Wickenden Award ('99). His research also include assessment of learning with technology with support from the Alfred P. Sloan and Andrew W. Mellon foundations.

Contact: Department of Physics and Astronomy, College of Natural Science, Michigan State University, East Lansing, Michigan 48824; Telephone: 517-333-6323, Fax: 517-353-5967; E-mail: thoennessen@nscl.msu.edu.

Guy Albertelli is currently a Specialist in Educational Technology at Michigan State University. He received his B.A. in Computer Science at Michigan State University in 1996 and his M.S. in Computer Science from Ohio State University in 1997. He first joined the CAPA development team while an undergraduate student at MSU, contributing the first computer-scored software for optically scanned individualized examinations. Since 1998, He has been the lead programmer and developer for CAPA and his work has resulted in a much more user-friendly system. His current project is the Development of a new ALN tool: LON-CAPA (Learning OnLine Network with a Computer Assisted Personalized Approach). His research also includes assessment of learning with technology with support Andrew W. Mellon foundations.

Contact: College of Natural Science, Michigan State University, East Lansing, Michigan 48824; Telephone: 517-432-5652, Fax: 517-353-5967; E-mail:  albertel@pilot.msu.edu.
 
Yihjia Tsai is currently an Assistant Professor of Computer Science and Information Engineering at Tamkang University in Taiwan, and has been a visiting scholar at MSU for several periods from 1998-2000. He received a B.S. in Mechanical Engineering from the Taiwan National University in 1985. Following a period of work in industry, he earned a M.S. (1995) and Ph.D. (1997) from Michigan State University. He was a member of the initial CAPA development team, was the principal programmer from its inception until 1998, and is actively participating in its current development. His research interests include fundamental aspects of communications in computers, as well as the use of computer technology in education. He is also co-awardee of the ASEE Benjamin J. Dasher Award ('98) and the William Elgin Wickenden Award ('99).

Contact: Computer Science and Information Engineering, Tamkang University, Taiwan; E-mail: tsai@cs.tku.edu.tw.