From Now On
The Educational Technology Journal

 

Vol 5 . . . No 5 . . . January/February, 1996

Evaluating the Impact of Technology:
The Less Simple Answer

by Doug Johnson

 

Goodness, schools have been spending a good deal of money on technology over the past few years. Computers, networks, printers, scanners, file servers, and CD-ROM drives are common sights in most schools - perhaps not in the numbers many students, teachers or parents would like - but certainly in quantities which should suggest this investment is having an impact on education.

Why then is it difficult, if not impossible, to find definitive studies which show the positive impact computers have had on teachers and learning?

Part of the answer lies in understanding that there are three major uses of technology in schools, and our approach to evaluating each use needs to be quite different.

The first use for computers in schools is to enhance professional productivity. Under this category can be lumped everything from * administrative software packages which keep student records, figure payroll, generate state reports, and schedule classes to * e-mail and word processing for improved communication to * specially designed teacher tools like computerized gradebooks, test/worksheet generators, and curriculum templates. I'd predict that using technology to record student progress in greater detail (by objective or benchmark) will be a real growth area over the next few years as states ask for greater accountability from schools for the dollars the schools spend. This category of technology use is what Zuboff1 would call "automating," - doing the same things which have always been done, only more efficiently with technology. The automation of most tasks is almost self-assessing. Either the task is more effectively or efficiently done with computer, or the educator or administrator simply returns to the traditional method of doing it. Professional productivity enhancement is where most educators start using technology to good effect.

The second use is for "automating" instruction using drill and practice software, ILSs, videotaped lessons, computer-animated picture books (Living Books), trivia recall games, low level problem-solving and simulation software (Oregon Trail, Where in the World is Carmen Sandiego, etc.). This is where a good deal of effort has gone in assessing the effectiveness of "educational computing" with poor, or at least mixed results.

My observations lead me to believe that the use of computers in teaching low-level thinking skills, while at times motivational for very young or at-risk students, is very expensive for the results achieved. (My cynical side says this use of educational computing is attractive to administrators who hope that well-designed programmed instruction can overcome the disastrous performance of students with poorly trained or incompetent teachers.) The assessment of this use of technology really has to be an assessment of total student gain of rather low-level thinking skills, and is extremely difficult to do for many reasons (as McKenzie details) - Hawthorne effect, bias of software producers who may be conducting the evaluations, lack of resources for controlled study groups, etc. Unfortunately, the dubious impact of "automating instruction" on education seems to be currently tainting the attitudes of decision-makers about _all_ uses of technology in schools.

That finally leaves us with the final use of technology in schools - as an information processing and productivity tool. The use by students at all grade levels of real-world productivity software like word processors, databases, spreadsheets, presentation programs, multimedia authoring tools, e-mail, video production equipment, digital reference materials, electronic indexes, and network search engines to complete complex, authentic projects is _the_ proper instructional use of technology. Here students will be asked to complete tasks similar to those they will be asked to do in jobs which require using information to solve problems - the kinds of jobs which are both better paying and give greater job satisfaction.

But big challenges present themselves when technology is used on a large scale as an information processing tool. First it requires a good deal more investment in time and effort on the part of teachers in learning how to use it. Anybody can learn to operate Reader Rabbit or Troggles Math in a few minutes, but learning to use a database to store, categorize and sort information can literally take hours of instruction, weeks of practices, genuine effort and guaranteed episodes of pure frustration. A teacher then must spend additional time developing lessons which incorporate the computer productivity skill into their specific subject area.

Second, the product of such instruction is not a neatly quantifiable score on an objective, nationally normed, quickly scored test. Conducting and assessing such projects require the ability to develop and apply standards, delay for long periods of time the satisfaction of task completion, and acknowledgement and acceptance that conclusions, evaluations and meanings which result from the efforts are often ambiguous. (Damn, just like in the real world!) And finally, students need more than the 20-40 minutes of lab access time per week to learn these uses of technology. That means more equipment and software, and making the technology available in more locations (including classrooms and media centers) than if computers are used simply as electronic worksheets of flashcards.

Assessment of this technology use, for me at least, needs to be done not to satisfy a state department, legislature, or academic body, but to inform the students themselves, their parents, and the community in which they live. It means using technology to build personal portfolios of thoughtful, creative work which students and teachers can share with parents; to present worthwhile and authoritative reports to classmates; and to make meaningful contributions to efforts aimed at solving school or community problems. It means being able to determine if the use of technology is making our children better citizens, better consumers, better communicators, better thinkers - better people.

Please write or e-mail comments to djohns1@west.isd77.k12.mn.us.

1. Zuboff, Shoshana. In the Age of the Smart Machine. Basic Books, 1988.
2. McKenzie, Jamison. From Now On, December 1995.

 

Doug Johnson, District Media Supervisor
I.S.D. 77, Mankato Public Schools
Box 8713, Mankato MN 56002-8713
Voice: 507-387-7698
Fax: 507-387-2496
E-mail: palsdaj@vax1.mankato.msus.edu
or: djohns1@west.isd77.k12.mn.us
http://www.isd77.k12.mn.us/staffdir/staff2/Johnson_Doug.html

Copyright Policy: Materials published in From Now On may be duplicated in hard copy format if unchanged in format and content for educational, non-profit school district use only. All other uses, transmissions and duplications are prohibited unless permission is granted expressly. Showing these pages remotely through frames is not permitted.
FNO is applying for formal copyright registration for articles.


Network 609