Evaluation Research: Some Problems and a Possible Solution
Keywords:
evaluation research, program evaluation, educational policy, decision-making, student government, evaluation methodologyAbstract
Much evaluation research purports to identify the goals of a program and assess the degree to which these goals are being achieved. It is the position of this paper that usually goals are not identified by the researcher but, rather, are created by the researcher. It is argued that this process increases the probability that negative evaluation findings will be ignored, and creates potentially serious difficulties when findings are taken seriously. Moreover, the typical process of educational evaluation research has the potential (a) to usurp the authority of legitimate policy makers by permitting researchers to choose program goals, and (b) to impose a homogeneous set of values on a number of programs even when these programs have a legitimate claim to diverse sets of values. In response to these and other problems, the authors developed an alternative evaluation design. This technique is described, and some of the results of its first application in an evaluation of the effectiveness of student governments in 37 secondary schools in Ontario are presented.
References
Alexander, W. E., & Farrell, J. P. Protests grow for student participation in high schools. School Progress, 1973, 42, 22-24.
Alexander, W. E., & Farrell, J. P. Student participation in decision-making. Toronto: The Ontario Institute for Studies in Education, 1975.
Bacon, F. [Novum organum] (D. Bronstein, trans.). Basic problems in philosophy (2nd ed.). Englewood Cliffs, N.J.: Prentice Hall, 1956. (Originally published, 1620.)
The Board of Education for the City of Toronto. Policy decisions of board 1969 to 1973, inclusive. Toronto: Board of Education, Administrative Services Department, 1974.
Buck, V. The organization as a system of constraints. In J. D. Thompson (Ed.), Approaches to organization design. Pittsburgh: The University of Pittsburgh Press, 1966.
Campbell, D. T. Assessing the impact of planned social change. Paper presented at the OECD seminar on Social Research and Public Policies, Dartmouth, N.S., September 12-15, 1974.
Carter, R. K. Clients' resistance to negative findings and the latent conservative function of evaluation studies. The American Sociologist, 1971, 6, 118-124.
Doby, J. T. Logic and levels of scientific explanation. In E. F. Borgatta (Ed.), Sociological methodology 1969. San Francisco: Jossey-Bass, 1969.
Eide, K. The planning process. In S. Elam & G. Swanson (Eds.), Educational planning in the United States. Itasca, Illinois: Peacock, 1969.
Etzioni, A. Modern organizations. Englewood Cliffs, N.J.: Prentice-Hall, 1964.
Gross, B. What are your organization's objectives? A general systems approach to planning. Human Organization, 1965, 18, 195-216.
Hall, F. Organizational goal determination in public schools. Unpublished doctoral dissertation, University of Toronto, 1975.
Homans, G. C. Social behavior: Its elementary forms. New York: Harcourt, Brace and World, 1961.
Kubler-Ross, E. On death and dying. New York: Macmillan, 1973.
McPartland, J., et al. Student participation in high school decisions: A study of 14 urban high schools. Washington, D.C.: U.S. Department of Health, Education and Welfare, 1971.
Mohr, L. B. The concept of organizational goal. American Political Science Review, 1973, 67, 470-481.
Perrow, C. The analysis of goals in complex organizations. American Sociological Review, 1961, 26, 854-866.
Provincial Committee on Aims and Objectives of Education in the Schools of Ontario, Report of the Committee. Living and learning. Toronto: Ontario Department of Education, 1968.
Rossi, P. Boobytraps and pitfalls in the evaluation of social action programs. In Proceedings of the Social Statistics Section of the Annual Meeting of the American Statistical Association. Washington, D.C.: American Statistical Association, 1966.
Scriven, M. Pros and cons about goal-free evaluation. Evaluation Comment, 1972, 3, 1-4.
Scriven, M. Goal-free evaluation. In E. R. House (Ed.), School evaluation: The politics and process. Berkeley: McCutchan, 1973.
Simon, H. On the concept of organizational goal. Administrative Science Quarterly, 1964, 9, 1-22.
Stewart, G. On looking before leaping. In Social indicators: Proceedings of a seminar. Ottawa: Canadian Council on Social Development, 1972.
Suchman, E. H. Evaluation research. New York: Russell Sage Foundation, 1967.
Worthen, B. R. A look at the mosaic of educational evaluation and accountability. Portland, Oregon: Northwest Regional Educational Laboratory, Paper Series, No. 3, 1974.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Canadian Journal of Education follows Creative Commons Licencing CC BY-NC-ND.
