Review of Measuring in Schools: How to Know if What We Are Doing Is Making a Difference, by Victoria L. Bernhardt
Review of Measuring in Schools: How to Know if What We Are Doing Is Making a Difference, by Victoria L. Bernhardt, 2017. Alexandria, VA: Association for Supervision and Curriculum Development, paper.
Victoria Bernhardt is an experienced evaluator of programs and instruction in schools. In this book, Bernhardt lays out all the key concepts for evaluation and also provides good checklists and protocols for how to evaluate results in schools. The language is very clear, and the examples are very basic.
The main thrust of the book is what to do about test results, really. Although the author does not provide any hard details about differences in data or how to analyze data, she does present basic and understandable discussions of disaggregation of data and other important processes that come after data has been collected. Entire teams of educators with little or no experience in statistics and data analysis could make good use of this book—as long as they are being led by someone who has that technical background.
The data is mainly quantitative in nature, though again results are talked around and there are no complicated graphs, bell-shaped curves, or arguments about which kind of Z or z educators should focus on in continuing improvement processes to get the whole school moving in a positive direction. Without giving away all the content, I will admit this is overall a clear and accessible book for groups of educators who need to deal with results and the processes for improving them. The above having been said, it is clear the author works on the quantitative side of evaluation.
Bernhardt confuses qualitative data with data that is the result of qualitative methods. On pages 54, 55, and 59 she shows this confusion by speaking as if they are the same thing. They are not.
Here is an example. Holding a focus group is making use of a qualitative method for gathering data. She calls this a school evaluation process rather than a method. Okay, well, the data coming from that is not necessarily “qualitative.” In fact, that is rather unlikely. Qualitative data (e.g., female, Asian, single parent) is categorical data. It is NOT simply the result necessarily of a qualitative method event like a focus group.
This confusion of qualitative data and data from qualitative methods is a pretty important weakness in this book, and it is important enough it makes me reluctant to recommend this text for use in classes or professional development. I dislike recommending books with such errors because even if the presenter, professor, consultant, etc., goes to the trouble of warning of the errors, chances are most readers of the book will not ever hear the warning.
It might be a kind of “shorthand” to use the term qualitative data this way. I have heard it before. However, a book about program evaluation should not use this shorthand approach when talking about data and methods that are at the heart of the topic being discussed. This confusion perpetuates the lack of understanding of qualitative research and qualitative data.
For clarity, the book gets a 9 out of 10 for overall explanation. It also gets a 9 out of 10 for the discussion of technical areas like disaggregation. It gets a 10 out of 10 for examples and flowchart design for how the evaluation should proceed. However, the book gets a 0 out of 10 for the discussion of qualitative methods and qualitative data. This is an important weakness.
Because of that weakness, I cannot recommend the book for classes, workshops, or professional development sessions.
Thomas Hansen, Ph.D., is an Independent Consultant with a variety of roles in Advocacy, Education, and Social Justice. He is involved in Grant Writing, teaching others to Write Grants, and serving as a Grant Reviewer in federal programs. He has written well over 100 articles, book reviews, and essays. He often teaches as an adjunct in nearby colleges and universities.