Our external evaluators, M. Jean Young and Susan Loucks-Horsley, provided feedback during the first two years of the project for the purpose of program improvement. During the last year, they focused on program effectiveness. These summative data are meant to inform those interested in implementing the VINE Follow-Through professional development model. The evaluation focused on two questions:
What strategies were effective in forming the teams and engaging them in designing Follow-Through learning experiences?
and
What is the impact of these strategies on the teachers and to what degree is change happening in their classrooms?Data collected to assess program effectiveness came from site visits to the VINE Summer Institutes, interviews, document reviews, and pre-coded teacher logs. Teacher participants at three evaluation sites and several non-participating colleagues in their schools or in comparable schools completed teacher logs. The evaluators compared teacher-participants with a matched sample of non-participants in a control-treatment study. This enabled the evaluators to determine program effects related to classroom teaching.
What the Evaluators Learned About
Team Building StrategiesAs described throughout the narratives in this publication, strategies emerged that were particularly useful in building teams. The evaluators’ findings included
1) Teams had the best success in promoting collaboration when they did the following:
- Had a goal or specific focus. For example, Kentucky’s performance event provided a focus for its team. The Seattle Team briefly floundered until it found a focus in their second year;
- Integrated project activities with district and state initiatives;
- Addressed personal as well as district, school, and classroom issues;
- Combined team building and task-oriented activities;
- Worked through growth cycles together; and
- Engaged in shared decision making.
2) Strategies most effective in promoting teacher leadership and teacher professionalism included team egalitarianism, practice leading meetings, and shared decision making. Many teacher-participants reported that they felt a greater sense of professionalism collaborating as equals with district and university educators.
3) Strategies most effective in helping teams extend VINE investigations revolved around promoting risk-taking that included
- mutual support;
- sharing information, resources and materials; and
- positive reinforcement by meeting regularly during the school year.
4) Teachers attributed their progress in using constructivist strategies to risk-taking, team support, and positive feedback. Teamwork promoted their impetus to change — trying out new things, sharing failures as well as success.
5) Constructivism as a guiding philosophy had drawbacks as well as positive outcomes. Teams creating their own understandings together led to egalitarianism, risk-taking, and other positive outcomes listed above. There were occasions, however, when constructivism was not effective, such as when team members who were experts on a topic or subject did not or were not able to exert their authority when expediency was needed. For example, a science educator on one team recognized that several teachers had limited science content knowledge and reported feeling powerless to help correct the situation because it would require direct intervention. The team was already operating under time limitations ruling out the possibility of engaging in professional development to increase subject matter content knowledge. The discussion and buy-in required by constructivism takes a great deal of time.
What the Evaluators Learned Regarding
Changes in Teacher PracticeThe evaluators assessed classroom practice using a variety of teacher log measurements that included what teachers were doing, what students were doing, how students were assessed, how lessons were introduced, and what materials were used.
For virtually every measure, teacher-participants used more "best practice" strategies, as advocated in the National Science Education Standards, than their colleagues. The differences in the two groups are particularly meaningful since the control group (teacher-participants’ colleagues) also exhibited many "best practice" techniques. The largest differences in "best practices" used by teacher-participants were that they
- had their students work on more long-term projects;
- engaged their students in more out-of-class activities; and
- used more laboratory equipment.
The control group tended to emphasize more traditional practices (such as lectures, recitations, seatwork, and textbook strategies) that if overly used are considered to be less effective. The largest differences in traditional practices used by teacher-participants’ colleagues (i.e., control teachers) were that they
- placed more emphasis on learning facts or definitions;
- lectured more;
- had their students complete worksheets more often or do more practice problems in class; and
- had their students do more classwork from the assigned textbook, workbooks, and worksheets.
While emphasizing constructivism created positive experiences and promoted "best practice" classroom strategies, teachers struggled with implementing the "minds-on" part of constructivism. This was, in part, due to teachers’ expressed understanding of constructivism as "students taking more control over their own learning." Less attention was paid to teachers guiding students in constructing their own understandings from their learning experiences.
What the Evaluators Presented as Their Overall Impressions
The evaluators concluded that the VINE Follow-Through Project had provided an invaluable opportunity to reflect on professional development activities and that participants developed useful strategies that helped teachers make changes in their classrooms. The evaluators also noted the following:
1. Progress on two project goals (building the capacity for ongoing change in the participants, and changing the learning experience for students) was apparent.
2. A model of professional development emerged that had shared features in the three school districts involved in the project, yet was tailored to each district.
3. There was active involvement of non-teacher participants. The project acknowledged how important it is for teachers to initiate and follow-through on important classroom changes but that they cannot do it alone.
4. The project continued to try to balance accomplishing tasks and building relationships.
From Changing What We Do by Karen Hollweg and Carole Kubota with Phyllis Ferrell, copyright 1998 by NAAEE
Excerpt Top "Changing..." Top Order Form NAAEE Home