Inventory and Assessment of K-12 and PTD Programs

Inventory and Assessment of K-12 and Professional Teacher Development Programs in the National Estuarine Research Reserve System

E. RESULTS

Program Evaluation and Performance Measurement

Approximately four Reserves reported that K-12 is not a focus at their Reserve or that they are creating new programs or redesigning their older programs. There are approximately eight Reserves where PTD is not a priority and very little to no programming occurs. Thus, at the time of the study no program evaluation is occurring at these sites.

As evidenced from the K-12 and PTD program summary section there is little formal evaluation occurring for most programs. The most common forms of evaluation are staff observation and tracking participant attendance counts. Thirty-one percent of the K-12 programs report having no formal evaluation, and only a quarter of the programs have any type of follow-up surveys, pre- and post-program surveys, or even feedback cards. For PTD programs, 22% of the programs have no formal evaluation and less than half have follow-up surveys, pre- and post-program surveys, or feedback cards.

The responses to the question “For Reserve programs that have been formally evaluated what are the results of the evaluations?” indicate that many programs do not have formal evaluation results. There is confusion about the definition of formal evaluation (noted in the methodology section.). Some believe that formal evaluation is done by an outside party and others think that it is an evaluation with quantitative results. For the purpose of the study, we are looking for quantitative results. The main reasons for the lack of formal evaluation are:

Evaluation takes time away from other education responsibilities
ECs indicate that they do not have a lot of experience evaluating programs (72% rated this skill a 5 or less on a 10 point scale.)

The responses to the question “Overall, do you think that K-12 and PTD programs at your Reserve were successful?” indicate that most ECs believe their programs are successful enough based on subjective evaluation methods including monitoring the number of repeat visits, anecdotal feedback from teachers via discussion groups, individual conversations, focus groups, and meeting their capacity for K-12 programs.

Three Reserves mentioned that they conduct formal evaluations and have the data to prove their success, and that the informal feedback they have received from teachers, parents, and science supervisors are positive.

From the responses to the telephone survey, the success of professional teacher development is mixed. In many cases the Reserves feel that it has been more difficult to recruit teachers in the past couple of years because budget cuts and the increased emphasis on meeting state standards decreases the amount of time that teachers spend away from the classroom for professional development. Teachers also want stipends for participation in workshops. Many Reserves believe they offer high quality programs but that the factors listed above hinder their success. Some ECs express the need for marketing help at the local level to increase the visibility of Reserve programs, particularly in the case of PTD.