The Partnered Learning Project

Project Handbook

A Guide to Early Planning for the Development of an Interprofessional Collaboration and Education Program in a Healthcare Setting

7.0  Monitoring, Evaluation & Reporting

Monitoring

Questions to Consider

Our Story

1. Early workplan development revealed that sharing certain human resources across all implementation sites necessitate sequential vs. simultaneous implementation reducing the available preparation time for at least one site.  

   Once it was evident than implementation could not happen simultaneously at each site, the team agreed to a expedite implementation at least one site and seek expedited review of all REB applications to ensure all project activities would be completed within the available time.  A schedule of regular meetings was established for the full length of the project.  A standard format was adopted for both agendas and minutes.  The format ensured regular monitoring of all activity areas and was designed to highlight all decisions and task assignments.



2. Where institution-specific processes and requirements were shared with the team early in the process the implementation planning maintained better momentum and proved more efficient.  Distance and the lack of pre-existing collaborative relationships made it more difficult to jointly assess progress.

   Increased use of technology for information sharing, virtual participation in meetings (via teleconference or video-conference), more frequent in-person contacts, and the establishment of a project liaison role increased the successful exchange of information with the geographically distant partner organization and helped move implementation activities ahead.



3. To ensure success it is essential to have a formal monitoring strategy in place.

   We implemented pre, mid and post placement meetings with the co-facilitators.  They served as a way to check in with the process, ensure that the placement was set-up and running well, and to provide insight and strategies for improvement.  Facilitators were also encouraged to debrief with each other informally during the placement to plan for the subsequent sessions.



4. We include both formal and informal monitoring strategies to monitor student, co-facilitator, and overall IPE placement progress.  We felt this was essential to keep promote the success of the IPE placement and facilitate IPE learning in our students.

   Formal monitoring strategies for the co-facilitators included the implementation of pre, mid and post placement meetings.  They served as a way to check in with the process, ensure that the placement was set-up and running well, and to provide insight and strategies for improvement.  During these meetings facilitators were encouraged to debrief with each other informally after each tutorial.  Guiding questions include “What went well?”, and “How can we improve for next time?”

   For the students several strategies were implemented.  During the introductory tutorials students were engaged in a discussion about how they would like to provide feedback.  We felt that his encouraged them to be engaged in the learning process and promote self-direction in the group.  Several groups incorporated a “Check in, check up, check out” system at the beginning of each tutorial to informally monitor the students’ perceptions of the process.

to top

Evaluation

Questions to Consider

Our Story

1. The implementation of the program began before all sites were fully engaged in the project.  The learning objectives for the workshops were adjusted to reflect each organization’s environment and culture as these emerged through the life of the project.  The workshops generated some valuable information that was not fully anticipated when the initial evaluation planning was completed.

   Both a structured and an open form of data collection were added to the program to gather important information resulting from the team workshops.  The inclusion of an open data collection component permitted the evaluation to focus on the specific learning goals of the session while still maintaining some consistency across all workshops.



2. The research method of this project was iterative and hence the evaluation data collection focus had to continue to be adjusted to maintain consistency with the emerging program enhancements.

   Data analysis and reporting discussions were expanded from the original planned approach to incorporate the data and experiences of later implementations and to ensure that the changes over time were fully considered in the data analysis work.



3. Evaluations for the students and the preceptors were based on examples in the Office of IPE toolkit.  The completion and return the evaluations by preceptors presented an ongoing challenge.  In spite of follow-up communications and reminders the preceptor evaluation return rate remained low.

   Focus groups with the preceptors and the clinical team may be a useful strategy to gather information from otherwise busy schedules.



4. Both quantitative and qualitative evaluations were used to elicit all important aspects of the project.

   The clinical team was not asked to provide an evaluation of the IPE placement experience.  This has since been recognized a valuable data collection opportunity for future consideration.



5. Both quantitative and qualitative evaluation data were used gathered about the IPC placement.  Qualitative questions for the student and the preceptor evaluations were adapted from the Office of IPE toolkit.  Students completed both pre- and post-placement evaluations to assess learning.  The Leucht survey (1990) was used with the students.  We encountered challenges in getting the preceptors to complete and return the evaluations.  Even after several follow-up communications and reminders a low percentage of preceptor evaluations were returned.

   An enhancement was the addition of individual interviews, conducted by the research assistant, with the preceptors and the clinical team members to gather their views on the student placement experience.  An alternative method to gather this type of information would be to hold a focus group meeting with staff and/or preceptors.  

   The clinical team as a whole was not asked to provide an evaluation of the IPE placement experience.  This was recognized as a gap that should be addressed in a future implementation.

to top

 Reporting & Feedback

Questions to Consider

to top

Our Story

1. The PLP project was required to submit quarterly status reports.  Organizational policy dictated that the finance department complete their section and file the report directly to the recipient.  Co-ordinating project reporting with other departments required extra time and communications.

   Through sharing information about the funder’s reporting requirements and negotiation with the finance department, the report project co-ordinator arranged that both sections of the report would be produced simultaneously and then submitted together by the finance department.  This strategy of co-ordinated preparation of both the financial and narrative portions of the reports resulted in less total lead time being required to prepare the quarterly reports.



2. The project team identified several potential audiences that would benefit from hearing and retaining a record of the insights and lessons learned from the project in relation to planning IPC/IPE programs in the future.  Midway through the project the steering committee decided it should formalize this feedback so that the knowledge would survive in an organized way beyond the life of the PLP project.

   A standard institutional report format was developed to efficiently capture and present the educationally relevant knowledge gained through the project implementation.  Representatives from each institution helped determine the most appropriate method of sharing this information within their institution.



3. Clinical teams that participated in the study asked for feedback from the project team.  This level of t4eam interest in feedback had not been fully anticipated during the initial planning stage.

   Insights gained from observing specific teams and facilitating their participation in the team workshop were recorded in the institutional feedback report.  The team considered for each institution the most appropriate way of sharing this information with the team members who participated in the workshops.  The strategies for providing the feedback ranged from email transmission to convening a final team meeting, making a short presentation and facilitating a final group discussion on the experience.  In each case the format was customized to reflect the expressed level of interest and availability of team members to attend a feedback session.  All feedback sessions were held near the end of the project to maximize the contextual information from the project as a whole that could be drawn upon when providing the feedback.

to Research