Analyzing Implementation & Impact:
Sustainable Data Sources
- Implementation
- Sustaining College & Career Competency Instruction
- Analyzing Implementation & Impact: Sustainable Data Sources
- Guidance for Individualized Supports
- Measuring Social-Emotional Growth Video Series
For quality implementation and sustainable competency instruction, leadership teams must commit to using data to inform their decisions and determine whether their efforts have the desired effect on students. While impact analysis is vital, implementation fidelity must also be examined in order to self-assess and continually improve our implementation efforts. The Implementation Data Matrix will guide your systematic analysis and refinement of practices.
The Implementation Data Matrix categorizes data from leadership, educators, and students using eight targeted questions designed to shed light on implementation strengths and reveal opportunities for improved support.
How do [leadership teams, educators, students] know …
That they have the knowledge and skills to implement the innovation?
That they are implementing the innovation?
That they are doing what is necessary for implementation often enough?
That they are engaging the right people?
That they are implementing the innovation well enough?
That they are following the process to improve implementation?
How to document adherence to their plan and adjustments to implementation?
That the innovation is working?
Leading Implementation
When implementing any schoolwide initiative, leadership practices create the enabling environment. Leadership teams must develop an implementation timeline that includes professional learning, coaching cycles, collaboration opportunities, and data analysis protocols. Watch the Sustaining Implementation Video Series to learn specific ways to support your staff in implementing competency instruction schoolwide. The eight videos, ranging from 7 to 18 minutes, address foundational issues like preparing your staff for competency instruction and training and coaching your staff as they provide quality instruction.
To set up ongoing data collection and analysis structures that promote and sustain schoolwide implementation, use the Implementation Data Matrix described above. Phrase the eight questions to start with “How will our leadership team know …” and discuss as a team. As shown in the data matrix, leadership teams continually reflect on their implementation, gather data, analyze the effectiveness of support structures, and monitor short-term and distal outcomes. Example data and processes that support leadership teams in self-assessing and reflecting include:
- Prioritized impact data, such as performance-based assessments, attendance, on-time work submission, course performance, and behavior referrals are disaggregated for student populations, reviewed by leadership, and shared with educators. See an example of self-regulation outcomes that schools have measured.
- Self-report data from educators and observation data collected by instructional coaches are aggregated to guide leadership in determining the effectiveness of their support structures.
- The CCC Framework Implementation Roadmap is updated twice per year along with a review of the timeline and sequence of instructional activities. The Roadmap outlines core constructs and guides the celebration of successes and identification of priorities for improved implementation. Watch this 9-minute video to learn more about utilizing the Roadmap.
- Leadership team artifacts, including documentation of data-based decisions, are archived to track adjustments and sustain enabling structures.
Enhancing Educator Practices
The crux of implementation lies in teachers’ integration of the practices into their daily structures. Often educators are judged for their implementation or lack thereof. Instead, student and class-wide data should empower educators and promote a professional community of educators and staff focused on developing resilient learners. As shown in the Implementation Data Matrix, educators reflect on their implementation, analyze the effectiveness of their instruction, and monitor their students’ short-term outcomes. Example data sources and processes that support educators in self-assessing and reflecting include:
- Students’ performance-based observation data are analyzed to determine each student’s growth across time and prioritize class-wide guided practice. Students’ knowledge assessments and artifacts provide evidence of each student’s learning and application of the competency.
- Educators analyze their students’ attendance, behavior, and course performance data along with competency growth to determine impacts.
- Evidence walks, conducted by instructional coaches within coaching cycles, document educator, student, and environmental application strengths. These data guide educator reflection and planning during coaching sessions.
- Educators facilitate guided/independent practice of the competency, and on a quarterly basis, they reflect on the effectiveness and plan for future adjustments.
- Collaborative team artifacts, including strengths of instruction, artifacts from practice opportunities, and adjustments to instruction and practice opportunities, are archived. These data discussions address strategies to overcome barriers and are shared with leadership to determine needed adjustments to enabling structures.
- Midyear and spring self-report surveys promote educators’ reflection on their instruction, guided practice, and students’ growth. These data inform coaching and sustainability planning.
Collaborating With Families
Families are instrumental in building their child’s intrapersonal and interpersonal skills. When viewed as partners, families reiterate the learning that occurs at school and apply the competency strategies at home. Family guidance for each competency builds the collective knowledge and supports meaningful collaboration. Families and educators celebrate their students’ growth and brainstorm ongoing practice opportunities. Examples of collaborating with families around data include:
- Competency data is shared and discussed at parent–teacher conferences. When possible, students present their own data, share the importance of the competency, and identify strategies they want to work on across environments.
- Families determine competency strategies they want to apply at home, and when possible, they share impacts. Information from families informs guided practice at school.
Watch this 13-minute video to learn more about engaging families in developing resilient learners.
Empowering Students
The purpose of implementing the Competency Framework is to develop resilient learners who collaborate to expand skills, express their wants and needs respectfully, and apply strategies to self-regulate and persevere. By doing so, youth become socially and emotionally engaged, career-equipped, lifelong learners. For students to be empowered, they need to self-assess using data and prioritize competency strategies that will be beneficial to practice. Examples of data and processes that help students self-assess and reflect include:
- Student artifacts, completed within instructional activities, include students’ reflection, planning, and checks for understanding.
- Constructive feedback, provided by educators to each student in association with instructional activities and practice opportunities, provides evidence on demonstration of skills. Students use this feedback to reflect on their strengths, growth, and areas for improvement.
- Students complete the Competency Reflection and Knowledge Test prior to and after initial instruction. These data promote individual student reflection and guide educators’ instruction.
- Reflective self-assessment activities on competency application are completed by students three times per year. Through teacher-facilitated planning sessions, students are supported in recognizing their current strengths and opportunities for continued growth in developing the competency.
Next Steps
While implementation fidelity and impact data collection may feel daunting, when we focus on data analyses that inform our ongoing implementation, each data point becomes purposeful and integral to the instructional process. The key is for data to be immediately available and useful—students use data to determine their own growth, families use data to prioritize competency strategies, educators use data to refine instruction and guided practice, and leadership teams use data to determine structures that sustain implementation. Impact data become part of the data analyses, integrated within implementation structures.
The Competency Framework Recognized Trainers are here to help you. Reach out to discuss your current data processes and brainstorm ways to enhance your implementation and impact. In addition to schoolwide professional learning, we offer competency coach training to build the capacity of local coaches to support and sustain implementation. Request a meeting to discuss your support needs.