MCS Presentation

Extending Success:Defining and Evaluating

Excellence in The Met Community School

Extended Day Programming

Mateus Baptista, Lauren Combs, Jesse DeLauder, Gretchen Mills

Wednesday, July 30, 14

Agenda

Context

Research Question

Literature

Design

Methods

Analysis

Findings

Recommendations


MCS

The Met Community School

Mandatory extended day for 9th grade students

Diverse programming

Experiential learning

Founded in 2003


Community Learning Centers

“In 1994, Congress authorized the 21st Century Community Learning Centers (21st Century) program to open up schools for broader use by their communities. In 1998, the program was

refocused on providing academic, enrichment, and recreational activities in public schools during

the after-school hours (centers also could be open before school, on weekends, and during the

summer). The program grew from an appropriation of $40 million in 1998 to $1 billion in 2002, where it

has remained.” (U.S. DOE, 2005)


Research Question

How can the MCS extended day program

measure and track progress towards

their internal goals?


Q1:What are the perceived goals of MCS?

Q2: Comparison of MCS program qualities with

those of high-performing after school programs

Q3: What are available indicators of success

within MCS?

How can the MCS extended day program measure and

track progress towards their

internal goals?


Literature

Qualities of similar high-performing after-school and

extended day program

Successful evaluation of after-school and

extended day programs

Methods of research and evaluation

The Met in Context

Research DesignAnalysisFindings

Recommendations


Qualitative

Quantitative

Interviews with students,

administrators, and extended day providers

Follow up questionnaire

Design


Methods

Interviews QuestionnaireN=11 (Two admin, five student, four provider)

Convenience Sample

Purpose:

1) Address how extended day goals are perceived

2) Compare MCS qualities to those of existing programs

3) Identify available indicators of success

Purpose:

Measure perceived program success towards three MCS goals


Goals

MCS Goals According to Director

Safety

College and Career Readiness

Social and Emotional Learning


Methods

Interview Protocol - Providers and AdministratorsPurpose: We are a research team interested in helping the MCS design an internal evaluation system. To do this, we would like to hear about your personal experience with the extended day program.

You will not be identified by your responses, and they will remain confidential—only to be accessed by our four-person research team.

Is it ok if we record this interview? 1. Please describe your role in the extended day program.

2. What do you see as the goals of the extended day program?

3. How do you know if the extended day program is meeting these goals?

5. What do you see are the strengths and weaknesses of the program?

6. How do you communicate to other staff/teachers within the program? How do you record challenges or successes for yourself?

7. Are there formal or informal opportunities for parental engagement?

8. How long have you worked at MCS?

9. What ethnicity do you identify with?

10. Is there anything else you would like to share that you have not yet had a chance to share?


Methods

Interview Protocol - StudentPurpose: We are a research team interested in helping the MCS design an internal evaluation system. To do this, we would like to hear about your personal experience with the extended day program.

You will not be identified by your responses, and they will remain confidential—only to be accessed by our four-person research team.

Is it ok if we record this interview?

1. Tell us about your experience with the extended day program at MCS.

2. Do you think it is important to have a program like this here in your community?

3. Did you continue on in the extended day program past 9th grade? Why did you do so?

4. How has the program affected your academics? Social life?

5. How is the program similar to what you do at school? How is it different?

6. What would you be doing after school if you weren’t coming to the program?

7. What is your favorite and least favorite part of the extended day program?

8. Do you have any ideas to make the extended day program better?

9. Do people listen when you have an opinion or concern about the extended day program?

10. Is there anything else you would like to tell us?11. What ethnicity do you identify with?12. What grade are you in?13. How many years have you attended the extended day program?


Methods

Follow-Up QuestionnaireThe Met Community School strives to provide its students with a safe environment, opportunities for social/emotional learning, and the skills necessary to succeed in college and career. To what extent do you think the extended day program is successful in achieving these goals?

Safety

College and Career Readiness

Social/Emotional Learning

1

1

1

2

2

2

3

3

3

4

4

4

5

5

5

Not at all successful

Somewhat successful

Very successful

Completed in private by ten of twelve interviewees and returned


Analysis5) Questionnaire: measures of central tendency

Qualitative

Quantitative

Subgoals

Qualities

Indicators

Q1

Q2

Q3

Subgoals

Qualities

Indicators

Subgoals

Qualities

Indicators

Safety College/Career Readiness


4) Post Coding: Frequency counts

1) Transcribe Interviews2) Initial Coding - General3) Final Coding - Categorical


Codes

Room for ImprovementLow participationDisconnect between provider & adminDisorganizationAdd programsMandatory for all gradesNo student input

College/Career ReadinessMultiple PathwaysExtend Learning TimesAcademic ProgrammingBuild Job SkillsNetworkingMCS Connection to Met SchoolInternships

Need for MCS Administration & OrganizationProfessional Development Opportunities

Internal AssessmentsStaff Supports

Diversity of ProgramsFormal/Informal Family Engagement

Staff AccountabilitySchool/Community Partnerships

Provider Anxiety

SafetyKeep Kids Out of Trouble

ParticipationAttendance

Safe EnvironmentCurb Delinquent

Behavior


Formal Student Leadership Opportunities

Emotional GrowthPeer Relationships

MentoringStudent EngagementFinding your Passion

Informal Student Leadership Opportunities

Enrichment Opportunities


Findings

Subquestion #1: What are the perceived goals of MCS?

When asked “what do you feel are the goals of the extended day program?” administrators and providers

responded...

*Students were asked, “Do you think it’s important to have a program like MCS in your community? If so, why?”

Q1

Goa

l Cod

es


Findings

MCS Qualities and Indicators by Goal

Q1

0

10

20

30

40

50

60

70Fr

eque

ncy

Talking About MCS Goals

SAFETY

Participation Attendance

Safe Environment Curb Delinquent

Behavior

COLLEGE AND CAREER READINESS

Academic Programming Build Job Skills

Networking MCS Connection to Met

Vocational Training/ Internships

SOCIAL-EMOTIONAL LEARNING

Student Leadership Emotional Growth Peer Relationships

Mentoring Student Engagement

Enrichment Opportunities


Findings

Perceived Success by Goal

Q1

4.7

4

4.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Safety College and Career Readiness Social-Emotional Learning

Perceived Success Toward MCS Goals


Quality:A feature or component of a high performing after-

school program.

Subquestion #2: Comparison of MCS program qualities with those of high-performing after school programs

AnalysisQ2Wednesday, July 30, 14

Grounded Theory Top-DownAcademic

programming

Enrichment programming

Fostering relationships

Safe environment

Diversity of programs

Participation

School community partnerships

Connection to school

Family engagement strategies

Student leadership

opportunities

Internal evaluation

Staff supports

Experienced leadership

Vocational Training and Internships


Findings

Comparison of MCS program qualities with those of high-performing after-school programs

0

5

10

15

20

25

30

SAFETY

COLLEGE & CAREER READINESS


N/A

Q2Wednesday, July 30, 14

Indicator:“A set of statements about activities

and behavior that can be used to measure the extent to which after-

school programs are providing high-quality opportunities for

participants.” (Providence After School Alliance)

Subquestion #3: What are available indicators of success within MCS?


Analysis

What are available indicators of success within MCS?

When asked about measuring success and positive outcomes, administrators, students, and providers mentioned...

Attendance

Delinquent behavior

Mentorship

Student showcase

Feelings towards school

Networking

Building job skills

Peer relationships

Emotional growth

Student empowerment


Findings

Indicators for Evaluation

0

2

4

6

8

10

12

14

16

18

SAFETY

COLLEGE & CAREER READINESS



Recommendations

Goals for MCS

MCS should set clear goals that

are communicated program-wide.

Recommendation:Mission statement

Finding:Emergent goals were informed

by the three stated goals, but interviewees did not have a clear goal framework.


Recommendations

Stated Goals from Director

Emergent Goals

Qualities

Indicators

Evaluation of MCS programming

should be informed by

emergent goals and best practices,

using the appropriate stated

indicators.


Recommendations

Emerging Goals Qualities Indicators


Find your passionEnrichment opportunities

College/Career Readiness

Extend classroom learning

Safety

Foster relationshipsDiversity of enrichment programsStudent Leadership (formal and informal)

Academic programmingInternshipsConnection to MetTrying something new

MentoringPeer relationshipsEmotional growthStudent engagement

NetworkingJob skillsFeelings towards schoolsStudent showcase

ParticipationSafe environment

Delinquent behaviorAttendance

Evaluation Measurements

Expectancy of successEffectance motivationEvaluation of students by provider and mentorFocus groups

Longitudinal research of college enrollment, job placement, and program satisfaction

Drop out ratesGraduation ratesStudent pre- and post- testsAssessment of student showcaseAttendance recordsStudent pre- and post- testsFocus groups


Limitations

Self- and organization-preservation bias

Limits of internal validity

Limited pre-existing data

Convenience sampling

Limits of external validity: unique situation

Summer Program

Small sample size


Next Steps3)Study on the overarching community effects

1)Study

performed during

extended day programming

2) Design evaluation tools


“So the strengths are just opening up so many doors of opportunity for these kids [who] may come from a background where they felt they wouldn’t amount to anything. Or they always could but they didn’t know how. It’s opening these doors so they can see which direction

they may want to go, or figure out something about themselves they didn’t know.

It’s just self-discovery.”-Provider & Former Student


MCS Presentation

Documents

Transcript of MCS Presentation