Reserve advance tickets and save $2 per person on Museum Entry.

Developing YOUth! Project

Measuring the long-term impact of STEM-based out-of-school time programs

February 13, 2017

The Foundation of Developing YOUth!

Posted by Aaron Price on February 13, 2017

The genesis of this project is rooted in planning for a study we ran in 2014–2015 to answer the “where are they now?” question of alumni of our adolescent development programs known as the Science Minors and Achievers1. One of the most surprising findings of that study was this:

Figure 1. STEM career interests while in the program (blue) and after they graduate from the program (yellow)

That is, female participants reported a greater increase in STEM career interest while in the program than compared to male participants. This is pretty exciting because it is well established in the research literature that high school female students tend to lose interest in STEM careers at a quicker rate than male students. So our program may hold a key to reversing that trend.

However, the study was based on a retrospective longitudinal design. In this case, it means we asked alumni to remember what they thought and felt going back 1–10 years. There are various strengths and weaknesses of such a design. The strength is that with maturity and experience, people are better able to compare memories. For example, their definition of what “STEM” means is more stable now than it probably was in high school and college. A downside is that, well, it is memory. Psychologists (and family dinner tables) have long established how poor memory is as an objective record keeping device. Another downside is that we had no control group. How do we know the female students in the program weren’t already interested in STEM careers prior to their involvement? Maybe they would have shown the same growth in STEM career interest just from maturation and aging?

The Developing YOUth! Project is designed to address those issues. By using a forward longitudinal design, we are following teens as they leave the program and enter a new life stage. For most that means going to college, but to some it may mean the military or heading straight into a career. We ask them questions, interview them and basically follow them in real time as they chart their future. Also, we have a control group of like-minded teens who did not join the program. So we can compare their futures and try to isolate what may be different between the program alumni vs. the general population.

We have high ambitions for the project. While we technically are funded to follow the teens for up to five years, we hope to find a way to follow them throughout life (The Framingham Heart Study, now going into its 68th year, is one of our dream models). We are also using a wide variety of research methods including online and paper tests, instant message surveys and other quantitative data alongside qualitative methods that include participatory action research and deep immersion ethnography (also known as participant-observation research). 

We will update this blog periodically to describe various aspects of the project in more detail. We will also use it to share early results. The goal is to make the project as transparent and responsive as possible to the community we serve. Stay tuned!

1 An overview of the results have been published in the July/August 2016 issue of Dimensions magazine. Details will be published soon in a white paper on this website, and an academic deep dive on one particular aspect of the results is in the last stages of preparation for submission to a research journal.