This paper proposes a nonparametric method for evaluating treatment effects in the presence of both treatment endogeneity and attrition/non-response bias, using two instrumental variables. Making use of a discrete instrument for the treatment and a continuous instrument for non-response/attrition, we identify the average treatment effect on compliers as well as the total population and suggest non- and semiparametric estimators. We apply the latter to a randomized experiment at a Swiss University in order to estimate the effect of gym training on students' self-assessed health. The treatment (gym training) and attrition are instrumented by randomized cash incentives paid out conditional on gym visits and by a cash lottery for participating in the follow-up survey, respectively.
A growing number of school systems use self-report surveys to track students’ social-emotional development as a tool to inform policy and practice. We use the first large-scale panel survey of social-emotional learning (SEL) to simulate how four constructs—growth mindset, self-efficacy, self-management, and social awareness—develop from Grade 4 to Grade 12 and how these trends vary by gender, socioeconomic status, and race/ethnicity among students participating in the survey for two consecutive years. With the exception of growth mindset, self-reports of these constructs do not increase monotonically as students move through school; self-efficacy, social awareness, and to a lesser degree self-management decrease after Grade 6. Female students report higher self-management and social awareness than males, but lower self-efficacy relative to males in middle and high school. Economically disadvantaged students and students of color report lower levels of each construct. These patterns highlight the need for policymakers to interpret changes in students’ self-reports over time in light of normative trends in social-emotional development and illustrate how such self-reports may nonetheless be used to set priorities and target interventions and resources.
Text-message based parenting programs have proven successful in improving parental engagement and preschoolers’ literacy development. This study seeks to identify mechanisms of the overall effect of such programs. It investigates whether actionable advice alone drives previous study’s results and whether additional texts of actionable advice improve program effectiveness. The findings provide evidence that text messaging programs can supply too little or too much information. A single text per week is not as effective at improving parenting practices as a set of three texts that also include information and encouragement, but a set of five texts with additional actionable advice is also not as effective as the three-text approach. The results on children’s literacy development depend on the child’s pre-intervention literacy skills. For children in the lowest quarter of the pre-treatment literacy assessments, providing one example of an activity improves literacy scores by 0.19 standard deviations less than providing three texts. Literacy scores of children in higher quarters are marginally higher with only one tip per week than with three. We find no positive effects of increasing to five texts per week.
Text-message programs are increasingly popular as low-cost interventions aimed at improving a variety of health and education outcomes. This study analyzes participant opt out decisions from a set of text messaging programs aimed at fostering parent-child interactions and improving school readiness. Exploiting random assignment of parents of young children to programs and rich data on text messages and recipients, we examine how program design and text and recipient characteristics predict program opt out. The results provide evidence that the text messaging programs reach the parents of traditionally less-resourced children and show that program design affects parent opt out. Programs that provide context and encouragement along with activities reduce opt out compared to programs that send activities alone. A high quantity of texts and more complex texts lead recipients to opt out at greater rates.
What are the effects of on-campus recreational sports and exercise on educational outcomes of university students? We randomize financial incentives to encourage students' participation in on-campus sports and exercise in two cohorts of college freshmen. Incentives increased participation frequency by 47% and improved grades by 0.14 standard deviations in the first cohort. The incentives were less effective in promoting participation and did not improve grades in the second cohort. In the first cohort, students primarily substituted off-campus with on-campus physical activities and appeared to be better able to integrate exercising with studying. Grades appear to have improved because of an enhanced effectiveness of studying and because students spend more time in class.
This study analyzes the effects of an increase in the cost of going to college on student finances and achievement. It exploits a unique policy change at a Swiss university whereby students faced an unexpected increase in tuition fees. This increase differed across students. The study uses this variation in a difference-in-differences strategy to identify and estimate the causal effect of the differential increase in fees. Results based on survey data suggest that students compensate the increase in fees with a reduction in consumption spending. In line with this finding, the estimated effects on on-time graduation, credits earned, and grades, based on data from administrative student records, are close to zero and insignificant.
This study investigates how exposure to a field of study influences students’ major choices. If students have incomplete information, exposure potentially helps them to learn about the scope of a field as well as how well the field matches their interest and abilities. We exploit a natural experiment where university students have to write a research paper in business, economics, or law during their first year before they choose a major. Due to oversubscription of business papers, the field of the paper is assigned quasi-randomly. We find that writing in economics raises the probability of majoring in economics by 2.7 percentage points. We show further that this effect varies across subfields: the effect is driven by assignment to topics less typical of the public's perception of the field of economics, suggesting students learn through exposure that the field is broader than they thought.
This paper discusses identifcation based on difference-in-differences (DiD) approaches with multiple treatments. It shows that an appropriate adaptation of the common trend assumption underlying the DiD strategy for the comparison of two treatments restricts the possibility of effect heterogeneity for at least one of the treatments. The required assumption of effect homogeneity is likely to be violated because of non-random assignment to treatment based on both observables and unobservables. However, this paper shows that, under certain conditions, the DiD estimate comparing two treatments identifies a lower bound in absolute values on the average treatment effect on the treated compared to the unobserved non-treatment state, even if effect homogeneity is violated. This is possible if the treatments have ordered treatment effects. That is, in expectation, the effects of both treatments compared to no treatment have the same sign, and one treatment has a stronger effect than the other treatment on the respective recipients. Such assumptions are plausible if treatments are ordered or vary in intensity.
Against the backdrop of high macroeconomic instability and the need to meet the demands of public spending, we analyze the trade-off between growth and volatility of tax revenues in Latin America. Short-run and long-run elasticities for a sample of 11 economies are estimated accounting for state-dependent asymmetric reactions. Controlling for composition of revenue sources and other idiosyncrasies, we find revenues above (below) its long-run equilibrium to react stronger (weaker) to business cycle dynamics. Our detailed elasticity estimates can give some orientation on how to stably reach higher tax levels on the way to develop an adequate internal tax system.
Welchen Stellenwert die Evaluation aktiver Arbeitsmarktpolitik in der Wahrnehmung zumindest der (politischen) Fachöffentlichkeit inzwischen hat, zeigt sich daran, dass die im Koalitionsvertrag zwischen CDU, CSU und FDP aus dem Jahr 2009 (CDU/CSU/FDP 2009) enthaltene Klausel, dass die arbeitsmarkt politischen Instrumente „auf den Prüfstand gestellt“ werden sollen, ganz automatisch in einen umfassenden Evaluationsauftrag übersetzt wurde. Sicherlich ist Arbeitsmarktpolitik heute eines der Politikfelder mit dem höchsten Maß an wissenschaftlicher Begleitung und Diskurs zwischen Wissenschaft und Politik. Evaluation steht im Dienste einer evidenzbasierten Politik.
The time children spend with their parents affects their development. Parenting programs can help parents use that time more effectively. Text-messaged-based parenting curricula have proven an effective means of supporting positive parenting practices by providing easy and fun activities that reduce informational and behavioral barriers. These programs may be more effective if delivered during times when parents are particularly in need of support or alternatively when parents have more time to interact with their child. This study compares the effects of an early childhood text-messaging program sent during the weekend to the same program sent on weekdays. We find that sending the texts on the weekend is, on average, more beneficial to children’s literacy and math development. This effect is particularly strong for initially lower achieving children, while the weekday texts show some benefits for higher achieving children on higher order skills. Results are consistent with the hypothesis that the parents of lower achieving students, on average, face such high barriers during weekdays that supports are not enough to overcome these barriers, while for parents of higher achieving students, weekday texts are more effective because weekdays are more challenging, but not so difficult as to be untenable for positive parenting.
School value-added models are increasingly used to measure schools’ contributions to student success. At the same time, policymakers and researchers agree that schools should support students’ social-emotional learning (SEL) as well as academic development. Recent attempts to measure schools’ influence on students' SEL show differences across schools, but whether these differences measure the true effect of schools is unclear. We examine the stability of school-by-grade effects on students' SEL across two years using a large-scale SEL survey administered in California’s CORE districts. We find that correlations among school effects in the same grades across different years are positive, but they are lower than those for math and English Language Arts (ELA). Schools in the top or the bottom of the school effect distribution are more persistent in their impacts across years than those in the middle of the distribution. Overall, the results provide evidence that these school effects measure real contributions to SEL. However, the low stability of effects from one year to the next draw into question whether including these school value-added measures of self-reported SEL in school performance frameworks and systems would be beneficial.
Teachers play a critical role in establishing classroom and school environments that contribute to students’ social and emotional development. This paper explores whether we can estimate a classroom-level measure of student growth in SEL by applying valueadded models to students’ SEL. We analyze data from the 2016 and 2017 administrations of student self-report surveys, which contain responses from roughly 40,000 students in Grade 5 within five of California’s CORE Districts. We estimate separate value-added models for each of the four SEL constructs assessed—growth mindset, selfefficacy, self-management, and social awareness—and for math and ELA academic growth. We find across-classroom-within-school variance of students’ SEL outcomes, even after accounting for school-level variance. The magnitude of classroom-level impacts on students’ growth in SEL appears similar to impacts on students’ growth in ELA and math, although the growth models of SEL do not perform as well as growth models of academic outcomes. Results suggest that across-classroomwithin-school impacts may be larger in magnitude than across-school impacts on students’ SEL growth. Finally, we show that there are generally low correlations between classroom-level growth in SEL and classroom-level growth in ELA or math; however, growth mindset stands apart from the other three SEL constructs in that there is a moderately strong relationship. By assessing whether we can develop a sound approach for measuring classroom-level impacts on students’ SEL, we aim to contribute to the growing body of knowledge about appropriate and innovative uses of data on students’ non-cognitive and socialemotional learning.