Three Essays on Policy Evaluation

Abstract

Over the last two decades there has been a proliferation of literature on program evaluation. Many researches in economics look at the causal effect of exposure of units to programs on some outcomes through econometric and statistical analysis. The units are typically economic agents such as individuals, households, markets, firms, counties, states or countries. The programs can be job search assistance programs, educational programs, vouchers, laws or regulations, drug therapies, environmental exposure or technology shocks. Rubin potential outcomes framework seems to be the dominant framework in which the aim is to compare the two potential outcomes for the same unit when he or she is exposed and not exposed to the program (or treatment)1. However, each unit can be only exposed to one levels of program: an individual may enrol or not in a training program or he (or she) may be subjected or not to policy. We can refer to this as the fundamental problem of causal inference (Holland, 1986; Imbens andWooldridge, 2008). The impossibility to compare the same individual at different treatment status induces to resolve the issue thinking in term of counterfactual. We need to compare distinct units at different levels of treatment. This means to compare different physical units or the same physical unit observed at different times. But each individual or unit who chooses to enrol in a program is (by definition) different from that who chooses not to enrol. These differences may invalidate causal comparison of outcomes by treatment status. Indeed, the fear in this econometrics literature is traditionally related to endogeneity, or self-selection, issues2. The simplest case for analysis is when assignment to treatment is randomized, and thus independent from the covariates as well as the potential outcomes. It is straightforward to obtain attractive estimators for the average effect of treatment in randomized experiments (e.g. the difference in means by treatment status). Although there have been some example 1Starting from the seventies, Rubin (1974, 1977, 1978) proposed to interpret the causal effect as comparison of so-called potential outcomes, namely pairs of outcomes define for the same unit given different levels of exposure to the treatment. This represent the dominant approach to the analysis of causal relationship in observational studies known with the label of Rubin Causal Model. 2Many of the initial theoretical studies focused on the use of traditional methods for dealing with endogeneity, such as fixed effect methods from panel data analyses and instrumental variables methods. Subsequently, the econometrics literatures has developed new approaches, requiring fewer functional form and homogeneity assumptions (Imbens and Wooldridge, 2008). 3/ 12 of experimental evaluations, they remain relatively rare in economics. More common is the case where economists analyse data from observational studies. Observational data generally create challenges in estimating causal effects referred to unconfoundedness, exogeneity, conditional independence, or selection on observable characteristics3. Estimation and inference of causal effect under unconfoundedness assumption requires that conditional on observed covariates there are no unobserved factors that are associated both with the assignment and with the potential outcomes4. Without unconfoundedness assumption there is no general approach to estimating treatment effects and various methods have been proposed (for a review, see Imbens and Wooldridge 2008). Where additional data are present in the form of samples of treated and control units before and after the treatment comparisons can be made through a difference-in-difference approach. The simplest setting is one where outcomes are observed for units observed in one of two groups (i.e. treated and control) and in one of two time periods (i.e. pre-treatment and post-treatment). Only units in one of the two groups, in the second time period, are exposed to a treatment. There are no units exposed to the treatment in the first period, and units from control group are never observed to be exposed to the treatment. To estimate the causal effect, the average change over time in the outcomes of control group is subtracted from the change over time in the outcomes of treated group. This double differencing removes biases in second period comparisons between the treatment and control group, that could be the result from permanent differences between those groups, as well as biases from comparisons over time in the treatment group, that could be the result of time trends unrelated to the treatment. Where the assignment of treatment is a deterministic function of covariates, comparisons can be made exploring continuity of average outcomes as a function of covariates. This setting, known as the regression discontinuity design, has a long tradition in statistics though only recently it has attracted much attention in the economics literature5. The basic idea is that assignment to the treatment is determined, either completely or partly, by the value of a predictor (i.e. an individual’s observable characteristic) being on either side of a common threshold. This generates a discontinuity in the conditional probability of receiving the treatment as a function of this particular predictor. Any other characteristic, between elected and unelected individual, is assumed to be smooth. As a result, any discontinuity of the conditional distribution of the outcome, as a function of this covariate at the threshold, is interpreted as evidence of a causal effect of the treatment6. 3For a review on this literature, see Imbens and Wooldridge (2008). 4Unconfoundedness implies that we have a sufficiently rich set of predictors for the treatment indicator, such that adjusting for differences in these covariates leads to valid estimates of causal effect. 5For recent review in the economics literature, see Van der Klaauw (2008), Imbens andWooldridge (2008) and Lee and Lemieux (2010). 6It may be useful to distinguish between two general setting, the sharp and the fuzzy regression discontinuity design. In the sharp regression discontinuity design, the assignment to treatment is a deterministic function of one of the observable covariates. In the fuzzy regression discontinuity design the probability of receiving the treatment 4/ 12 This thesis presents three essays of policy evaluation using the above quasi-experimental approaches. The research covers two different type of policies. On the one hand, we assess the effects on crime induced by a marijuana decriminalization policy exploiting the reforms still ongoing in the United States, on the other hand, we evaluate the impacts of the labour market reforms on labour market outcomes by using the recent changes in Italy occurred after the law 92/2012 (the so-called Fornero reform) like identification tool. Depending on the specific subject, the analysis is carried out from a specific empirical point of view. The first essay sheds light on the relationship between Medical Marijuana Laws and crimes in United States using counties level data. The set of judicial rules on the therapeutic consumption, production and distribution of cannabis at State level—started since 1996 in the United States—is known as Medical Marijuana Law (MML). It recognises the medical value of marijuana and provides a legal defence for patients who used and possessed marijuana under recommendation of a physician. The purpose of policy was the pain reduction for which the States allow doctors to prescribe marijuana as a pain killer also for general complaints related to pain, such as migraines, back pain and other pathologies. But, since the list of illness is quite broad, de facto, MML allows wide possibility for recreational use of marijuana masked like therapeutic consumptions (Chu, 2012). Hence, the assessment of policy on crime seems suitable. The research closely examining the importance of policy dimensions and the timing of the core elements of MMLs. In the U.S. States there have been three main actions that have involved the cannabis use for medical purpose: the mere decriminalization of marijuana, the permission of home cultivation for patients and caregivers, the licence for selling marijuana in authorized dispensaries. We interpret dimensions as design choices of policy maker on legal marijuana market by distinguishing between demand side approach, aimed to merely decriminalize cannabis, and supply side approach, directed to provide legal sources of supply for marijuana. This permits to explain the possible transmission channel trough which Medical Marijuana State Laws can affect crime. We test three possible links between drugs liberalization reforms and crime (i.e. pharmacological, economic, and systemic channels) finding evidence for only one of them (i.e. systemic channel). The analysis uses the Uniform Crime Reporting Program Data (UCR, 2013) which reports the number of arrests by type of offence from 1994 to 2014 at the U.S. county level. Since we have data of treated and control counties before and after the implementation of MML, we employ difference-in-difference approach by considering several types of crime such as violent and property crimes, and also felonies for narcotic possession (i.e. cocaine, heroine need not change from zero to one at the threshold. The design only requires a sufficiently large discontinuity in the probability of assignment to the treatment at the threshold. 5/ 12 etc.). We exploit the assessment of Medical Marijuana Law to highlight an important question in program evaluation concerning the heterogeneity of treatment effect. Even if the average treatment effect is zero, it may be important to establish whether a targeted implementation of intervention or different levels of treatment across the population could affect average outcome. We find that a simple dichotomous indicator of Medical Marijuana Law (i.e. the average treatment effect on all the U.S. States that passed the policy) may mask crucial dynamics underlying the relationship between policy and crime. Assuming a homogeneous impact of policy on crime, regardless the action implemented, the dichotomous indicator of MML captures only the net effect of the regulatory tools put in place by the legislator. On the contrary, the policy decomposition in key dimensions allows to discover different results which suggests a heterogeneous effects on crime according to the specific regulatory actions put in place by the legislator. In detail, for burglaries, larcenies, and cocaine drug possession, the mere application of demand side approach increases the crime in counties that passed the policy compared to counties without MML. While, the joint application of demand and supply approach— which establish legal sources for supply marijuana — may be able to realize a crowding-out effect on these offences. The findings support the idea that the licit competition on the marijuana market, triggered by the policy, could push out the illegal trade decreasing the crime. Finally, we find a net reduction in murders and a net increase in synthetic drug possession for the U.S. counties subject to the Medical Marijuana Law relatively to counties never passed the policy. The second and the third essays assess the impact of law 92/2012, implemented in Italy in 2012 (the so-called Fornero reform), on different labour market outcomes. The law 92/2012 introduced numerous changes regarding employment relationships amending past discipline. First. It substantially changed the discipline concerning the dismissals in firms above 15 employees. The reform established that in case of unfair dismissal, the dismissed worker has no longer the right to be reinstated as in the pre-reform period and receives a monetary compensation that ranges between 12 and 24 months pay. Thus the reform significantly reduces the firing cost borne by large firms. Second. Starting from January 2013, the Fornero reform also changed the discipline on apprenticeships concerning to the minimum duration of contract (no less than six months), the maximum number of apprentices that an employer can hire per each skilled worker (passed from 1:1 to 3:2), and the minimum number of apprentices that an employer must stabilize into permanent contracts for hiring a new apprentice (at least the 30% of apprentices hired in the last 12 months). Third. The Fornero reform implemented a new incentive program in favour of employers that recruit (on fixed-term or open-ended contracts) or stabilize into permanent agreements a worker aged 50 or more years. 6/ 12 The second essay (carried out with Giovanni Pica) estimates the effect of employment protection legislation on the flow of monthly hirings on open ended contracts using the aforesaid labour market reform passed in Italy in 2012. Much empirical research has focused on the effects of dismissal costs on labour market outcomes. The evidence suggests that EPL decreases employment inflows and outflows with little effect on employment and unemployment stocks. The reason is that firing costs act, in expected discounted value, as hiring cost reducing the willingness of the firms to both fire and hire workers (Bentolila and Bertola, 1990; Blanchard and Portugal, 2001). The most recent studies identify the causal impact of employment protection on labour market outcome exploiting within-county variation in EPL either across firms (e.g. of different size) or workers (e.g. of different age and/or tenure). The essay presented is in the line with within-county approach which allows to better control for time-varying unobserved characteristics that may affect labour market outcomes (act as confounding factors) compared to cross-country analyses. The presence of both treated and control firms observed before and after the policy — where the assignment of treatment depends in deterministic way from the number of workers employed — allows to implement a difference-in-difference approach jointly to a regression discontinuity design. We thus exploit the differential law change between firms with more and less than 15 workers comparing hirings in firms just above and below the 15 employee threshold before and after the reform (July 2012). The analysis is based on monthly data drawn from Italian Social Security (INPS) record for the period 2012 and 2014. The data provide information on the number of newly hired workers by firms size, province, sector, contract type, age and gender at a monthly frequency. The findings suggest that the reform raises monthly hirings on open-ended contracts by about 5.1 percentage points. The quantification of results reveals that the reduction of dismissal costs after the reform have induced about 4000 hirings per month in firms with more than 15 workers relative to firms with less than 15 workers. The effect of the reduction in EPL is not homogeneous across workers’ types. The increase seems to be more pronounced for full-time, young, and blue-collar workers. Conversely, we find no significant effect on the number of conversions of temporary contracts into permanent ones. The third essay evaluates the impact of labour policies aimed to improve the job possibilities for workers categorized as vulnerable (particularly in labour markets with stringent employment protection)7. Given the increasingly complicated transition from school to works, the youth appear a group more vulnerable compared to the past. Here the apprenticeship contract performs a crucial role by improving the job possibility and the stability of young workers (Berton et al., 2007; Casale et al., 2014). At the same time, the low employment rates for older workers pushed most OECD countries 7Evidences suggest that labour market prospects for youth and other marginal groups seem to worsen as a consequence of stringent EPL (Allard and Lindert, 2007; Bertola et al., 2007; Skedinger, 2010). 7/ 12 to experiment specific employment protections with the purpose to protect them from unemployment or/and to improve their job finding rates (Chéron et al., 2011). The Fornero reform intervenes by changing the discipline of apprenticeship in Italy and implementing a new incentive program for workers aged 50 or more years. The reform asymmetrically acted on the apprenticeships by changing the discipline in firms with more than 10 employees leaving the rules for firms below 10 unchanged. Likewise, the new incentive program for workers aged 50 or more years, passed with the Fornero reform, cut the hiring costs in firms that recruit workers over-50, leaving unaffected the costs for hiring workers under-50. These discontinuities in the regulation as well as the simultaneous presence of treated and control groups observed before and after the policy allow to implement a difference-in-deference method jointly to a regression discontinuity design. This quasi-experimental method permits to evaluate the causal effect of reform on the monthly hirings of apprentices and workers over-50. We thus exploit the differential law change in apprenticeships between firms with more and less than 10 employees, comparing the hirings and the conversions into open-ended contracts of apprentices in firms just below and above the 10 employees threshold before and after the reform (January 2013). Similarly, we compare the recruitments and the conversions into permanent contracts of workers with more and less 50 years before and after the reform. Also this analysis uses monthly data draw from Italian Social Security (INPS) record for the period 2012 and 2014. The findings suggest that the change in apprenticeships increase the stabilization of apprentices into open-ended contracts by about 3.9 percentage points in firms with more than 10 employees relative to firms with less than 10. We also find a positive association between law 92/2012 and the new recruitments of apprentices by about 7.1 percentage points in firms with more than 10 employees relative to firms with less than 10 employees. The employer incentives for hiring and stabilizing the workers aged 50 or more years positively affect the recruitments into open-ended contracts of workers over-50 relative to workers under-50 by about 1.6 percentage point. We also find a positive association between the incentive program and the hirings into fixed-term contracts of workers over-50 relative to workers under-50. Conversely, we don’t find effects for the conversions into open-ended contracts of workers aged 50 or more years. [edited by author]

Proposito della tesi di dottorato, dal titolo “Three Essays on Policy Evaluation”, è quello di sottolineare come l’impatto di una politica (economica e non) possa essere valutato secondo un approccio rigoroso, quasi-sperimentale, quando architettata adeguatamente a tale scopo. A tal proposito, sono mostrati tre esempi di valutazione delle politiche, nel corso dei quali si espongono ed affrontano le principali problematiche legate a questo tipo di esercizio.

L'interesse dell'approccio adoperato nella presente tesi è dato dall'ampia utilizzazione del metodo difference-in-difference che consente di stimare, in vari contesti, gli impatti che politiche di differente natura possono avere sulle variabili socio-economiche.

Ciascuno degli esercizi di valutazione costituisce un capitolo della tesi. Ogni uno di essi ha comportato una rassegna della letteratura in materia (allo scopo di inquadrare il tema trattato), la ricerca dei dati e l’elaborazione di uno specifico modello econometrico finalizzato all’identificazione del nesso causale.

Il primo capitolo valuta l’impatto che le politiche di decriminalizzazione della cannabis per scopi terapeutici (Medical Marijuana Laws), adottate nei singoli Stati degli Stati Uniti, hanno avuto sulla criminalità a livello di contea. L’impatto della riforma è valutato rispetto a differenti tipologie di crimini, tratti dal UCR del FBI, classificabili in reati legati alla persona (i.e. omicidi), alla proprietà (i.e. furti) e all’uso di altre sostanza stupefacenti (i.e. cocaina). Dal lavoro emerge come ciascuno Stato americano si differenzi dagli altri sia per le tempistiche che per le modalità attuative della decriminalizzazione. L’originalità del lavoro sta nello sfruttare questa eterogeneità. Esso, infatti, propone una scomposizione della politica di decriminalizzazione della cannabis in interventi chiave, legati alle specifiche azioni adottate in tema di approvvigionamento e di distribuzione della marijuana. Tale scomposizione permette di identificare come l’impatto sui crimini commessi vari a seconda del timing e delle modalità di approvazione della decriminalizzazione. La scomposizione consente, inoltre, di ipotizzare un possibile canale di trasmissione attraverso il quale la Medical Marijuana Law impatterebbe sui crimini. I risultati empirici suggeriscono che la semplice decriminalizzazione della marijuana avrebbe un impatto positivo sulla criminalità se non accompagnata da (contestuali) interventi finalizzati ad istituire fonti legali di approvvigionamento della sostanza.

Il secondo capitolo studia il ruolo che i meccanismi di protezione dell’occupazione (employment protection legislation) esercitano sui flussi in entrata del mercato del lavoro. A tale scopo, il lavoro stima l’impatto sulle assunzioni (e conversioni) in contratti a tempo indeterminato indotto dalla riforma del mercato del lavoro in Italia – legge n. 92 del 2012 (cd. riforma Fornero) – che ha ridotto i costi di licenziamento in carico alle imprese con più di 15 dipendenti, lasciando inalterata la situazione per le imprese con meno di 15 dipendenti. Il lavoro, oltre ad inquadrare la tematica con una rassegna della letteratura, sfrutta l’asimmetrico impatto della riforma (per le imprese appena sopra e appena sotto i 15 dipendenti) per stimare un modello difference-in-difference in un contesto di regression discontinuity design, utilizzando dati mensili INPS. I risultati empirici suggeriscono che la riduzione delle protezioni del lavoro incrementi le assunzioni in contratti a tempo indeterminato. Tale effetto risulta non omogeneo tra i diversi gruppi di lavoratori, mostrandosi più pronunciato per gli assunti full-time, più giovani ed operai. Al contrario, non emerge un chiaro effetto rispetto alle conversioni in contratti a tempo indeterminato.

Infine, il terzo capitolo esamina le politiche del lavoro aventi l’obiettivo di aumentare le possibilità occupazionali di categorie ritenute più vulnerabili, quali i giovani lavoratori e gli over-50. A questo scopo, si utilizza la riforma dell’apprendistato e l’introduzione di un nuovo schema di incentivi per i lavoratori over-50, in forza in Italia a partire da gennaio 2013 (legge 92/2012), per stimare l’impatto che tali tipologia di interventi hanno sulle nuove assunzioni (sia a tempo determinato che indeterminato) e sulle conversioni in contratti a tempo indeterminato. Nello specifico, la riforma dell’apprendistato ha introdotto, per le imprese con più di 9 dipendenti, nuovi obblighi di stabilizzazione degli apprendisti assunti ed ha innalzato il rapporto tra gli apprendisti e i lavoratori qualificati presenti in azienda. Al contrario, la riforma lascia inalterata la situazione per le imprese sotto i 10 dipendenti. Lo schema di incentivi per l’assunzione e la stabilizzazione dei lavoratori over-50 consta, invece, in un significativo abbattimento (50 per cento) dei contributi a carico impresa per i lavoratori con più di 50 anni, lasciando inalterata la situazione per i soggetti più giovani. Le suddette circostanze hanno reso possibile l’implementazione di un modello difference-in-difference in un contesto di regression discontinuity design, utilizzando dati mensili INPS. I risultati empirici dimostrano che la riforma dell’apprendistato ha effettivamente favorito la stabilizzazione degli apprendisti in contratti a tempo indeterminato. Lo schema di incentivi per gli over-50 sembrerebbe indurre nuove assunzioni tanto in contratti a tempo indeterminato quanto a tempo determinato. L’effetto sulle conversioni, invece, sembrerebbe trascurabile. [a cura dell'autore]