Community Features and School Achievement GapsAnd some other stuffDaniel AndersonJune 22, 20181 / 22

IntroductionMuch recent focus on open data in research generally
Open data tend to be rare in educational researchPrivacy concerns

2 / 22

IntroductionMuch recent focus on open data in research generally
Open data tend to be rare in educational researchPrivacy concerns
NCLB Required Publicly Available Data

2 / 22

IntroductionMuch recent focus on open data in research generally
Open data tend to be rare in educational researchPrivacy concerns
NCLB Required Publicly Available Data

School-level data 
Percent proficent in each of at least four proficiency categories
Disaggregated by student subgroups
2 / 22

Illustrating one approach

Simulate some data from two distributions
- Group 1: $n = 200$ , $μ = 200$ , $σ = 10$
- Group 2: $n = 500$ , $μ = 210$ , $σ = 8$
Cut Scores: 190, 205, 215 (totally made up)

3 / 22

Reardon & Ho Method⁽²⁾

Calculate the empirical CDF of each distribution
Pair the ECDFs
Calculate the area under the paired curve
Transform it to an effect-size measure (standard deviation units)

(2) Reardon, S. F., & Ho, A. D. (2015). Practical issues in estimating achievement gaps from coarsened data. Journal of Educational and Behavioral Statistics, 40, 158–189.

4 / 22

Reardon & Ho Method⁽²⁾

Calculate the empirical CDF of each distribution
Pair the ECDFs
Calculate the area under the paired curve
Transform it to an effect-size measure (standard deviation units)

(2) Reardon, S. F., & Ho, A. D. (2015). Practical issues in estimating achievement gaps from coarsened data. Journal of Educational and Behavioral Statistics, 40, 158–189.

4 / 22

Reardon & Ho Method⁽²⁾

Calculate the empirical CDF of each distribution
Pair the ECDFs
Calculate the area under the paired curve
Transform it to an effect-size measure (standard deviation units)

(2) Reardon, S. F., & Ho, A. D. (2015). Practical issues in estimating achievement gaps from coarsened data. Journal of Educational and Behavioral Statistics, 40, 158–189.

4 / 22

Transformation to effect size

$V = \sqrt{2} Φ^{- 1} (A U C)$

$V$ = Cohen's $d$ under the assumption of respective normality
In this case, Cohen's $d$ = -1.07 and $V$ = -1.06.

5 / 22

Why does this all matter?

6 / 22

Purpose7 / 22

PurposeEvaluate VV with empirical data
7 / 22

Purpose

Evaluate $V$ with empirical data
- Compare estimates from full data and manually coarsened data. Compare results to Cohen'd $d$ estimated with full data.

7 / 22

Purpose

Evaluate $V$ with empirical data
- Compare estimates from full data and manually coarsened data. Compare results to Cohen'd $d$ estimated with full data.
Apply these methods to publicly available data to investigate between-school differences in achievement gaps

7 / 22

Purpose

Evaluate $V$ with empirical data
- Compare estimates from full data and manually coarsened data. Compare results to Cohen'd $d$ estimated with full data.
Apply these methods to publicly available data to investigate between-school differences in achievement gaps
- Specifically using geo-spatial mapping, overlaying data about the surrounding area

7 / 22

Results8 / 22

Comparing $V_{c}$ and $d$

Both continuous: $r = 0.87 / 0.86$ .

9 / 22

Comparing $V_{d}$ and $d$

$r = 0.73 / 0.72$

Comparing $V_{c}$ and $V_{d}$

$r = 0.83 / 0.89$

10 / 22

Distribution of differences: $V_{c}$ and $d$

$μ = - 0.12, σ = 0.15$

$μ = - 0.16, σ = 0.16$

11 / 22

Comparing $V_{d}$ and $d$

$μ = - 0.10 / - 0.13, σ = 0.21 / 0.23$

Comparing $V_{d}$ and $V_{c}$

$μ = 0.02 / 0.03, σ = 0.17 / 0.15$

12 / 22

Substantive Investigations13 / 22

Achievement Gap Distributions

Reminder: School-level Distributions

14 / 22

Alameda CountyMedian Housing Cost15 / 22

Alameda Countynn Identifying as Black16 / 22

Alameda Countynn Income/Poverty Ratio > 2.017 / 22

ConclusionsEffect size appeared well estimated from coarsened dataVV was similar to Cohen's dd with these empirical data

The vast majority, but not all, schools had sizeable estimated achievement gaps
Clear geographic clustering of achievement gaps was evidentDid Not appear to depend on the Census data investigated here

18 / 22

Really quick: These slides!19 / 22

Reproducibility & Transparency

I care a lot about it
Slides are pretty (I think)
Super customizable (css)
Open source, transparent
slidex helps you transition from Microsoft Powerpoint!

20 / 22

21 / 22

Thanks so much!

Slides available at datalorax.com/talks/sicss-seattle

22 / 22

Help

Keyboard shortcuts

↑, ←, Pg Up, k

Go to previous slide

↓, →, Pg Dn, Space, j

Go to next slide

Home

Go to first slide

End

Go to last slide

Number + Return

Go to specific slide

b / m / f

Toggle blackout / mirrored / fullscreen mode

Clone slideshow

Toggle presenter mode

Restart the presentation timer

?, h

Toggle this help

Community Features and School Achievement Gaps

And some other stuff

Daniel Anderson

June 22, 2018

Introduction

Introduction

Introduction

Illustrating one approach

Reardon & Ho Method⁽²⁾

Reardon & Ho Method⁽²⁾

Reardon & Ho Method⁽²⁾

Transformation to effect size

Why does this all matter?

Purpose

Purpose

Purpose

Purpose

Purpose

Results

Comparing $V_{c}$ and $d$

Comparing $V_{d}$ and $d$

Comparing $V_{c}$ and $V_{d}$

Distribution of differences: $V_{c}$ and $d$

Comparing $V_{d}$ and $d$

Comparing $V_{d}$ and $V_{c}$

Substantive Investigations

Achievement Gap Distributions

Alameda County

Median Housing Cost

Alameda County

$n$ Identifying as Black

Alameda County

$n$ Income/Poverty Ratio > 2.0

Conclusions

Really quick: These slides!

Reproducibility & Transparency

Thanks so much!

Introduction

Help

Community Features and School Achievement Gaps

And some other stuff

Daniel Anderson

June 22, 2018

Introduction

Introduction

Introduction

Illustrating one approach

Reardon & Ho Method(2)

Reardon & Ho Method(2)

Reardon & Ho Method(2)

Transformation to effect size

Why does this all matter?

Purpose

Purpose

Purpose

Purpose

Purpose

Results

Comparing VcVc and dd

Comparing VdVd and dd

Comparing VcVc and VdVd

Distribution of differences: VcVc and dd

Comparing VdVd and dd

Comparing VdVd and VcVc

Substantive Investigations

Achievement Gap Distributions

Alameda County

Median Housing Cost

Alameda County

nn Identifying as Black

Alameda County

nn Income/Poverty Ratio > 2.0

Conclusions

Really quick: These slides!

Reproducibility & Transparency

Thanks so much!

Introduction

Help

Reardon & Ho Method⁽²⁾

Reardon & Ho Method⁽²⁾

Reardon & Ho Method⁽²⁾

Comparing $V_{c}$ and $d$

Comparing $V_{d}$ and $d$

Comparing $V_{c}$ and $V_{d}$

Distribution of differences: $V_{c}$ and $d$

Comparing $V_{d}$ and $d$

Comparing $V_{d}$ and $V_{c}$

$n$ Identifying as Black

$n$ Income/Poverty Ratio > 2.0