College Scorecard ROI methodology
This report explains how EDsmart Data uses the U.S. Department of Education College Scorecard together with IPEDS labels to produce ROI-style rankings and tables. It is the reference we point to when a chart or ranking needs a single, explicit definition of “return,” “cost,” and “who is in the denominator.”
TL;DR
We never treat Scorecard as a complete accounting of every dollar a student pays or earns. We combine published fields (price, debt, completion, earnings by fixed windows) under rules we name on each page, flag suppression and small programs, and separate institution-level medians from any individual outcome claim.
1. What “ROI-style” means on EDsmart Data
“ROI-style” is shorthand for comparisons that relate observed costs and financing to observed earnings (or, on some pages, to completion or debt service capacity) using federal releases. It is not a personal financial plan, a tax forecast, or a guarantee of future wages.
Where we publish a single index (for example a value score on a state ranking), we document weights, direction (higher-is-better vs lower-is-better), and normalization (e.g., z-scores within a cohort, winsorization caps, or percentile ranks). If a page does not list weights, treat the visualization as exploratory rather than a certified composite.
2. Primary sources and roles
- College Scorecard — Outcomes (earnings after entry, completion, repayment), student-body descriptors, and institution-level price and debt summaries derived from federal collections. We cite the Scorecard data site and the refresh we used for that page’s vintage.
- IPEDS — Sector, level, control, completions, and finance tables when we need a denominator that Scorecard does not carry cleanly, or when we align charts to NCES reporting categories. We use IPEDS as the authority for “who counts as public four-year” when that rule set is applied.
- BLS OEWS (state extracts) — Only when a page explicitly ties program or regional wages to occupation; we name geography and OEWS vintage in the footnote.
3. Earnings windows and cohorts
Scorecard publishes earnings for FAFSA-receiving cohorts at fixed intervals after entry (commonly six and ten years, depending on field and suppression). We state the window (e.g., “six years after first entering”) on every chart that uses earnings.
We do not mix windows within one ranking column. If a table compares institutions, each column is one vintage and one definition. When the Department revises historical cohorts, we bump the page “Last updated” string and, where material, add a short “what changed” note.
4. Cost and debt fields
Published tuition and fees describe the sticker that appears in catalogs for a typical program mix; they are not net price after grants. Average net price fields summarize what recipients paid by income band where the Department publishes them; bands can be suppressed for small cells.
Median debt summarizes borrowing among recipients who entered repayment, not every enrollee. Debt-to-earnings style cuts use the same pairing rules documented on the relevant data page (for example debt-to-earnings by major).
5. Sector and geography filters
When a ranking says “public four-year in California,” we mean institutions that meet the stated IPEDS/Scorecard mapping for that cohort (control = public, predominant award = bachelor’s or mixed undergraduate, main campus in CA unless otherwise noted). Multi-campus systems may appear as separate Scorecard units; we do not merge campuses unless the page says we did.
6. Suppression, small N, and outliers
Scorecard suppresses small cells to protect privacy. Missing cells are not zeros. We avoid ranking institutions with missing fields in the specific column being sorted, or we move them to an “insufficient data” bucket and say so.
Extreme values (very small programs with volatile earnings medians) may be winsorized or footnoted on interactive pages; static exports describe the rule in the table caption.
7. What we do not claim
- We do not attribute causality (“this college caused higher earnings”) from observational data alone.
- We do not substitute projected lifetime income when a page is built only on published fixed windows.
- We do not merge Pell-only and all-student earnings series in one column without relabeling.
8. QA and reproducibility
Each published ranking or data page lists data vintage (Scorecard release, IPEDS survey year). Internal builds log the extract hash or file date; when we publish a downloadable appendix, the download link appears on the ranking page.
9. Related work on the site
Ranking colleges by ROI (interactive national table) · Top colleges for ROI · Site methodology · Editorial policy