Abstract

Managers fear that the preferential hiring of military veterans leads to the selection of employees who score worse on merit criteria. Using all non-classified records in the U.S. federal government’s Central Personnel Data File from 1973-1997, we compare the educational backgrounds of employees who received veterans’ preference benefits with the educational backgrounds of employees who did not receive veterans’ preference.

• This is timely given what is happening among federal employees. Hiring preferences are under assault by innuendo.

What We’re Up To

We eliminate uncertainty due to sampling variation with the population of unclassified data. This allows us to focus on the uncertainty surrounding which employees to compare in our analysis.

• Replicate prior evidence and media accounts that preference beneficiaries possess weaker educational credentials than non-recipients.

• Make this relationship disappear—and, in some instances, invert—when we account for employees’ occupations and work circumstances, their jobs, using exact stratification.

• Takeaway: The assessment of perceptions of hiring quality requires careful attention to the appropriate basis for comparing employees. Ending veterans preference because of differences in career trajectories or educational attainment is empirically unjustifiable.

What is Veterans’ Preference?

• van Riper’s History of the United States Civil Service notes it is one of the oldest personnel policies in the federal government.

• Veterans’ preference predates the competitive civil service exam according to the Merit Systems Protection Board.

• Lewis (2012): Under the policy, eligible military veterans—as well as the spouses and mothers of veterans who were disabled or killed in combat—receive either added points on their federal service applications or hiring priority over candidates categorized in the same quality tier.

Prior Literature and a Changing Landscape

Quite frankly, I have both positive and normative motivations that I must lay bare. The topic is of both public and academic interest.

Federal Eye

Academic Research is More Uniform

In various forms for almost a century, Miller (1935), Ordway (1945), Lewis (2012) maintain that these procedures force managers to distribute jobs to preference recipients, even if other candidates appear to be more deserving of a position.

• Lewis (2012) analysed 1% samples of the U.S. Office of Personnel Management (OPM) Central Personnel Data File (CPDF) and found that military veterans enter federal service with less education than nonveterans. When considering new hires that perform ``white collar’’ jobs, Lewis found that veterans were less likely than nonveterans to have completed college or to have earned a graduate degree. Furthermore, veterans averaged one year less education than their nonveteran peers.

• As an aside, 1% samples are pretty dangerous for extreme imbalance.

Related Literature

• Lewis 2012: 247: ``Federal personnel data for the past decade show that veteran new hires are older and less educated than nonveteran new hires, and that they do not advance as far in the first 15 years of their careers as nonveterans hired into the same grades at the same time, suggesting that veterans’ preference may be lowering the performance of the federal civil service.’’

• Johnson and Walker (2018) use pay grade and pay increase data to show equivalent advancement tendencies for preference recipients and non-recipients when entry positions are rendered comparable.

The Data

Unlike the 1% samples, in the summer of 2009, the National Archives and Records Administration (NARA) released a copy of the U.S. Office of Personnel Management’s (OPM) Central Personnel Data File (CPDF) under a FOIA filed by Tim Johnson. This copy of the CPDF contained all non-sensitive employee records from 1973 to 1997. The data are a person-year panel with more than 3.8 million individual persons.

The Empirics of Veterans’ Preference and Education

Table

A Graphical View

Figure 1

Interpretation

• dark grey bar represents the proportion of the category shared by both recipients and non-recipients.
• black bars extending above the dark grey bars represent the proportion of non-recipients within the category in excess of the proportion of preference recipients.
• white bars represent the degree to which a higher proportion of preference recipients reside within the category.
• Education is increasing, left to right, e.g. more black bars on the right-hand side of the diagram means a higher percent of non-recipients possess greater levels of education.

Statistical tests

• Means: Common but not technically correct: two-sample t-tests [Welch]
• Medians: Wilcoxon rank-sum/Mann-Whitney U [coin::wilcox_test]

The data are ordered; metric statistics are less than ideal. The Wilcoxon rank-sum simply ranks the data and assesses the sums of the ranks for preference recipients and non-recipients. There are three ways to obtain a probability measure for the null hypothesis of equality of medians: - exact permutations, - simulation, and - a normal approximation.

The one painful compromise

• The exact distribution is preferred.
  • Can’t compute it for the huge stratum
• So we simulate with 1 million permutations for each stratum.
  • Then we invoke a root-n argument to approximate p-values to within 0.001 with which to form distributions over strata relying on asymptotic normality.
  • It’s really important to emphasise that those p-values are an empirical probability measure because we have a population and some foundational inferential statistical issues are messy in this very unique example.

The Conventional Wisdom

Table 2

• Negative signs -> preference recipients are lower
• Positive signs -> preference recipients are higher.
• Enormous negative t-statistics/Wilcoxon z-statistics.
• Preference recipients have lower levels of education.

The Critical Issue: Whom to Compare?

• Preference recipients are not altogether common.
• It is quite likely that they are not randomly distributed through jobs in the federal service.

Table 1 Excerpt

The Counterfactual of Interest

• Key argument: the job is a hidden confounder.
• We are fortunate to have measures of five key characteristics of jobs.
  • Year of entry: the year the employee entered federal service
  • Duty station: where they are employed
  • Agency: the agency that employs them
  • Occupation: 1519 categories of work
  • Grade: GS grades are most common but there are others.

The Precise Question

Our counterfactual of interest is defined by the job as opposed to the person.

• Had veterans’ preference not played a role in installing the job’s incumbent, how would educational attainment differ for the holders of that job?

Jobs as Years of Entry

Jobs as Occupations

Wilcoxon statistics for Occupations

Jobs as Occupations: Clerk/Assistant [303]

The top two results have switched signs: 5-point and 10-point Veterans’ Preference recipients have higher levels of education; the evidence for spouse/mother is a bit mixed.

Education for Clerk/Assistant [303]

Education by Preference Status for Clerk

Note the abundance of white on the right side of the figures. Higher education levels for preference recipients.

A Job as Year of Entry, Grade, Occupation, Duty Station, Agency

Most stringent job and education

We still retain lots of comparable data

Notes: The y-axis is censored at 200. The result from this is the top row of the next table.

All Definitions of Jobs

• Grey and black cells contain greater than 20% with p-values less than 0.05.
• Italics and bold represent p<0.05 in the omnibus column.

Discussion

• Prior evidence has led to what Pearl (2012, 176) calls a distorted causal interpretation. The broad universal association is quite different than the conditional associations when the job becomes the counterfactual. Though it is claimed that any deviations from merit screening permit less-qualified individuals to enter the federal service, merit screening is only one of many filters.

• On balance, we find very little evidence that veterans’ preference worsens the educational attainment of job incumbents.

• How could we account for the finding that veterans receiving preference appear to possess equal or greater education in many cases? The GI Bill comes to mind.

Big Picture

Ala Pearl’s discussion of Simpson’s Paradox, given that veterans’ preference accrues to veterans that have unique access to educational benefits, it is quite easy to see how a relationship in the aggregate showing lower levels of education among preference recipients should fail to replicate for at least some jobs if veterans distribute themselves across occupations differently than nonveterans and veteran status correlates with potential educational benefits.

A Postscript

• Joint work with Justus Eaglesmith [NARA] and Tim Johnson.
• Can LLM’s identify statistical distributions?
• No.

What distributions?

Distributions

The Prompt

Prompt

The Results

Results