Archive | News Analysis RSS feed for this section

Missing Institutional Heritage

Some think tanks style themselves “universities without students,” but most are what I call “mission-driven” institutions.  A mission-driven think tank produces (ideally rigorous) public policy research.  But it is also supposed to stand for something: a particular vision of a just society.  Last week’s high-speed train wreck at the Heritage Foundation should give pause to every supposedly mission-driven think tank that has a neglected or under-defined institutional mission.

My former Harvard classmate, Jason Richwine, resigned from Heritage last Friday amid public outcry over his 2009 doctoral thesis, “IQ and Immigration Policy.”  In it, he argues that Hispanic immigrants as a group have a lower average IQ than do U.S. citizens born in the U.S., and he recommends that IQ tests be administered to prospective immigrants as one criterion for admission to the country.

Some immigration reform supporters imbibed more than a little Schadenfreude as the news cycle imploded.  Not I.  I remember Jason as a generous and genial man who refused to accept compensation in return for patiently tutoring me for an analytical methods qualifying exam in 2007.  I find Jason’s research interests disturbing and his policy recommendations wrongheaded.  But he is not an unkind person, and I feel badly for him and for his family.

I have neither time nor inclination to digest a stack of research about what Richwine calls “the construct of IQ” in order to develop an informed opinion about the quality of his empirical work.  Reports are mixed.  Tufts Professor Daniel Drezner is unimpressed, while Harvard Professor Richard Zeckhauser, who served on Richwine’s dissertation committee, calls it “careful.” (Disclosure: Zeckhauser is a former professor of mine.)

But even if Richwine’s empirical claims were well supported, I would find his policy proposal—that aspiring immigrants to the U.S. should be admitted or denied based in part on their IQ scores—philosophically objectionable for reasons that could well indicate that Richwine is a bad ideological fit for Heritage.  Responding to Slate’s Dave Weigel, Zeckhauser usefully distinguishes the empirical from the normative:

“[M]y view is that none of his advisors would have accepted his thesis had he thought that his empirical work was tilted or in error. However, Richwine was too eager to extrapolate his empirical results to inferences for policy.”

Policy recommendations necessarily depend on a combination of empirical findings and ideological commitments about the nature of a just society.  From the perspective of justice, IQ importantly differs from acquired job skills.  Job skills reflect individuals’ choices to improve their life prospects.  It is certainly, sadly, true that not everyone has an opportunity to acquire such skills—a consideration in favor of welcoming at least some low-skilled immigrants.  Nonetheless, a policy of favoring prospective immigrants who have job skills is more intuitively defensible than favoring some over others based on one narrow measure of undeveloped talent.  Richwine’s proposal reasonably offends people who believe that our laws—even our immigration laws—should not deny important life opportunities to individuals based on prejudgments about their inborn capacities.

Heritage is a mission-driven think tank—it exists to promote a specific, conservative vision of a just society, and to develop policy proposals that are consistent with that vision.  Success in this mission depends critically on hiring scholars who share Heritage’s animating values.  If Heritage is institutionally committed to a conception of equal opportunity that precludes legal discrimination on the basis of inherent characteristics rather than choices and accomplishments, then Richwine is a poor ideological fit there.

But if this is the correct story, then Heritage should not have hired Richwine in the first place.  Opportunistic firing (or even “resignation acceptance”) practices don’t somehow excuse careless hiring practices.  Indeed, both are costly to wonks and to the institutions they inhabit in multiple ways.

Most obviously, a supposedly mission-driven think tank with no clear idea about who fits under its ideological tent doesn’t really have a mission.  I don’t mean to suggest that a think tank’s mission should be narrow, or that its scholars must agree on everything.  On the contrary, a think tank that demands lockstep agreement will never retain smart scholars.  But if a think tank has a handful of truly defining commitments, those commitments should be clearly articulated lest the institution become media-driven or money-driven in lieu of advancing its mission.

Moreover, mission-driven think tanks that lack clear standards of inclusion—a category that includes nearly all of them—can be fearful places to work.  One need not dabble in the dark arts to find oneself suddenly unemployed for straying too far from an ideological reservation with invisible fences. How far is too far?  Most of the time, wonks find out only when they are handed their walking papers, asked to abandon cherished projects, or pressured to recant unacceptable statements.

Fear of incompetency can promote research excellence.  Fear of being branded a heretic often precludes it.  Most policy wonks live with too little of the former kind of fear and far too much of the latter.  I believe that many mission-driven think tanks could achieve a better balance by reforming their hiring and firing practices simultaneously, bearing in mind a few general guidelines:

Clarity about shared ideological commitments – Think tank mission statements are usually lofty-sounding but vague.  If a think tank has a set of core ideological commitments about the nature of justice and the proper role of the state, these should be specifically articulated somewhere, so that both financial supporters and scholars have clear expectations when they decide whether to invest time or money in the institution.

No mandatory empirical beliefs or policy conclusions – Insofar as a think tank holds itself out to be engaged in public policy scholarship as opposed to mere issue advocacy, it cannot be committed to predetermined findings and conclusions.  These are the results of good public policy research, not the premises.  A wonk who fears learning the “wrong” thing will fail to learn anything at all about the world, and her think tank will fail to make a meaningful new contribution to the public policy debate in which she participates.  Unhealthy, research-spoiling fear squanders scholars’ talents and supporters’ money.

Greater emphasis on empirical skills – Good empirical skills are the best preservative of truth under pressure. Public policy scholars often add value by developing new public policy proposals, an essentially interdisciplinary task that requires empirical findings, ideological commitments, a savvy awareness of the location of the Overton window, and a working knowledge of any potential legal hurdles to implementation.  Policy wonks do not necessarily need to be able to produce peer-reviewed social science research, but I’m going to go out on a limb and maintain that they should be able to understand and evaluate it if they address empirical questions in the course of their work.

Term contracts for policy scholars – I understand why mission-driven think tanks don’t grant tenure.  Although adults’ ideological commitments are usually fairly stable, they can shift over time, sometimes dramatically.  But the hasty departures that accompany media firestorms ill serve both think tanks and scholars.  I wouldn’t blame Heritage at all for concluding that Richwine’s personal views are inconsistent with Heritage’s principled ideological commitments.  Indeed, they are inconsistent with mine.  But what we saw last week was a desperate damage control effort, not a sober reevaluation of Heritage’s staffing profile. Ubiquitous at-will employment arrangements in the think tank world diminish research quality and creative thinking by fueling the debilitating kind of fear.  They also encourage a long-term pattern of scandal mongering and sometimes-feigned outrage among partisans and in the press.  Replacing at-will agreements with standard two- or three-year employment contracts could paradoxically empower think tank executives to make thoughtful personnel decisions by tying their hands until the smoke clears.

Heritage was humiliated last week because its leaders failed to make personnel decisions thoughtfully in accordance with a clear institutional vision.  Let it serve as a cautionary tale for other mission-driven think tanks.  In a world of clear—not narrow!—standards, policy scholars could determine with more confidence where their own ideological commitments make them a good fit.  In a world without them, we can expect more embarrassing departures and, I fear, more subtly compromised research.

A Shilling on the Side

Buzzfeed’s Rosie Gray reports that former Bush speechwriter Joshua Trevino secretly organized a gaggle of professional writers to opine about Malaysia.  In return, the writers received payments from public relations firms representing the government of Malaysia.  Their columns—which praised the Malaysian government or criticized its political opponents without disclosing the financial relationship between author and subject—appeared in various media outlets, including National Review Online, the Huffington Post, Ricochet, the Washington Examiner, and the San Francisco Examiner.

Some of Trevino’s recruits were affiliated with think tanks at the time, including the Heartland Institute, the American Center for Democracy, and the Manhattan Institute, where I used to hang my hat.  However, none of these writers seem to have published their public relations contract work on think tank websites (with the possible exception of this early 2011 piece by Heartland’s Ben Domenech), nor do they appear to have used their think tank affiliations in their Malaysia bylines.

Which raises the question:  When is your night job a problem?

It’s not uncommon for a policy scholar to have a second source of income.  Young and not-so-famous think tankers are often paid barely enough to cover a modest rent and the clothes they wear for television interviews.  As a novice, I had several colleagues who held second jobs, usually tending bar.  One turned out to be a budding maestro of mixology, but others were not particularly good at their off-hours gigs.  (I personally would have been a painfully incompetent bartender.)  A think tank has no reason to look askance at its bad bartenders, because the two tasks are completely unrelated.  Bad bartending doesn’t reflect on the quality of anyone’s performance as a policy wonk.

Undisclosed public relations contract work, however, does reflect poorly on the primary professional efforts of think tank scholars.  Non-disclosure of a financial relationship with the subject of a published article violates the ethical standards of both journalists and public relations professionals.  It is therefore not surprising that National Review Online, the Washington Examiner, and the Huffington Post have disavowed the Malaysia-themed PR placements, or that the Wall Street Journal has administered a spanking to the writers involved.  There is a difference between off-duty incompetence and an off-duty ethical lapse.

Of greater concern, in my view, is the nature of the lapse.  A think tanker’s failure to disclose public relations contract work as such raises the possibility that he doesn’t see any difference between his wonkish activities—policy scholarship and opinion journalism—and paid client representation.

I used to be a litigation attorney.  I see nothing inherently ethically suspect about paid client representation.  It is just a different activity than either scholarship or journalism.  An advocate for a client takes the client’s best interests as a starting point, and then he makes the strongest possible case for them.  A client-advocate must not lie, but he is also not directly in the business of seeking the truth of a matter reasonably contested.  He is a cog in a larger epistemic process that we hope enables judges and readers to glean wisdom from the public conflict of competing interests.

This is why the mere act of hiring an outstanding lawyer doesn’t cause the public to believe that a criminal defendant is innocent.  On the contrary, if you hire a really fancy lawyer, the public tends to assume that you do, indeed, have a serious legal problem.  Similarly, if you hire a fancy public relations firm, people tend to think that you have probably done something embarrassing or unpopular.  Professional advocates can and do help their clients, but no one mistakes their public advocacy for an independent, expert appraisal.

By contrast, a think tank scholar’s opinion is thought to carry some authority, because he is not being paid to advance anyone’s interests in particular.  Wonks usually apply some combination of abstract ideological principles and well-honed analytical skills to a set of facts in an effort to illuminate the key features of a potentially confusing event.  I afford a reputable think tanker’s opinion some deference in the many policy areas in which I am not an expert, because I assume that it reflects the current sincere belief of a smart person who knows the subject matter much better than I do.

At best, public relations work is an imprudent second calling for a policy scholar, who thus debases the public currency of his expertise. Secret public relations contracts are ethically problematic, because they fool readers like me into affording the arguments of paid client-advocates the benefit of the doubt that we accord to those of independently compensated experts.   Think tanks would be wise to bar their scholars from undisclosed PR moonlighting, which diminishes institutional credibility by raising questions about whether what their scholars do on the job is really so different.  Reputationally, it seems to me that once you’re in for a shilling, you’re in for a pound.

Straw Poll Fallacy

Good think tanks do research, and they also do advocacy, but think tanks that fail to make any distinction between the two squander valuable reputational capital.

Last Friday, my former MI colleague, Josh Barro, scolded the Florida-based James Madison Institute for conducting a “push poll” about the state’s federally-subsidized Medicaid expansion plans.  “This isn’t a poll designed to figure out how Floridians feel about the Medicaid expansion,” Barro complained, “it’s one designed to get them to say they oppose it, so the organization commissioning the poll can say it’s unpopular.”

Cato Institute health policy guru Michael Cannon, also a former colleague of mine, had apparently reviewed the poll questions for the James Madison Institute before the poll hit the field.  Cannon fired back:

Medicaid expansion is not a benefits-only proposition. When a poll only asks voters about benefits, the results are meaningless. Yet to my knowledge, JMI’s poll is so far the only poll that has asked voters about both costs and benefits. All other polls—for example, the hospital-industry poll Barro cites—ask only about benefits, as if the costs don’t exist or shouldn’t influence voters’ evaluation of the expansion. Those polls are “push” polls, while JMI’s poll is the only honest poll in the field.

I consulted an experienced GOP-leaning political pollster in the Washington, DC area to get the skinny.  The pollster, responding on condition of anonymity, expressed “serious concerns about the poll.” To wit:

First, it’s not a true survey of registered voters, because they focus mostly on pulling from registration lists those who voted in at least two of the last four elections. You can’t say that’s representative of Florida registered voters, though you could say its representative of likely voters. That’s a distinction that should be made clear, as it will bake in a slight right-leaning skew compared to straight-up registered voters.

I stopped reading and started writing this email when I hit that first debt question. A good poll would have asked a more “clean read” without loading up a big message before the ask about how important the debt is. The interviewer says “well everyone else cares about the debt, so, how concerned are you?” Really not good. This is the kind of question you push further down in the questionnaire as a message test, not as a legitimate gauge of concern about debt.

Then I got to the question [posed as] “some say we need reform” vs. “some say we need to preserve a government program.” How [often do] Democrats actually say, “we must preserve a government program!” Never. They say, “we must preserve needed health services for our poorest citizens,” etc. A good poll puts our best message against their best message. Already, the poll is putting up a weak version of the opposition’s position.

The point thus goes to Barro, though I’m sympathetic to Cannon, who is not a pollster and was only asked to review these questions for the accuracy of their substantive claims about Medicaid.

Such bad methods reflect poorly on the James Madison Institute, which holds itself out to be a research and educational organization, complete with a Research Advisory Council primarily composed of university-based social scientists. Think tank research isn’t expected to be peer-reviewed academic journal fodder, but it usually aspires to inform the public policy debate by telling us something new about the world we live in.  Think tank findings are often presented in light of researchers’ prior ideological commitments, but they should not merely be talking points in support of predetermined conclusions.

Not surprisingly, a James Madison Institute press release reveals that a division of the Florida-based public relations* firm Cherry Communications conducted the Medicaid expansion poll under contract.  “[While] a polling firm’s first goal is to create situational awareness,” the DC-area pollster explained, “a PR firm’s first goal is to create good headlines. These are each valuable but are not the same thing.”  Nor are they necessarily mutually exclusive:

There are really two different ways to approach designing a poll. One is if you want an accurate read on public opinion to guide strategic decision making. The other is to “message test” and to figure out how best to move opinion and build a communications plan. You can do both in one survey as long the “clean read” part comes first.

The fundamental problem here is that this poll was conducted with public release in mind and to show right off the bat that conservative messages on the issue work. This is a PR firm’s goal clearly. There’s no time taken to get the clean read.

The James Madison Institute hasn’t yet responded to my request for comment, but it isn’t hard to surmise what happened here: the communications department probably commissioned a poll as a way to get airtime for the Institute’s message on Medicaid expansion.  But a poll isn’t just a message.  A poll is a social scientific method, which why a lousy poll from a think tank casts doubt on the quality of its other research.

Journalists and policymakers afford more weight to think tank research than they do to press releases from PR firms because think tanks aren’t supposed to just spin.  The James Madison Institute may have rationalized this survey as the digital equivalent of liquid courage for skittish pols, but it should worry instead about what techniques like these suggest about its institutional values.  Reputation matters, because media and government consumers often don’t have the time or expertise to independently assess the quality of every report.  I am less likely now than I would have been last week to take anything in the James Madison Institute’s new policy brief on Medicaid expansion at face value, because I have reason to question the organization’s commitment to good research methods.

UPDATE:  I have just been informed that the Cherry Communications website I linked above belongs to a different firm with the same name as the “Cherry Communications” referenced in the James Madison Institute press release, whose division, “Public Insight,” conducted the Medicaid expansion poll.  I have eliminated the incorrect link, and I apologize to both firms for the error.

The James Madison Institute has offered some comments regarding the poll, and Jim Cherry of the Cherry Communications whose division conducted the Medicaid expansion poll has offered to comment also, so stay tuned for a follow-up post.

*Cherry Communications is really better characterized as a Republican political consulting firm specializing in phone-based services such as voter identification calls, persuasion/advocacy calls, get-out-the-vote calls, surveys, and polls.  It’s website is here.