What is “Pay for Play?”

No think tank wants to be known as a “pay for play” organization, but the concept is rather like “pornography.”  People think they know “pay for play” when they see it, but its necessary and sufficient conditions remain obscure.  That’s too bad, because think tanks have a lively interest in determining exactly what “pay for play” ought to mean.  You can’t make policies about something unless you can define it.

Perhaps it is easier to say what “pay for play” is not.  The aspersion can’t simply refer to an earmarked donation from a benefactor with a passionate ideological conviction.  Think tanks like the Heritage Foundation, the Center for American Progress, and the Cato Institute attract support from individuals who want to advance a particular conception of a just society, and such donors often have a favorite issue.  I used to work on education policy at Cato.  I never fundraised, but I sensed that my work was easy to support because libertarian philanthropists tend to care deeply about school reform.  Research support is not ethically suspect just because a supporter, unsurprisingly, considers the research important.

Nor is a contract between a think tank and a funder “pay for play” just because the think tank specifically agrees to answer that funder’s research question.  Such deals are usually called “contract research.”  The Urban Institute and the Rand Corporation do a lot of contract research for both public and private clients, and they receive nary a sideways glance for it.

Lack of timely disclosure is a common element of financial relationships that give rise to the occasional think tank scandal.  But all by itself, lack of disclosure seems insufficient to make a funding relationship an example of “pay for play.”  Most obviously, very small donations don’t seem to require disclosure.  The Heritage Foundation reports that it has “hundreds of thousands” of individual members, making it “the most broadly supported think tank in America.”  Heritage’s 2011 annual report lists hundreds, but certainly not hundreds of thousands, of supporters.  It doesn’t seem like a problem that the think tank does not disclose the identity of every small contributor.

Whether think tanks ought to disclose very large contributions is a more debatable question.  But even very large contributions can’t be characterized as “pay for play” if they are truly unconditional.  A posthumous gift of $1 million, conveyed anonymously by an executor to a lucky think tank, would probably be as welcome as the discovery of a buried treasure underneath the auditorium.

The term “pay for play” suggests that some sort of quid pro quo is involved—that the supporter is getting something out of the relationship besides the warm feeling of a good deed done.  The question is: what exactly is that something else?

What characteristics must a funding relationship have in order to count as “pay for play” in the think tank world?  Make your suggestions in the comments.

Perspective: Thomas Carothers on Balancing Research with Engagement in Washington, DC

This post is a part of a recurring series titled, “Perspectives on Think Tank Ethics and Governance.”  Each Perspective post will feature highlights from a personal interview.  In this week’s post, Thomas Carothers, Vice President for Studies at the Carnegie Endowment for International Peace, comments on the Washington, DC think tank world, and on the challenge of striking an appropriate balance between research and media engagement.

Dr. Carothers, what are the best institutional practices and policies, in your view, for promoting scholarly integrity in a research environment that is oriented in part toward real changes in the world as opposed to simply academic exploration?

CAROTHERS:  Well, the think tank world, at least in Washington where I take part in it, is a complicated world. [There are] a lot of pressures to do different things, and there are issues about the goals of what it does and its methods. There is surprisingly little reflection on it, and certainly no oversight. It is a group of self-appointed actors who try to be useful, and try to do what they want to do, and not many people ever question what they do. So I’m glad that you are doing what you are doing, because I think it is good to reflect on it.

It is also constantly changing because of how political life in the United States is changing. The information revolution is changing what think tanks do as well. The first issue that you face at a think tank at a strategic level, and in terms of thinking about what you really want to do, is simply, what is your goal? Different think tanks have different approaches to that. Some have a fairly well-defined institutional political goal: “we have a set of political ideas that we would like to win out in political contests and American politics.” So, if Think Tank X has a particular conservative agenda, and says, “We think on these issues we would like to see the United States move this direction,” then that’s your goal. Other think tanks, on both the left and right in Washington, have a political conception of their role. We don’t, and neither do some other, I’d say some of the best think tanks in Washington.

We view our goal is somewhat broader than that. It is related to a set of principles which I think are not left/right, and not specific to Washington. They are principles about peaceful international cooperation and positive engagement of the United States with the world. I believe that the United States can be a partner in global affairs in trying to solve some of the biggest problems facing humanity. That doesn’t lead you to any particular political platform or particular position and political debates, because you have reference to a higher set of principles. It also means that your target is not only US policymakers, but policymakers and international institutions, other governments, citizens in different parts of the world. Where some think tanks think, “our goal is these particular congressional committees plus these executive branch people,” we have quite a wide intended audience. Most of the policy issues we work on, there is a large policy community with very different actors working on [them]. So, you choose your goal. Then, you decide, “How are we going to achieve it?”

Research, in the narrow conception of the term, is just one part of a larger, interconnected process in which you, ideally, are trying to provide some original insight. You hopefully have a kernel of insight on some particular issue, but that has to be part of a larger process of outreach or connection to the policy community. That’s hard. Policy processes are overwhelmed with people wanting to influence them. You’re one of many different actors trying to have an impact. One of the hardest things to know in a think tank is, in sort of simple terms, what percentage of your resources do you commit to the original research versus the overall process? Is research 10%? 90%? 50% of your overall resources? You can argue in different ways that you should invest very heavily in the process in order to make whatever research you do effective. Or, you can argue it the other way, that you really need to do some first rate research — original, innovative, useful – and that’s where you should put your core. That’s a continual tension.

How does Carnegie answer that question, in terms of percentage commitment of resources?

CAROTHERS:  Well, in the past, 10, 15, or 20 years ago, I would say that there was a higher percentage of our resources generally going to the research core. But as the environment became more competitive over the years, here in Washington, and actually in every policy capital, and as communications became more complex – it’s just more of a communicative world – we spend more and more on communications and processes. Now, if you have an overall budget for a project, 20 or 30% at least would be devoted to communications.  That’s continually growing, the communications side. Communications is easier and easier, in some ways.  You can reach more people. But it’s more and more expensive as you do that. Not just the hardware, all the computers and servers, but you need to have a social media coordinator, various web coordinators, a traditional media coordinator.  You need to have Congressional outreach. You need to have a lot of different people communicating what you do to different parts of the audience. So, we try hard to keep a balance between the two. We are firmly committed to continuing to do original work.

It’s tempting not to do that and just become a process organization, so one of the pressures in Washington is, “is our name in the paper all the time? Do we see our people on television?” It becomes very easy to mistake the means for the ends, and then the means become the ends. You will have a board meeting, and there is a sheet handed to the board that says we were cited in the New York Times X times in 2012. And the board members say, “that’s great. Next year, let’s do 2X,” rather than saying, “well that’s great, but are we being cited because we’re doing really important, original research, or are we just being cited a lot because we had a great media coordinator who is constantly calling reporters? How is it that you’re getting this access and why?” There is a transition to say, let’s have lots of web hits, lots of newspaper sites, lots of television appearances rather than asking the more complicated question of, “are we making a positive contribution?” Punditry, which is, in a simple sense, being an opinion maker and being out there, is a continual temptation for think tankers—to simply be pundits rather than what I would consider serious policy researchers.

Would it be fair to consider this a sort of a moral hazard for scholars? If they are professionally rewarded and recognized for being in the media a lot, is there a temptation to spend marginally less time on deep research, in order to spend marginally more time on this kind of engagement?

CAROTHERS:  I’d say it is. I guess it’s a moral hazard. On the other hand, none is not enough. In other words, if you just have a scholar who goes into his or her office and closes the door and comes out after five years with a long, complicated book that no one ever reads, it may be beautiful and deep, but it’s useless. It is a stone thrown into the ocean that just sinks to the bottom. So it isn’t as though thinking about and working on [media engagement] is bad. It’s simply a balance and being true to your mission, and not doing it for its own sake. That’s what makes it complicated, I think. There isn’t an ideal ratio or percentage.

There are some issues for which media work is quite valuable. If there is a really fierce public debate over whether we should intervene militarily in Syria or not, getting your people out to be in the key debates on the Sunday morning talk shows is probably useful in policy terms. This is an issue on which the court of public opinion is open. On other issues, being on television is just beating a dead horse, or not all that important. Television appearances in some cases can be, I think, more useful than others. It involves qualitative, case-by-case judgments about why we are putting time and energy into certain things. It’s true that there are moral hazards, but they’re not on a simple sliding scale, where we can just say, “Be careful, don’t do this. Instead do that.” That’s what makes it tricky.

Straw Poll Fallacy

Good think tanks do research, and they also do advocacy, but think tanks that fail to make any distinction between the two squander valuable reputational capital.

Last Friday, my former MI colleague, Josh Barro, scolded the Florida-based James Madison Institute for conducting a “push poll” about the state’s federally-subsidized Medicaid expansion plans.  “This isn’t a poll designed to figure out how Floridians feel about the Medicaid expansion,” Barro complained, “it’s one designed to get them to say they oppose it, so the organization commissioning the poll can say it’s unpopular.”

Cato Institute health policy guru Michael Cannon, also a former colleague of mine, had apparently reviewed the poll questions for the James Madison Institute before the poll hit the field.  Cannon fired back:

Medicaid expansion is not a benefits-only proposition. When a poll only asks voters about benefits, the results are meaningless. Yet to my knowledge, JMI’s poll is so far the only poll that has asked voters about both costs and benefits. All other polls—for example, the hospital-industry poll Barro cites—ask only about benefits, as if the costs don’t exist or shouldn’t influence voters’ evaluation of the expansion. Those polls are “push” polls, while JMI’s poll is the only honest poll in the field.

I consulted an experienced GOP-leaning political pollster in the Washington, DC area to get the skinny.  The pollster, responding on condition of anonymity, expressed “serious concerns about the poll.” To wit:

First, it’s not a true survey of registered voters, because they focus mostly on pulling from registration lists those who voted in at least two of the last four elections. You can’t say that’s representative of Florida registered voters, though you could say its representative of likely voters. That’s a distinction that should be made clear, as it will bake in a slight right-leaning skew compared to straight-up registered voters.

I stopped reading and started writing this email when I hit that first debt question. A good poll would have asked a more “clean read” without loading up a big message before the ask about how important the debt is. The interviewer says “well everyone else cares about the debt, so, how concerned are you?” Really not good. This is the kind of question you push further down in the questionnaire as a message test, not as a legitimate gauge of concern about debt.

Then I got to the question [posed as] “some say we need reform” vs. “some say we need to preserve a government program.” How [often do] Democrats actually say, “we must preserve a government program!” Never. They say, “we must preserve needed health services for our poorest citizens,” etc. A good poll puts our best message against their best message. Already, the poll is putting up a weak version of the opposition’s position.

The point thus goes to Barro, though I’m sympathetic to Cannon, who is not a pollster and was only asked to review these questions for the accuracy of their substantive claims about Medicaid.

Such bad methods reflect poorly on the James Madison Institute, which holds itself out to be a research and educational organization, complete with a Research Advisory Council primarily composed of university-based social scientists. Think tank research isn’t expected to be peer-reviewed academic journal fodder, but it usually aspires to inform the public policy debate by telling us something new about the world we live in.  Think tank findings are often presented in light of researchers’ prior ideological commitments, but they should not merely be talking points in support of predetermined conclusions.

Not surprisingly, a James Madison Institute press release reveals that a division of the Florida-based public relations* firm Cherry Communications conducted the Medicaid expansion poll under contract.  “[While] a polling firm’s first goal is to create situational awareness,” the DC-area pollster explained, “a PR firm’s first goal is to create good headlines. These are each valuable but are not the same thing.”  Nor are they necessarily mutually exclusive:

There are really two different ways to approach designing a poll. One is if you want an accurate read on public opinion to guide strategic decision making. The other is to “message test” and to figure out how best to move opinion and build a communications plan. You can do both in one survey as long the “clean read” part comes first.

The fundamental problem here is that this poll was conducted with public release in mind and to show right off the bat that conservative messages on the issue work. This is a PR firm’s goal clearly. There’s no time taken to get the clean read.

The James Madison Institute hasn’t yet responded to my request for comment, but it isn’t hard to surmise what happened here: the communications department probably commissioned a poll as a way to get airtime for the Institute’s message on Medicaid expansion.  But a poll isn’t just a message.  A poll is a social scientific method, which why a lousy poll from a think tank casts doubt on the quality of its other research.

Journalists and policymakers afford more weight to think tank research than they do to press releases from PR firms because think tanks aren’t supposed to just spin.  The James Madison Institute may have rationalized this survey as the digital equivalent of liquid courage for skittish pols, but it should worry instead about what techniques like these suggest about its institutional values.  Reputation matters, because media and government consumers often don’t have the time or expertise to independently assess the quality of every report.  I am less likely now than I would have been last week to take anything in the James Madison Institute’s new policy brief on Medicaid expansion at face value, because I have reason to question the organization’s commitment to good research methods.

UPDATE:  I have just been informed that the Cherry Communications website I linked above belongs to a different firm with the same name as the “Cherry Communications” referenced in the James Madison Institute press release, whose division, “Public Insight,” conducted the Medicaid expansion poll.  I have eliminated the incorrect link, and I apologize to both firms for the error.

The James Madison Institute has offered some comments regarding the poll, and Jim Cherry of the Cherry Communications whose division conducted the Medicaid expansion poll has offered to comment also, so stay tuned for a follow-up post.

*Cherry Communications is really better characterized as a Republican political consulting firm specializing in phone-based services such as voter identification calls, persuasion/advocacy calls, get-out-the-vote calls, surveys, and polls.  It’s website is here.

About This Blog

A recent leadership struggle at the Cato Institute and a changing-of-the-guard at the Heritage Foundation have sparked a lively public debate about what think tanks do, how they do it, and about whether or not it is possible to do what think tanks do in an ethical manner. This blog will explore that last question from an optimistic perspective:  that it is possible for think tanks to maintain high ethical standards, and that most think tank executives and scholars on all parts of the political spectrum strive conscientiously to do so.  I am less certain that those of us in the public policy world know as much as we would like to know about how to accomplish this goal.

Different think tanks have vastly different institutional missions, cultures, and funding arrangements, but all appear to share a central challenge: reconciling scholarly aspirations with the imperative of relevance.

On one hand, think tank research usually aspires to be scholarly in the sense that it makes a new contribution to the public policy debate.  Wonks need not use methods associated with traditional academic disciplines, and indeed academic methods are often unsuitable for answering broad, practical questions about what to do.   Nonetheless, policy analysis is a highly-skilled job.  Tasks such as modelling a particular legal, social, or economic process, gathering and presenting new information about the prevalence of some social problem, or predicting the likely effect of a proposed reform can be done well or poorly by a set of fairly objective standards, and doing them well is the only way to come up with useful new insights.

Principled political commitments can and do inform empirical inquiry.  They are often the basis for deeming a research question important, and they are relevant to a scholar’s ultimate recommendations.  But they can’t replace empirical inquiry.  For this reason, I believe that insofar as think tanks hold themselves out to be engaged in the pursuit of new policy knowledge, they have an institutional obligation to establish working conditions for scholars that foster and reward good epistemic practices.

Think tank executives work very hard to accomplish this task, but it is an inherently challenging one, because think tanks are not supposed to produce brilliant analyses that no one reads.  They are on a mission to change the world we live in by producing research with “impact.”  Think tank scholars are thus expected to publish work that gets attention from the media and serious consideration from policymakers.  If there is no constituency within these two groups that is receptive to a scholar’s findings, those findings may receive little notice, leaving that scholar’s career languishing.

It can also be professionally dangerous for think tank scholars to generate findings that attract the wrong kind of notice, or which are are deemed helpful to the wrong people.  Some think tanks are more explicitly committed than others to advancing a particular notion of a just society through their work.  But insofar as all think tanks are committed to doing work that has an impact on public policy, it’s implausible that even the most ecumenical “university without students” is utterly institutionally indifferent to the kind of impact its scholars have.  Because public policies affect people’s lives, such indifference might even be considered irresponsible.  A think tank has a legitimate interest in choosing to support research that is likely to advance its institutional conception of positive social change.  Moreover, insofar as it solicits donations on this basis, it has an obligation to its supporters to do so.

These institutional imperatives, however reasonable, create a moral hazard for think tank scholars.  To be ethical, a think tank scholar must adhere to good epistemic practices regardless of where they lead her.  To be successful, she must generate findings that will interest constituencies in media and government and arrive at conclusions that do not appear to undermine her employer’s overarching institutional commitment to advance the cause of a just and humane society.  In my experience, scholars work very hard to maintain good research practices amid potentially conflicting professional imperatives.  But it is not always an easy task, and widespread reluctance to discuss this challenge can give rise to a sense of isolation that makes it more difficult.

I hope that this blog kindles a constructive conversation within the public policy community about how best to meet the ethical challenges of our work, because think tanks are valuable institutions.  While academics often focus on methodological minutiae, think tank scholars address pressing public questions.  While academics sometimes conceal the ideological convictions that inform their research, think tank scholars freely disclose their prior commitments.  While academics frequently write only for each other, think tank scholars magnify the impact of important, university-based research by translating it for a more general audience.  I’ve spent most of my career as a think tank scholar because I believe that think tanks have an important positive impact on the policy-making process that academia cannot replicate or replace.

I decided to start writing about think tank ethics because I believe that a community of smart, high-minded people who have a complicated ethical challenge in common can figure out how to meet that challenge even better if we talk to each other about it.  This blog will offer news analysis of events related to think tank ethics and governance, highlights from interviews with some of the many leading think tank executives and scholars who have generously shared their time and insight with me, and some preliminary analyses and proposals about which I’d like to receive feedback before they become part of the book that I will soon begin to write on this subject.  Please keep me on my toes with critical engagement in the comments or, better yet, at your own personal or institutional blog.  A competition of ideas on the subject of think tank ethics and governance can only benefit the institutions on which so many of us in think tanks, academia, government, media, or philanthropy rely for professional fulfillment, research, insight, or positive social change.