Missing Institutional Heritage

By on May 13, 2013 in Governance, Mission, News Analysis

Some think tanks style themselves “universities without students,” but most are what I call “mission-driven” institutions.  A mission-driven think tank produces (ideally rigorous) public policy research.  But it is also supposed to stand for something: a particular vision of a just society.  Last week’s high-speed train wreck at the Heritage Foundation should give pause to every supposedly mission-driven think tank that has a neglected or under-defined institutional mission.

My former Harvard classmate, Jason Richwine, resigned from Heritage last Friday amid public outcry over his 2009 doctoral thesis, “IQ and Immigration Policy.”  In it, he argues that Hispanic immigrants as a group have a lower average IQ than do U.S. citizens born in the U.S., and he recommends that IQ tests be administered to prospective immigrants as one criterion for admission to the country.

Some immigration reform supporters imbibed more than a little Schadenfreude as the news cycle imploded.  Not I.  I remember Jason as a generous and genial man who refused to accept compensation in return for patiently tutoring me for an analytical methods qualifying exam in 2007.  I find Jason’s research interests disturbing and his policy recommendations wrongheaded.  But he is not an unkind person, and I feel badly for him and for his family.

I have neither time nor inclination to digest a stack of research about what Richwine calls “the construct of IQ” in order to develop an informed opinion about the quality of his empirical work.  Reports are mixed.  Tufts Professor Daniel Drezner is unimpressed, while Harvard Professor Richard Zeckhauser, who served on Richwine’s dissertation committee, calls it “careful.” (Disclosure: Zeckhauser is a former professor of mine.)

But even if Richwine’s empirical claims were well supported, I would find his policy proposal—that aspiring immigrants to the U.S. should be admitted or denied based in part on their IQ scores—philosophically objectionable for reasons that could well indicate that Richwine is a bad ideological fit for Heritage.  Responding to Slate’s Dave Weigel, Zeckhauser usefully distinguishes the empirical from the normative:

“[M]y view is that none of his advisors would have accepted his thesis had he thought that his empirical work was tilted or in error. However, Richwine was too eager to extrapolate his empirical results to inferences for policy.”

Policy recommendations necessarily depend on a combination of empirical findings and ideological commitments about the nature of a just society.  From the perspective of justice, IQ importantly differs from acquired job skills.  Job skills reflect individuals’ choices to improve their life prospects.  It is certainly, sadly, true that not everyone has an opportunity to acquire such skills—a consideration in favor of welcoming at least some low-skilled immigrants.  Nonetheless, a policy of favoring prospective immigrants who have job skills is more intuitively defensible than favoring some over others based on one narrow measure of undeveloped talent.  Richwine’s proposal reasonably offends people who believe that our laws—even our immigration laws—should not deny important life opportunities to individuals based on prejudgments about their inborn capacities.

Heritage is a mission-driven think tank—it exists to promote a specific, conservative vision of a just society, and to develop policy proposals that are consistent with that vision.  Success in this mission depends critically on hiring scholars who share Heritage’s animating values.  If Heritage is institutionally committed to a conception of equal opportunity that precludes legal discrimination on the basis of inherent characteristics rather than choices and accomplishments, then Richwine is a poor ideological fit there.

But if this is the correct story, then Heritage should not have hired Richwine in the first place.  Opportunistic firing (or even “resignation acceptance”) practices don’t somehow excuse careless hiring practices.  Indeed, both are costly to wonks and to the institutions they inhabit in multiple ways.

Most obviously, a supposedly mission-driven think tank with no clear idea about who fits under its ideological tent doesn’t really have a mission.  I don’t mean to suggest that a think tank’s mission should be narrow, or that its scholars must agree on everything.  On the contrary, a think tank that demands lockstep agreement will never retain smart scholars.  But if a think tank has a handful of truly defining commitments, those commitments should be clearly articulated lest the institution become media-driven or money-driven in lieu of advancing its mission.

Moreover, mission-driven think tanks that lack clear standards of inclusion—a category that includes nearly all of them—can be fearful places to work.  One need not dabble in the dark arts to find oneself suddenly unemployed for straying too far from an ideological reservation with invisible fences. How far is too far?  Most of the time, wonks find out only when they are handed their walking papers, asked to abandon cherished projects, or pressured to recant unacceptable statements.

Fear of incompetency can promote research excellence.  Fear of being branded a heretic often precludes it.  Most policy wonks live with too little of the former kind of fear and far too much of the latter.  I believe that many mission-driven think tanks could achieve a better balance by reforming their hiring and firing practices simultaneously, bearing in mind a few general guidelines:

Clarity about shared ideological commitments – Think tank mission statements are usually lofty-sounding but vague.  If a think tank has a set of core ideological commitments about the nature of justice and the proper role of the state, these should be specifically articulated somewhere, so that both financial supporters and scholars have clear expectations when they decide whether to invest time or money in the institution.

No mandatory empirical beliefs or policy conclusions – Insofar as a think tank holds itself out to be engaged in public policy scholarship as opposed to mere issue advocacy, it cannot be committed to predetermined findings and conclusions.  These are the results of good public policy research, not the premises.  A wonk who fears learning the “wrong” thing will fail to learn anything at all about the world, and her think tank will fail to make a meaningful new contribution to the public policy debate in which she participates.  Unhealthy, research-spoiling fear squanders scholars’ talents and supporters’ money.

Greater emphasis on empirical skills – Good empirical skills are the best preservative of truth under pressure. Public policy scholars often add value by developing new public policy proposals, an essentially interdisciplinary task that requires empirical findings, ideological commitments, a savvy awareness of the location of the Overton window, and a working knowledge of any potential legal hurdles to implementation.  Policy wonks do not necessarily need to be able to produce peer-reviewed social science research, but I’m going to go out on a limb and maintain that they should be able to understand and evaluate it if they address empirical questions in the course of their work.

Term contracts for policy scholars – I understand why mission-driven think tanks don’t grant tenure.  Although adults’ ideological commitments are usually fairly stable, they can shift over time, sometimes dramatically.  But the hasty departures that accompany media firestorms ill serve both think tanks and scholars.  I wouldn’t blame Heritage at all for concluding that Richwine’s personal views are inconsistent with Heritage’s principled ideological commitments.  Indeed, they are inconsistent with mine.  But what we saw last week was a desperate damage control effort, not a sober reevaluation of Heritage’s staffing profile. Ubiquitous at-will employment arrangements in the think tank world diminish research quality and creative thinking by fueling the debilitating kind of fear.  They also encourage a long-term pattern of scandal mongering and sometimes-feigned outrage among partisans and in the press.  Replacing at-will agreements with standard two- or three-year employment contracts could paradoxically empower think tank executives to make thoughtful personnel decisions by tying their hands until the smoke clears.

Heritage was humiliated last week because its leaders failed to make personnel decisions thoughtfully in accordance with a clear institutional vision.  Let it serve as a cautionary tale for other mission-driven think tanks.  In a world of clear—not narrow!—standards, policy scholars could determine with more confidence where their own ideological commitments make them a good fit.  In a world without them, we can expect more embarrassing departures and, I fear, more subtly compromised research.

Tax Day Images From Think Tanks

By on April 15, 2013 in Data Visualization

 

CBPP 4-15Heritage 4-15CBPP 4-15-2

Cato 4-15

Mercatus 4-15

 

Perspective: David Boaz on Partisanship, Intellectual Engagement, and Taking the Long View

By on April 12, 2013 in Partisanship, Perspectives

This post is a part of a recurring series titled, “Perspectives on Think Tank Ethics and Governance.”  Each Perspective post will feature lightly edited highlights from a personal interview.  In this week’s post, David Boaz, the executive vice president of the Cato Institute, comments on partisanship, intellectual engagement, and the importance of long-term thinking.  I worked as a policy analyst at the Cato Institute between 2002 and 2006, and I remained an unpaid “adjunct scholar” there until 2012.  The interview took place on October 26, 2012.

Mr. Boaz, the Cato Institute has a reputation that is probably unique in Washington for its degree of commitment to the principle of nonpartisanship. Can you tell me a little bit about the institution’s history with respect to this idea of nonpartisanship?

BOAZ: Well I think our nonpartisanship stems mostly from the fact that we don’t like either one of the major parties. I have to be careful saying “we.” There are a lot of people here [at the Cato Institute], and I am sure some of them have partisan views. But I think the founders, those of us who’ve been here for a long time, the Board of Directors, we tend to be pretty skeptical of both major parties and therefore it’s easy to avoid partisanship.

I think that from any think tank the work should be judged as the work, and it doesn’t matter if you know somebody worked in the Clinton administration or the Bush administration, you should still be able to judge their work. Is it properly sourced? Do the numbers add up? Is it addressing the real problems or dodging the tough issues? All those things can be judged, I think, without knowing [analyst’s identities]. We could publish studies blind, like SAT applications or orchestra auditions, and then we wouldn’t be distracted by who is the author or what’s his real point.

But, we do know that when you come into a discussion, if you’re hired to represent a business, a union, a political party or any other specific interest group, then it is difficult to remain independent. So being independent of both parties—feeling yourself independent of both parties, feeling internally that that’s not an issue for you, that you don’t have to worry about whether your work it’s going to be helpful to or approved by leaders of any political party—I think that’s important.

And obviously, where you get your money is significant, if the money is coming from partisan sources. That doesn’t mean just that somebody is a Democrat or somebody is Republican.  Most Americans are either Democrats or Republicans, and according to a lot of polls, even if they say they are independents, they are really pretty much Democrats or Republicans. But there is a difference between people who vote one way or another and those who possibly set up organizations with the intention of advancing a particular party’s interest.

A lot of folks who promote the idea of being in a partisan coalition argue that it is the only way you can really influence what happens in Washington, DC. How do you push back against arguments like that?

Well, if you’re trying to change people’s ideas, and surely over the long term that’s what intellectuals want to do—they want to persuade more people that their own ideas are right, whether that idea is something narrow on education policy or something broad, like Marxism or libertarianism—you can’t be just in one coalition and talk only to the people in that group. And I know it’s easy for people to do that. I do think that our current media environment, particularly social media, make it really easy to play to the crowd. You’ll get a lot more “likes” and “retweets” if you say something that a body of people who are following you will already agree with.

On the other hand, I think you’re more likely to change people’s minds if you make arguments without fear or favor, and if you make them in a way that is accessible to people of all political views. This is one argument for always showing your work, showing your data. One of the arguments against making religiously-based claims in public policy is, well, it’s kind of like a black box: if I don’t start out believing the same thing you do about religion, there’s no reason for me to agree with your conclusion.

On the other hand, if you take a concept that you came to on the basis of your faith, but you demonstrate with data, reason, or other kinds of generally accessible arguments, that the policy conclusion you’re getting to seems to have good results to people who don’t necessarily share your faith, then you’ve made stronger argument. So, I think if you want to change people’s minds, then you don’t want to only talk to people who already agree with you.

It sounds to me like you are making a comparison here between a short time horizon and a long time horizon. Do you think that think tanks need to be focused more on the longer term?

That is certainly part of it, you’re right. If your time horizon is the next election, then you are going to think about policies that work there. It’s also true, I suppose, that if your time horizon is short, you think about policies that are politically feasible within the current Congress or the current administration. If you think about the long term, then yes, I think you’re less likely to be swayed by partisan concerns, and being less swayed by partisan concerns allows you to think long term.

If you are interested in changing the intellectual dialogue over the long term, then you don’t want to be caught up in immediate partisan or electoral concerns. I would argue that changing the intellectual dialogue over the long term will, in the end, change the political dialogue.  And it will presumably change policy, if in fact you succeed in changing people’s ideas. But it is a long-term process.

Debasing the Currency of Criticism

By on April 10, 2013 in Research Quality

Public policy research is often methodologically mediocre.  You might think, therefore, that more methodological criticism would help!  Unfortunately, I don’t think it will be that simple. Technical critique is too often an emergency escape hatch for tired ideologues reluctant to litigate serious challenges to their principled commitments.  Paradoxically, research quality may suffer as a result.  A drained swamp lowers all boats.

Nearly every activity, including research, can be evaluated according to two types of standards: internal and external.  Internal standards define the activity engaged in, while external standards evaluate that activity in light of the purpose it serves, the values it reflects, or its likely practical effects.  For simple activities, internal standards are less controversial than external standards.  We can usually more-or-less agree about what some activity involves in order to determine whether someone is or isn’t engaged in it.  External standards are often controversial because they reflect disputed moral, political, or aesthetic values.

“Speech,” for example, is the activity of verbally expressing ideas using language.  This is an internal standard for speech, and there are various ways in which our efforts to speak may fall short of meeting it.  An infant who cries for a bottle expresses herself verbally, but because she does not know how to articulate words, her cries fail to qualify as “speech.”  If you have laryngitis, you may form words, but if no sounds result, you, too, are failing to speak.  If my mouth is numb following a dental procedure, I may emit sounds but have difficulty forming words.  You might say that I am “speaking badly” if my mumbling efforts are partially understandable.  In the context of an internal standard, “bad” speech lies on a continuum between what is inarguably speech and what fails to be speech at all.

But there is another, completely different way in which speech can be “bad.”  Speech is often called “bad” if it is spiteful or inaccurate.  Speech can also be bad in the sense that it promotes harmful behaviors or unjust policies.  These are all external standards for speech.  To raise the question of whether or not some particular speech is bad by some external standard is to raise philosophical questions about the validity and importance of that external standard as well as empirical questions about how the speech in question measures up.  Speech can be very, very bad by an external standard while being good by the internal standards of speech.  It makes perfect sense to say that Mussolini gave “bad speeches,” while at the same time conceding that he “spoke very well.”

Research of all types can be evaluated by both internal and external standards.  Ideological commitments are external standards for the wonkish activity of public policy research.  They should therefore function as figurative book-ends: as a basis for evaluating the worthiness of a particular research question, and as a basis for recommendations in light of a research result.  In between, a wonk must adhere to the internal methodological standards applicable to public policy research in order to do her job well.

Because research is a complicated activity, its internal standards are disputed among experts and poorly understood by the public.  Most public policy research combines ideologically-charged subject matter with contestable methodological choices.  The result is a grave moral hazard:  critics with fundamentally ideological complaints can till once more the exhausted soil of first principles…or, they can apply their efforts to the more pliant field of methodological critique!

Denying the validity of inconvenient research frees advocates from the challenging task of accommodating inconvenient facts.  As Economist blogger Will Wilkinson explains, this rhetorical strategy is the chattering class’ current Nash equilibrium:

Perhaps it’s wishful on my part to think, as I do, that most economically literate observers really do understand that raising the minimum wage will screw up the prospects of a fair number of poor young workers.  Those who favour raising the minimum wage anyway just think that, all things considered, that’s a price we ought to be willing to pay. But they can’t say that, just as second-amendment enthusiasts can’t say that an occasional grim harvest of kindergartners is a price we ought to be willing to pay for the freedom to own guns.

Charges of ineptitude have become so pervasive that the research signal gets lost in the media noise.  Every study bearing on a controversial issue, if it is reported on at all, will be criticized by ideological hostiles as a failure.  The resulting foxhole solidarity discourages wonks from calling out fellow travelers’ weakest work.  Methodological criticisms are roughly equally effective regardless of their merit, because few journalists, pols, and voters can evaluate their strength, and even fewer actually will.

Knowledgeable wonks and writers should identify genuinely weak research, but we should be equally ready to praise methodologically sound work even when its findings and prescriptions are unwelcome.  When ostensible technical fault-finding is actually sublimated ideological frustration, it threatens to reduce research quality at think tanks by debasing the value of technical criticism.  Good research is expensive and difficult to produce, while the bad stuff is relatively cheap and easy.  If no one signals the objective difference, bad research will inevitably crowd out the good.  Perhaps a consortium devoted to recognizing the best research at each participating think tank could begin to realign incentives towards producing high-quality work.

Ideology, Partisanship, and Scholarship

By on April 8, 2013 in Mission, Partisanship, Research Quality

Think tanks are often ideological and sometimes partisan, but critics who deploy these charges interchangeably miss a distinction that makes a difference.  “Once seen as non-ideological ‘universities without students,’ the American think tank has, in many cases, become a partisan stalking horse,” Pacific Standard’s Emily Badger complained.  On the relationship between partisanship and ideology, she averred:  “Those two terms…have become increasingly synonymous in modern politics.”  But while partisanship and ideology can each create conflicts of interest for think tank scholars, the two pressures are distinct, and they often conflict with each other.  Only by separating one from the other can we think carefully about how to insulate think tank research from influences that will diminish its quality.

When we refer to a person’s “ideology,” we are usually talking about a stable set of somewhat general (but not maximally general) moral and political principles. So understood, ideological principles serve as normative guideposts for, well, nearly everyone who ventures opinions about public policy. Indeed, even some fully-fledged political philosophers see value in these commitments. John Rawls called them “considered convictions,” and he believed that they play a legitimate role in philosophical deliberation. Elected representatives, opinion journalists, policy wonks at explicitly ideological institutions, grass roots advocates, and family members who discuss current events over the dinner table invoke their ideological commitments as a sort of magnetic north when they explore public policy ideas.

Studies suggest that adults have fairly stable ideologies, but they can evolve over time.  Some people experience a dramatic ideological conversion once or twice in their lives, either spurred by philosophical or religious study or as a result of coming to grips with facts about the world that cast doubt on the validity of their formerly-held principles.

Academics, traditional reporters, nonpartisan government analysts, and other political participants with highly technocratic orientations sometimes purport to eschew “ideology.” But as The Atlantic’s Conor Friedersdorf recently pointed out, technocrats need ideologies, too! Ideology is what analysts consult in order to decide which research questions are socially important and which are unworthy of exploration. As former Cato Institute research fellow Will Wilkinson explained:

There’s no avoiding the fact that, if you’re doing anything with policy at all, you’re trying to achieve some goal. If you think that the goal is one that’s worth having, you have to have some rational justification for why that’s the end that we ought to be aiming at.

Even those whose research agendas are set by others—lowly research assistants, OMB economists, and Iranian nuclear scientists come to mind—must at least assure themselves that their analytical skills are not furthering some evil purpose. To answer the question, “Is my job morally acceptable, or must I quit?” a researcher must consult her ideology. When academics undertake projects for the purpose of “advancing the science” of their discipline by refining its methods, they must ask and answer the same question.

“Partisanship” is a different thing. Political partisanship has to do with being a team player in a coalition that seeks electoral victory. To be “partisan,” therefore, is to speak and act for the purpose of advancing the electoral prospects of one’s party. Ideological convictions frequently determine the political party with which a person chooses to make common cause. This is a always choice of lesser evil, though, because ideologies can be as unique as snowflakes, while political parties are coalitions. A religious conservative does not share an ideology with a libertarian, nor does a labor democrat share an ideology with a deep ecologist.

Inevitably, the demands of partisanship will conflict with the demands of ideology. In 2003, for example, small government conservatives at the Heritage Foundation had to decide whether or not they would support President George W. Bush’s creation of Medicare Part D—a huge and unfunded new federal entitlement program. In that case, the think tank’s ideological commitments rather impressively prevailed over the pressures of partisanship. On the other hand, Heritage’s Obama-era about-face on the ideological acceptability of individual health insurance mandates raised questions about whether partisanship does not sometimes masquerade as ideological purity.

Both ideology and partisanship can create conflicts of interest when a public policy scholar hopes, for partisan or ideological reasons, that her analysis will have a particular outcome. Suppose that an analyst at Think Tank A is ideologically committed to a principle of noninterference with the internal conflicts of foreign nations. She may examine the effect of interventionist policies on the federal budget in the hope that her findings will persuade people concerned about the deficit to support a less interventionist foreign policy. If she discovers that foreign policy has little or no effect on the deficit, she will be disappointed.  Similarly, a partisan may hope to vindicate the education reforms of a particular president during re-election season. If he discovers that the reforms actually did not benefit students, he will be disappointed by his findings as well.

This kind of conflict does not bedevil wonks alone. Traditional academics also hope that their research will reflect favorably on their ideological or partisan commitments. Princeton political scientist Martin Gilens, whose personal views are liberal, recently published a book in which he reports that George W. Bush’s policies reflected the preferences of poor and working class voters far more accurately than did those of Presidents Clinton or Johnson.  Gilens jokingly described his surprise and chagrin:

Certainly for Bush 43—George W. Bush—I would not have expected high levels of responsiveness to anybody except maybe the most affluent.  Like a good political scientist, when I found these results, I said there must be some mistake in the coding.  I did everything I could, you know, to make them go away.  But they were very persistent.

Ideology is not wholly dispensable for public policy scholars, regardless of where they work. Therefore, ideological conflicts of interest cannot always be sidestepped.  Because good research is a lot like good driving, such conflicts may negatively impact research quality despite a scholar’s honest best efforts.  Ideological conflicts should therefore be acknowledged and carefully managed by both scholars and institutions in a way that protects research quality.  At explicitly ideologically committed institutions, I believe that this challenge is among the biggest challenges in think tank ethics.

A scholar’s partisan allegiances, if any, have no necessary role to play in her research process. This is not to say that a passionate partisan is capable of entirely ignoring the partisan implications of her research, but it seems like an unambiguously worthy goal. Indeed, nearly all U.S. think tanks make public claims of nonpartisanship. These commitments are encouraged by the tax code, but when and where they are taken seriously, they promote good research practices by minimizing an unnecessary source of bias. Indeed, it seems to me that nonpartisanship is especially valuable to think tanks with ideological missions. It is challenging, but I believe possible, to maintain good research standards in an ideologically committed institution. Partisanship simultaneously undermines both organizational goals.

Perspective: Edwin Meese III Discusses Institutional Mission Fidelity and the Advantage of Broad-Based Support

By on March 14, 2013 in Funding, Mission, Partisanship, Perspectives

This post is a part of a recurring series titled, “Perspectives on Think Tank Ethics and Governance.”  Each Perspective post will feature highlights from a personal interview.  In this week’s post, Edwin Meese III, Ronald Reagan Distinguished Fellow Emeritus at the Heritage Foundation and former chairman of Heritage’s Center for Legal and Judicial Studies, comments on mission fidelity, management practices, and the advantage of a broad base of support.  The interview took place on October 25, 2012.

Mr. Meese, we were talking about having an inner discipline about being committed to the mission. [How do you] not allow the possibility of specific donations to divert you from your principles?

MEESE:  As you say, we don’t do contract work, [and] we don’t take any governmental funds from any government. We try to be very transparent in all that we do. We are clear about what our mission is, clear about what our principles are, clear about what our objectives are, clear about what our overall philosophy is.  [P]eople who desire to become members of Heritage, who desire to contribute to Heritage Foundation, understand that that they are contributing because they want to be part of and support the work we do rather than the other way around.

Everybody has a mission statement, and it’s often highly general. So how do you figure out what [the mission statement] means in the context of specific research programs?

MEESE:  Well, for one thing, we set up our primary principles of individual liberty, limited government, free market economics, strong national defense, support of what we call “traditional American values.” That [last one is] the only one that’s a little bit nebulous there, and by that we mean family and religion, we mean patriotism, we mean things like a commitment to work, individual responsibility.  Then, we have our mission: to promote a society in which freedom, opportunity, prosperity, and civil society flourish. We say clearly, we are nonpartisan, but we are conservative, so we are committed to a philosophy but not to a political party or any candidates.

It’s hard to resist partisan pressures when they arise. How do you do that?

MEESE:  Well, it’s not hard, because we’ve been doing it so long, and everybody who is here understands that.  But we are attentive to the issue, and we have a system within our management structure so that we provide guidance to all of our people, particularly in an election year, as to how they are to act and how they are to respond to things like questions from a political organization of any sort.  [T]he organizers must understand that we would give the same thing to their opponents.

For example, if they call, we provide briefings to candidates across the board. Now, obviously, one party is more likely to ask questions and to seek information from a conservative organization than the other party, but we do [briefings] on a nonpartisan basis. The second the thing we do is, we don’t get involved in support—Heritage itself—in support of legislation or opposition to specific legislation. We do have a 501(c)(4) organization, Heritage Action for America is a sister organization, and they they are able to lobby, but again, not to support specific candidates or anything like that.

A lot of the research that scholars at think tanks do is more applied than what’s going on in academic departments. Where do you see the position of think tanks and what they do, related to academia on one side and activists on the other?

MEESE:  I would say that that we pride ourselves on [our] information being factual, on being supported by evidence, by looking at counterarguments, and by trying to provide recommendations in terms of what policy will be. Our policy ideas, if you will, can be useful, and hopefully would be adopted by people of whatever political party, based on the overall direction of our philosophy. Our President, Ed Feulner, has said that he often hears from Democrat congressman that they look at Heritage materials—our issue briefs, for example—and they can use everything up to the final page, which has the the conclusion and the recommendations. They said they tear that page off and use the rest of it. We consider that a good thing.

Do you have any internal ethics policies or conflict of interest policies and guidelines in the organization?

MEESE:  Gosh, only on political activity. We have clear-cut rules on political activity. But we do have a management structure, so somebody from the beginning people in the organization, policy analysts who start here of as well as support people, all the way to the president, they have someone that can go to for advice and guidance on any of the, what you might call ethics issues. And likewise, we have we have reviews for factual matters, so we have a series of editorial reviews of anything coming out of here.

So, you have a quality review process, where you vet work before it’s published.

MEESE:  Sometimes we may even have more than one. Say we are doing something on national security law. We will want to make sure that it is been vetted by people who are our experts on national security as well as by our legal experts. If they were talking about something like the Law of the Sea Treaty, they would want to have it vetted by, not only their experts from the navigational standpoint, the economics standpoint, but also from the constitutional legal standpoint.

So, this process of review is to make sure that we have [good] quality in terms of accuracy, as well as of course readability and those other qualities. That is part of our regular process. [E]very Monday morning at our management meetings, where we have our our senior management and our next level of management, we have a review of all the topics that are being researched, and what you might call “major policy documents.”

If you were going to give some advice to a new, small think tank that is just getting off the ground and trying to develop best practices, what advice would you give them?

MEESE:  Well, first of all, hire good scholars. Look carefully at people’s background for, obviously, intellectual capability, as well as the other things you need, writing style and all that. Look at their background in terms of their education, and ascertain to make sure they have intellectual integrity. Those qualities you need. And then, of course, a thorough review of the policies of the organization—the things we’ve been talking about, the mission, the philosophy, and those kinds of things—hopefully before they are hired.

The next thing, of course, would be that they understand the review process, and then particularly having a process of how you handle what you might call complaints, criticisms, that sort of thing.  [L]et’s say somebody publishes paper and the next day you have EJ Dionne or someone like that criticizing the paper.  We would want to first speak to the analyst who wrote the paper and probably that person’s superior officer to see whether the criticism was in any way justified or accurate. If it isn’t, we decide what kind of rebuttal we want to have, whether we’re just going to let it go because it’s inconsequential or nobody believes that person anyway. Or, they’ve got a good point, and we go back and check our informational resources to make sure we are accurate. If we’re right, we defend the position. If somebody’s made a mistake, we correct it. In other words, we maintain absolute integrity in terms of factual matter.

How do you handle instances of empirical disagreement when they crop up within the organization? For example, if one person thinks program X tends to increase literacy, and another person can’t find that effect, how do you hammer those things out?

MEESE:  It goes to the next level of management. If we get to a total impasse, then we would figure out how to handle it. Usually it’s not so much on a factual matter as it is, like you say, an interpretation or an idea of how we are proceed from there. Sometimes, take a tough subject like immigration policy, as we have a couple years ago, where we were essentially at odds with what George W. [Bush] wanted to do, and some of the others, and we had people on both sides within the organization, several sides that case. So, what we will often do is put together a working group, and I’m often asked to head them up, or someone else fairly senior here. We would bring all people together and argue it out and ultimately either reach a consensus, which is often what we are able to do, or just have to flat make a decision.

I took a look at your annual reports, and I noticed that you seem to recognize a lot of your donors.

MEESE:  We want to recognize, particularly the major donors, but we also have 700,000 people who are members, so even what we put in our annual report is just a very small fraction of our total body of contributors.

Would you say that there are any advantages to having a very broad donor base in terms of the mission, or being able to maintain a measure of independence from any one donor?

MEESE:  Absolutely, that’s very important. The broader the base, the better, not being dependent on any one individual, or one organization, or one foundation. We are probably, of all the major research and education foundations, less dependent on corporate support than most. It’s certainly less than 6% [of annual revenues]. I’m not sure exactly, but somewhere very, very low. And that’s for a reason, because we have had situations when corporations disagree with us or want us to move in a particular direction that’s favorable towards their ideas or their products. We just don’t do that, so we deliberately keep our corporate support as a small percentage of our total support.

A Shilling on the Side

By on March 12, 2013 in Governance, News Analysis

Buzzfeed’s Rosie Gray reports that former Bush speechwriter Joshua Trevino secretly organized a gaggle of professional writers to opine about Malaysia.  In return, the writers received payments from public relations firms representing the government of Malaysia.  Their columns—which praised the Malaysian government or criticized its political opponents without disclosing the financial relationship between author and subject—appeared in various media outlets, including National Review Online, the Huffington Post, Ricochet, the Washington Examiner, and the San Francisco Examiner.

Some of Trevino’s recruits were affiliated with think tanks at the time, including the Heartland Institute, the American Center for Democracy, and the Manhattan Institute, where I used to hang my hat.  However, none of these writers seem to have published their public relations contract work on think tank websites (with the possible exception of this early 2011 piece by Heartland’s Ben Domenech), nor do they appear to have used their think tank affiliations in their Malaysia bylines.

Which raises the question:  When is your night job a problem?

It’s not uncommon for a policy scholar to have a second source of income.  Young and not-so-famous think tankers are often paid barely enough to cover a modest rent and the clothes they wear for television interviews.  As a novice, I had several colleagues who held second jobs, usually tending bar.  One turned out to be a budding maestro of mixology, but others were not particularly good at their off-hours gigs.  (I personally would have been a painfully incompetent bartender.)  A think tank has no reason to look askance at its bad bartenders, because the two tasks are completely unrelated.  Bad bartending doesn’t reflect on the quality of anyone’s performance as a policy wonk.

Undisclosed public relations contract work, however, does reflect poorly on the primary professional efforts of think tank scholars.  Non-disclosure of a financial relationship with the subject of a published article violates the ethical standards of both journalists and public relations professionals.  It is therefore not surprising that National Review Online, the Washington Examiner, and the Huffington Post have disavowed the Malaysia-themed PR placements, or that the Wall Street Journal has administered a spanking to the writers involved.  There is a difference between off-duty incompetence and an off-duty ethical lapse.

Of greater concern, in my view, is the nature of the lapse.  A think tanker’s failure to disclose public relations contract work as such raises the possibility that he doesn’t see any difference between his wonkish activities—policy scholarship and opinion journalism—and paid client representation.

I used to be a litigation attorney.  I see nothing inherently ethically suspect about paid client representation.  It is just a different activity than either scholarship or journalism.  An advocate for a client takes the client’s best interests as a starting point, and then he makes the strongest possible case for them.  A client-advocate must not lie, but he is also not directly in the business of seeking the truth of a matter reasonably contested.  He is a cog in a larger epistemic process that we hope enables judges and readers to glean wisdom from the public conflict of competing interests.

This is why the mere act of hiring an outstanding lawyer doesn’t cause the public to believe that a criminal defendant is innocent.  On the contrary, if you hire a really fancy lawyer, the public tends to assume that you do, indeed, have a serious legal problem.  Similarly, if you hire a fancy public relations firm, people tend to think that you have probably done something embarrassing or unpopular.  Professional advocates can and do help their clients, but no one mistakes their public advocacy for an independent, expert appraisal.

By contrast, a think tank scholar’s opinion is thought to carry some authority, because he is not being paid to advance anyone’s interests in particular.  Wonks usually apply some combination of abstract ideological principles and well-honed analytical skills to a set of facts in an effort to illuminate the key features of a potentially confusing event.  I afford a reputable think tanker’s opinion some deference in the many policy areas in which I am not an expert, because I assume that it reflects the current sincere belief of a smart person who knows the subject matter much better than I do.

At best, public relations work is an imprudent second calling for a policy scholar, who thus debases the public currency of his expertise. Secret public relations contracts are ethically problematic, because they fool readers like me into affording the arguments of paid client-advocates the benefit of the doubt that we accord to those of independently compensated experts.   Think tanks would be wise to bar their scholars from undisclosed PR moonlighting, which diminishes institutional credibility by raising questions about whether what their scholars do on the job is really so different.  Reputationally, it seems to me that once you’re in for a shilling, you’re in for a pound.

Why Conflicts of Interest Matter

By on March 10, 2013 in Governance, Research Quality

Conflicts of interest matter in the think tank world, because good research is a lot like good driving. If you are like most people, you make an honest effort to avoid getting into car accidents. You look both ways at stop signs, check your blind spot before you change lanes, and drive more slowly in rain or snow. You probably don’t drive drunk, and perhaps you, like me, no longer answer the phone behind the wheel.

Still, you don’t drive as carefully as you might, and your auto insurance policy is partly to blame. “But an accident is a dangerous, inconvenient, and humiliating event regardless of insurance,” you may protest. That’s a fair point. Almost no one who buys insurance thinks: “Awesome. Now I can get into all the car wrecks I want!” Yet, this fact—that well-insured drivers are, all else equal, more likely than others to be involved in serious auto accidents—has been proven about as well as anything can be using large-scale data sets.

Economists refer to this as a “moral hazard” effect—a term that misleadingly implies that an adequate exercise of will power can always overcome it. But that isn’t entirely true when it comes to driving, because driving well isn’t a single decision that we make by deliberately reflecting on all of the reasons that we have. Instead, good driving is an accumulation of a million tiny choices, many of which are habitual or semi-conscious. We don’t fully notice how incentives affect this complex pattern of conduct because we can’t identify every distinct decision point at which they come into play. If we tried, we’d never get out of the driveway.

Incentives influence think tank research in exactly the same way.  Every step of the research process involves thousands of tiny decisions, from the formulation of Boolean searches, to the choice of some variable’s functional form, to the book you didn’t finish because someone asked you to join them for lunch. No matter how conscientiously think tank scholars strive to do excellent research, a conflict of interest makes it likely that, in myriad tiny ways, they are doing their jobs less well. As my former colleague, Tim Lee, once wrote, “Getting the right answer is hard, and you are just less likely to do it if you have a huge financial incentive not to.”

It doesn’t follow that no conflicted research should be done, any more than the moral hazard effects of auto insurance make it the case that we shouldn’t carry any auto insurance. I carry generous auto insurance because I think that avoiding financial ruin and being able to fully compensate others for any mistakes I make behind the wheel are more important, all things considered, than a slight, subconscious reduction in my driving quality. I just believe that think tanks ought to apply that same kind of cost-benefit scrutiny to their policies concerning conflicts of interest.

When I ask think tank executives how they insulate their scholars from incentives that could undermine good research practices, they mention their “bully pulpit” strategy, their think tank’s “value proposition,” or their scholars’ “inner discipline.” Most think tank executives work admirably hard to establish a culture of honesty and excellence. In my experience, and to their credit, think tank executives and scholars usually do “the right thing” when they are faced with a clear moral dilemma. Good cultural values can make light work of the decisions that we actually notice and think about.

But culture alone isn’t enough, because conflicts of interest can subtly bias research in ways that scholars aren’t fully aware of and therefore can’t fully overcome. How can think tanks nudge their scholars in the direction of excellence? It seems to me that think tanks should steer clear of funding arrangements that create conflicts of interest if alternative arrangements are possible, and that they should establish personnel policies that protect the careers of scholars who generate disappointing answers to empirical questions. That few think tanks have established clear policies on such subjects suggests to me that they rely too exclusively on individual integrity, when they should also pay attention to the power of incentives to promote, or undermine, good research practices.

“About Us”

By on March 8, 2013 in Data Visualization

About Us

A word cloud of “About Us” statements from the websites of think tanks ranked in the top 10 among think tanks in the United States by the University of Pennsylvania’s 2012 Global GoTo Think Tank Report.

Image courtesy of Many Eyes.

Perspective: Sarah Rosen Wartell on Ethics, Fundraising, and “Pay for Play”

By on March 7, 2013 in Funding, Perspectives

This post is a part of a recurring series titled, “Perspectives on Think Tank Ethics and Governance.”  Each Perspective post will feature highlights from a personal interview.  In this week’s post, Sarah Rosen Wartell, President of the Urban Institute, comments on ethics, fundraising, and the reputational cost of “pay for play.”  The interview took place on January 31, 2013.

You have emphasized that there are a great diversity of organizations out there that consider themselves think tanks. It is hard to talk about what think tanks do in a general sense. How do you feel that diversity is reflected in a diverse set of ethical questions or constraints? How do different institutional models raise different kinds of ethical questions?

WARTELL:  Even “ethics” is a word that has a set of value propositions implied in it. If you are an institution [that] is crystal clear that [it has] a philosophy, [that philosophy] is embedded in the mission.  It underlies the scholars that they choose, the issues that they choose, etc. Does that work therefore raise an ethical question? Inevitably, there are choices that are made about what work to initiate, what work to publish, and when to publish it. I don’t think there’s anything unethical about an institution that says that its mission is to advance a set of values to then shape its work agenda in a way that advances those values.

Transparency seems to be the key to ethical behavior in that regard. I think the risk is where people—and no institution is without its own lenses, no people are without their own lenses—where you purport to advance fact in a highly skewed way. I do think there are ethical questions there, but I don’t think that there is a problem with a particular think tank choosing to publish work that advances their agenda and not publish work that I might want to publish that advances a different vision.

I think the bigger source of ethical dilemmas in the think tank world comes from the sources of funding. [R]emember, at its founding [the Urban Institute] had very prominent corporate support. My guess is [that] in those days, it was not particularly agenda driven. That was just sort of what companies did. These days there’s a great deal of funding for think tanks that comes from the business community, and I don’t think there’s anything wrong with that. In fact, I hope that we can do a better job of participating in that as a source of revenue, particularly as a source of discretionary revenue that allows you to skate to where the puck is going to be. I also think that those relationships tend to create more information sharing.  If you tend to simply sit in your think tank, and you don’t engage with people in industry, you’re less likely to have [all the relevant] knowledge.  So, I think those financial relationships with corporations often bring a kind of knowledge and perspective [to] the institution that’s very positive.

But there are cases where the outcomes and agenda of an institution get driven by the funding sources. I take money from the Ford Foundation.  Is my agenda driven by the values of the Ford Foundation? Sure. The questions they choose to fund here affect our work. So why is it any different, and I think it is different, if Wells Fargo wants to fund us? In that case, it’s very important to create firewalls and have good judgment. It’s one thing to collect information and get perspective. It’s another thing to let anybody have review or approval over products, to let anybody fund particular products as opposed to an issue area or a set of activities, like conferences, where there’s branding and transparency of the involvement. I think some of the biggest dilemmas in the think tank world are really around this question of retaining your credibility as an institution who comes to independent conclusions without being driven by your sources of funding.

In my prior place of employ, [the Center for American Progress], we did have a corporate giving program. It was for general support. It was unrelated to any advice, etc. But there was a sort of constant churn, inevitably, when they didn’t like something that was coming out of another part of the institution. Some companies said, “Well, I don’t want to support you anymore.” Generally, they were spreading their money around town. That’s the way they do this. But sometimes it became uncomfortable for them, and they would withdraw support, and you had to have a value proposition that that’s not going to cause us to stop doing a body of work.

At different institutions, there are [other] perceptions.  When I hear people talk about different think tanks, they will tell you that this place or that place is more driven by [funding].  I think almost all of them believe that they have policies in place that protect them against being driven by funder’s agendas.  But, people will say disparagingly of a place that it’s a “pay to play” shop.  As a think tank manager, you really want to guard against the perception of that.  I don’t know that I think it’s always fair when it is [said], but I think it is one of the more prevalent ethical issues that think tanks face.

What makes something “pay for play?” Is it pre-publication review? Is it a commitment to reach a particular conclusion? Is it a focused financial interest in the outcome? I’d love to get a better handle on the concept.

WARTELL:  I’ll be honest: I haven’t thought that much about what makes something unacceptable. [T]here are occasional scandals around think tanks. I can remember, when we were starting [the Center for American Progress], it happened that some foreign government gave a great deal of money to, I want to say it was Heritage but don’t remember…

I think [that story concerned] Heritage. Frankly, the Malaysia-Heritage story and that story involving Abramoff are the only ones I know about. I wish I had better folk knowledge.

WARTELL:  I think maybe those were the only ones that ever came out at that level of reporting. I am sure that there are others where the tank took a hit reputationally. I mean, you can go out and hire any economic research firm, and a trade group will do it a lot of the time, to answer a question. The report has some credibility, absolutely, and certain firms bring more credibility than others do. That is just like government-contracted research. There is nothing illegitimate or unethical or anything about that.

Which goes back to my point: transparency. I think the key thing here is if you are doing work because a source of funds provides that work, and the source of funds has an interest in the outcome, [you are criticized if] you are not letting it be known that you have a source of funds.  A bunch of universities were criticized for taking oil sector money. I think there has probably been very similar criticism of universities around tobacco, but I don’t know for sure. There was a report published [by the Center for American Progress] called “Big Oil University,” and there was controversy around the report itself. It was about universities that had set up research shops that were funded with a great deal of [oil money].

Were they researching a subject about which oil companies have a stake?

WARTELL:  Clean energy, in many cases. You’re not going to be able to get resources to [do] that kind of work if you are not, in many cases, taking private sector money. So the real question is about whether [you have] the controls and processes in place, and universities have gone through this process with a great deal more rigor probably, of trying to figure out what their standards are.  Think about engineering schools, and the like. Tons of work [is] done that there is probably a commercial interest in for firms. Is that unethical? Again, I don’t think so. Transparency is hugely important. Healthcare research, pharmaceutical company support for drug research is huge. Think tanks are, I think, less far along in terms of the development of the jurisprudence, if you will, about how to think about this.

That’s where I’m hoping to help.

WARTELL:  Well, let me know what you think, because we are trying to figure out our own policies in this regard.