In December, 2019, JPE published “The Social Cost of Carbon with Economic and Climate Risks”, a paper I wrote with Yongyang Cai and Thomas Lontzek. I am not listed as an author but the first footnote states that I was a co-author. The history behind that fact is an example of the “tyranny of the top five”, a phrase popularized by Jim Heckman and Sidharth Moktan in their paper “Publishing and Promotion in Economics: The Tyranny of the Top Five” https://www.aeaweb.org/articles?id=10.1257/jel.20191574.
The whole story is long and complicated but the key fact is simple: Jim Heckman was handling the submission at JPE, but declared that if I criticized the methodological demands of a former JPE editor then he would reject the paper. My criticisms would not appear in the paper, but instead appear in other documents, perhaps even a blog. That did not matter to Heckman. If I engaged in scholarly criticism of a former JPE editor outside of the paper, he would reject the submission. I took my name off the paper to defuse Heckman’s threats.
The issue: Economists should admit how little they know
I next describe the underlying methodological issue, which I regard as being very important if economics is to have any credibility as a “science.” This paper was originally submitted to JPE in November, 2012. 11 1/2 months later we received a few mostly unfriendly referee reports. Over the course of the next year, we submitted two revisions. I described the basic model and findings in a previous post. In November, 2014, the JPE editor rejected the third revision, which was a surprise given some of the comments.
One of the referees said that “I think that the authors have done a very good job in merging so many elements of the evolution of the climate and economic systems, and solving such a rich and complicated model. I am quite sure that this will remain the most comprehensive climate change model to be solved in the economics literature for a very long time.” In this paper, we applied this “most comprehensive climate change model” to estimate the social cost of carbon, a.k.a., the optimal carbon tax. It was obvious that DSICE could also be applied to many other issues, as demonstrated by other published papers built on it.
However, the JPE editor and referees said that there was no economic content to the paper because we reported the results for a variety of Epstein-Zin utility function parameters, and got different results for different parameter choices. We relied on estimates of those parameters that had appeared in the macro finance literature. Those estimates differed for the usual reasons: different data, different auxiliary assumptions, etc. I find it interesting that those papers were deemed to have “economic content” but our work which showed what those estimates implied about the SCC did not have “economic content.”
Instead, JPE said we should have settled “down on a specification of the model and a set of parameter values that provide a reasonable description of dynamics.” We all know that there is great uncertainty about both the right structure of economic models as well as the specific values for the key parameters. I view arguing for one vector of parameter values as essentially a fraud, and my applied papers, including previous papers published in JPE, always examine a variety of parameter values and look for robust features of the results. I do not make outrageous claims for specific values of parameters, even when demanded to do so by a journal, because that would be dishonest.
Heckman’s response and threats
I expressed my opinions in an email to Heckman. He told us to change the title of the paper, resubmit it to JPE, and he would handle the new submission. I have often expressed strong opinions against the value of this extreme form of calibration, and lectured on the value of Uncertainty Quantification. This issue is central to my views on the value of computational economics. It was a major theme in the RDCEP project which I helped create. Therefore, highlighting the methodological demands used by a journal editor in deciding the fate of a submission is appropriate in discussing how economics should be done. The new submission was a new submission, and the old submission was in the past.
In August, 2015, Heckman sent us six, mostly hostile, referee reports, a rather speedy turnaround by economics standards. In a followup email conversation after that, Heckman made his threats: criticize the previous JPE editor and the paper will be rejected.
Our progress on the revision was slow. Both of my co-authors were on the job market then settled into new jobs. We worked mostly on responding to the criticisms, producing several long memos addressing the many issues. When we finished the revision in December, 2017, I expected that it would take significant time before we received the second round of referee reports and even longer to reach a final decision. I wanted to get out from under Heckman’s gag order. I took my name off the paper so that my name would not interfere with the proper scholarly evaluation of the submission. In May, 2018, the paper was accepted.
Documentation
I have gathered my communications with people at JPE and the University of Chicago on this matter. I thank Jim Heckman, Harald Uhlig and others who have give me permission to post these documents. The documents are available at https://kenjudd.org/documentation-for-tyranny-of-the-top-five/https .
I believe that documentation is the only way to avoid misunderstandings that may arise if I just characterize the conversations.
Conclusion: Economist’ fears of editorial animus are justified
This only purpose of this blog post is to document the nature of the evaluation process at economic journals. Many people worry about referees and editors punishing them for legitimate criticisms of the literature, or for not being in the right “fight club.” This experience proves that they should be worried. The only unique aspect of this episode was that I pushed back on these threats.
This was not just a Heckman versus Judd matter. I informed many people connected with JPE about these threats and no one voiced any criticism of them. Many of the editors who apparently approved of Heckman’s threats have also served as editors at some of the other “top five journals”. I have described this matter to many people. No one in leadership positions in economics has expressed any disapproval of JPE’s threats, or even any surprise. As far as I can tell, no top economics journal has promised that their editors would not make similar threats.
Nothing I say here or in the documentation will be a surprise to most people. Many of them are aware of similar interactions with journals but say nothing, which is a rational response. The contribution of this blog post is to take a specific example and make it “common knowledge”.
Many economists claim that economics is a science. I dare them to find a real scientist with serious experience as an editor of a scientific journal who will declare publicly that the actions of UC and JPE are consistent with the standards of science.
The problem is not confined to economics. I contacted the leadership of of the University of Chicago describing this episode, including Provost Diermeier (a political scientist) and President Zimmer (a mathematician). The Provost told me that nothing done by UC professors violated UC standards, but President Zimmer’s office is also responsible for that response. President Zimmer’s wikipedia page reports:
As Zimmer noted in an address to the Chicago Humanities Festival in 2017, the work of faculty and students to confront new and different ideas through education and research “only happens at the highest level in an environment of rigor, questioning, and free and open discourse”.
I look forward to President Zimmer explaining how UC’s approval of the actions of its professors in this matter is consistent with his views on creating a supportive environment for education and research.