The Validity and Accuracy of Commonly Used Ideology Measures: A Consumer's Guide
DOI:
https://doi.org/10.15763/issn.2374-7781.2004.25.0.201-220Abstract
Are measures of legislator ideology derived from behavior accurate and valid? Past research says yes. However, the benchmarks used to reach these conclusions are often also based on legislators’ public actions. Non-ideological factors that cause legislators to take specific issue positions may be highly related across measures and mistakenly lead scholars to believe that action-based estimates are valid. This question is important because scholars frequently wish to use action-based ideology estimates as explanatory variables. Without independent validation, it is unclear whether the results of these studies are valid or the product of measurement error. Applying an ideological benchmark that is not based on legislators actions, I evaluate the validity of several commonly used ideology measures. The results show that action-based ideology measures produce valid estimates of legislator ideology.References
Adams, Greg D., and Chris Fastnow. 1998. On the Difference Between Legislative Ideology Measures. Presented at the annual meeting of the Midwest Political Science Association. Chicago.
Adcock, Robert, and David Collier. 2001. Measurement validity: A Shared Standard for Qualitative and Quantitative Research. American Political Science Review 95:529-546. http://dx.doi.org/10.1017/S0003055401003100
Ansolabehere, Stephen, James M. Snyder Jr., and Charles Stewart. 2001. The Effects of Party and Preferences on Congressional Roll Call Voting. Legislative Studies Quarterly 26:533-572. http://dx.doi.org/10.2307/440269
Bailey Michael, and David Brady. 1998. Heterogeneity and Representation: The Senate and Free Trade. American Journal of Political Science 42:524-544. http://dx.doi.org/10.2307/2991769
Berelson, Bernard R., Paul F. Lazarsfeld, and William N. McPhee. 1956. Voting. Chicago: University of Chicago Press.
Bernstein, R.A. 1989. Elections, Representation and Congressional Voting Behavior: The Myth of Constituency Control. New Jersey: Prentice Hall.
Bernstein, Robert, and William Anthony. 1974. The ABM Issue in the Senate, 1968-1970: The Importance of Ideology. American Political Science Review 68:1198-1206. http://dx.doi.org/10.2307/1959156
Bianco, William T. 1994. Trust: Representatives and Constituents. Ann Arbor: University of Michigan Press. http://dx.doi.org/10.3998/mpub.14057
Bishin, Benjamin G. 2003. Independently Validating Ideology Measures: a look at NOMINATE and Adjusted ADA Scores. American Politics Research 31:404-425. http://dx.doi.org/10.1177/1532673X03031004004
Brace, Paul, and Melinda Gann Hall. 1995. Studying the Courts Comparatively: The View from the American States. Political Research Quarterly 48:31-42. http://dx.doi.org/10.1177/106591299504800101 http://dx.doi.org/10.2307/449117
Brimhall, Dean R., and Arthur S. Otis. 1948. Consistency of Voting by Our Congressmen. Journal of Applied Psychology 32:1-14. http://dx.doi.org/10.1037/h0058213
Brunell, Thomas L., William Koetzel, John Dinardo, Bernard Grofman, and Scott L. Feld. 1999. The R2 Conservative Interest Group Ratings. Legislative Studies Quarterly 24:87-101. http://dx.doi.org/10.2307/440301
Burden, Barry C. 2004. Candidate Positioning in U.S. Congressional Elections. British Journal of Political Science 34:211-227. http://dx.doi.org/10.1017/S000712340400002X
Burden, Barry C., Gregory A. Caldeira, and Tim Groseclose. 2000. Measuring the Ideologies of U.S. Senators: The Song Remains the Same. Legislative Studies Quarterly 25:237-258. http://dx.doi.org/10.2307/440370
Campbell, Angus, Philip E. Converse, Warren E. Miller, and Donald E. Stokes. 1960. The American Voter. Chicago: University of Chicago Press.
Campbell, Donald T., and Donald W. Fiske. 1958. Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin 56:81-105. http://dx.doi.org/10.1037/h0046016
Carson, Richard T., Joe A. Oppenheimer. 1984. A Method for Estimating the Personal Ideology of Political Representatives. American Political Science Review 78:163-178. http://dx.doi.org/10.2307/1961255
Centers, Richard. 1961. The Psychology of Social Classes. New York: Russell and Russell.
Cohen, Linda R., and Roger G. Noll. 1991. How to Vote, Whether to Vote: Strategies for Voting and Abstaining on Congressional Roll Calls. Political Behavior 13:97-127. http://dx.doi.org/10.1007/BF00992292
Dennis, Christopher D. 1988. The Revenue Side of Budgetary Politics: The Impact of Ideology on the Tax Reform Act of 1976. Presented at the annual meeting of the Southwestern Political Science Association Meeting, Dallas.
Desposato, Scott. 2001. Legislative Politics in Authoritarian Brazil. Legislative Studies Quarterly 26:287-320. http://dx.doi.org/10.2307/440205
Erikson, Robert S., Gerald C. Wright, and John P. McIver. 1993. Statehouse Democracy: Public Opinion and Policy in the American States. Cambridge: Cambridge University Press.
Farris, Charles D. 1958. A Method for Determining Ideological Groupings in Congress. Journal of Politics 20:308-338. http://dx.doi.org/10.2307/2127042
Fowler, Linda L. 1982. How Interest Groups Select Issues for Rating Voting Records of Members of the U.S. Congress. Legislative Studies Quarterly 7:401-414. http://dx.doi.org/10.2307/439365
Franklin, Charles H. 1984. Issue Preferences, Socialization, and the Evolution of Party Identification. American Journal of Political Science 28:459-478. http://dx.doi.org/10.2307/2110900
Franklin, Charles H. 1989. Estimation Across Data Sets: Two Stage Auxiliary Instrumental Variables Estimation (2SAIV). Political Analysis 1:23-40. http://dx.doi.org/10.1093/pan/1.1.1-a
Gage, N.L., and Ben Shimberg. 1949. Measuring Senatorial 'Progressivism'. Journal of Abnormal and Social Psychology 44:112-117. http://dx.doi.org/10.1037/h0055470
Groseclose, Tim, Steven D. Levitt, and James M. Snyder Jr. 1999. Comparing Interest Group Scores across Time and Chambers: Adjusted ADA Scores for the U.S. Congress. American Political Science Review 93:33-50. http://dx.doi.org/10.2307/2585759
Hall, Richard, and Bernard Grofman. 1990. The Committee Assignment Process and the Conditional Nature of Committee Bias. American Political Science Review 84: 1149-1166. http://dx.doi.org/10.2307/1963257
Herron, Michael C. 1999. Artificial Extremism in Interest Group Ratings and the Preferences versus Party Debate. Legislative Studies Quarterly 24:525-542. http://dx.doi.org/10.2307/440260
Herzon, F. 1980. Ideology, Constraint, and Public Opinion: The Case of Lawyers. American Journal of Political Science 24:233-258. http://dx.doi.org/10.2307/2110864
Hill, Kim Quaile. 2001. Multiple-Method Measurement of Legislatorsí Ideologies. Legislative Studies Quarterly 26:263-274. http://dx.doi.org/10.2307/440203
Hill, Kim Quaile, Stephen Hannah, and Sahar Shafquat. 1997. The Liberal-Conservative Ideology of U.S. Senators: A New Measure. American Journal of Political Science 41:1395-1413. http://dx.doi.org/10.2307/2960495
Hunt, A. Lee Jr., and Robert E. Pendley. 1972. Community Gatekeepers: An Examination of Political Recruiters. Midwest Journal of Political Science 16:411-438. http://dx.doi.org/10.2307/2110091
Hutchings, Vincent L. 1998. Issue Salience and Support for Civil Rights Legislation Among Southern Democrats. Legislative Studies Quarterly 23:521-544. http://dx.doi.org/10.2307/440238
Hyman, Herbert H. 1969. Political Socialization: A Study in the Psychology of Political Behavior, 2d ed. New York: The Free Press.
Jacobson, Gary C. 2003. Terror, Terrain and Turnout: Explaining the 2002 Midterm Elections. Political Science Quarterly 118:1-22. http://dx.doi.org/10.1002/j.1538-165X.2003.tb00384.x
Jackson, John E., and David C. King. 1989. Public Goods, Private Interests and Representation. American Political Science Review 83:1143-1164. http://dx.doi.org/10.2307/1961662
Jackson, John E., and John W. Kingdon. 1992. Ideology, Interest Group Scores and Legislative Votes. American Journal of Political Science 36:805-823. http://dx.doi.org/10.2307/2111592
Jennings, M.K., and R.G. Niemi. 1968. The Transmission of Political Values form Parent to Child. American Political Science Review 62:169-184. http://dx.doi.org/10.1017/S0003055400115709 http://dx.doi.org/10.2307/1953332
Jennings, M. Kent, and Laura Stoker. 1999. The Persistence of the Past: The Class of 1965 Turns Fifty. Presented at the annual meeting of the American Political Science Association, Atlanta, GA.
Kalt, Joseph P., and Mark A. Zupan. 1984. Capture and Ideology in the Economic Theory of Politics. American Economic Review 74:279-300.
Kau, James B., and Paul H. Rubin. 1979. Self Interest, Ideology, and Logrolling in Congressional Voting. Journal of Law and Economics 22:365-384. http://dx.doi.org/10.1086/466947
Kau, James B., and Paul H. Rubin. 1993. Ideology, Voting and Shirking. Public Choice 76:151-172. http://dx.doi.org/10.1007/BF01049347
Kazee, Thomas A., and Mary C. Thornberry. 1990. Where's the Party? Congressional Candidate Recruitment and American Party Organizations. Western Political Quarterly 43:61-80. http://dx.doi.org/10.1177/106591299004300105 http://dx.doi.org/10.2307/448505
Key, V.O. 1963. Public Opinion and American Democracy. New York: Knopf.
Krehbiel, Keith. 1986. A Technique for Estimating Congressmen's Ideal Points. Journal of Politics 48:97-115. http://dx.doi.org/10.2307/2130927
Krehbiel, Keith. 1990. Are Congressional Committees Composed of Preference Outliers? American Political Science Review 84:149-163. http://dx.doi.org/10.2307/1963634
Lazaerfeld, Paul F., Bernard Berelson, and Hazel Gaudet. 1948. The Peoples Choice, 2d ed. New York: Columbia University Press.
Lebo, Matthew, and Adam McGlynn. 2004. The Dynamic of Party Unity: Party Competition Models of Congressional Voting. Presented at the annual meeting of the Midwestern Political Science Association.
Levitt, Steven D. 1996. How Do Senators Vote? Disentangling the Role of Voter Preferences, Party Affiliation and Senator Ideology. American Economic Review 86:425-441.
Londregan, John. 2000. Estimating Legislatorsí Preferred Points. Political Analysis 8:35-56. http://dx.doi.org/10.1093/oxfordjournals.pan.a029804
MacRae, Duncan. 1958. Dimensions of Congressional Voting: A Statistical study of the House of Representatives in the 81st Congress. Berkeley: University of California Press.
Merelman, R. 1969. The Development of Political Ideology: A Framework for the Analysis of Political Socialization. American Political Science Review 63:750-767. http://dx.doi.org/10.2307/1954426 http://dx.doi.org/10.1017/S0003055400258565
Miller, Warren E., Donald R. Kinder, Steven J. Rosenstone, and the National Election Studies. 1988, 1990, 1992. American National Election Study: Pooled Senate Election Study [Computer File]. 2nd release. Ann Arbor, MI: University of Michigan, Center for Political Studies [producer], 1993. Ann Arbor, MI: Inter-University Consortium for Political and Social Research [distributor], 1993.
Miller, Warren, and Donald E. Stokes. 1963. Constituency Influence in Congress. American Political Science Review 57:45-56. http://dx.doi.org/10.2307/1952717
Nie, Norman N., Sidney Verba, and John R. Petrocik. 1979. The Changing American Voter. Chicago: University of Chicago Press. http://dx.doi.org/10.4159/harvard.9780674429147
Page, Benjamin I. and Robert Y. Shapiro. 1992. The Rational Public: Fifty Years of Trends on Americans' Policy Preferences. Chicago: University of Chicago Press. http://dx.doi.org/10.7208/chicago/9780226644806.001.0001
Peltzman, Sam. 1984. Constituent Interest and Congressional Voting. Journal of Law and Economics 27:181-210. http://dx.doi.org/10.1086/467062
Petrocik, John R. 1991. An Algorithm for Estimating Turnout as a Guide to Predicting Elections. Public Opinion Quarterly 55:643-647. http://dx.doi.org/10.1086/269286
Poole, Keith T., and Howard Rosenthal. 1985. A Spatial Model for Legislative Roll Call Analysis. American Journal of Political Science 29:357-385. http://dx.doi.org/10.2307/2111172
Poole, Keith T., and Howard Rosenthal. 1997. Congress: A Political History of Roll Call Voting. New York: Oxford University Press.
Reeher, Grant. 1996. Narratives of Justice: Legislatorís Beliefs about Distributional Fairness. Ann Arbor: University of Michigan Press. http://dx.doi.org/10.3998/mpub.10312
Rice, Stewart. 1924. Farmers and Workers in American Politics. In Studies in History, Economics and Public Law, eds. Faculty of Political Science Department of Columbia University. New York: Columbia University.
Rosenstone, Steven J., and John. M. Hansen. 1993. Mobilization, Participation and Democracy in America. New York: Macmillian.
Seelye, Cathrine Q. 2004. http://www.nytimes.com/2004/04/12/politics/campaign/12KERR.html.
Segal, Jeffery A., and Albert D. Cover. 1989. Ideological Values and the Votes of U.S. Supreme Court Justices. American Political Science Review 83:557-585. http://dx.doi.org/10.2307/1962405
Seidman, David. 1975. Simulation of Public Opinion: A Caveat. Public Opinion Quarterly 39: 331-342. http://dx.doi.org/10.1086/268232
Shaffer, William R. 1989. Rating the Performance of the ADA in Congress. Western Political Quarterly 42:33-51. http://dx.doi.org/10.2307/448654 http://dx.doi.org/10.1177/106591298904200104
Sherif, Muzafer. 1935. A Study of Some Social Factors in Perception. Archives of Psychology 27.
Sherif, Muzafer, and Hadley Cantril. 1947. The Psychology of Ego-Involvements: Social Attitudes and Identifications. New York: Wiley and Sons. http://dx.doi.org/10.1037/10840-000
Smith, Eric R.A.N., Richard Herrera, and Cheryl Herrera. 1991. The Measurement Characteristics of Congressional Roll Call Indexes. Legislative Studies Quarterly 25: 283-295.
Snyder, James. 1992. Artificial Extremism in Interest Group Ratings. Legislative Studies Quarterly 25:319-345. http://dx.doi.org/10.2307/439733
Uslaner, Eric M. 1999. The Movers and the Shirkers. Ann Arbor: University of Michigan Press.
Downloads
Published
Issue
Section
License
Authors who publish with American Review of Politics agree to the following terms:
The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
Attribution: other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
Non-Commercial: the materials may not be used for commercial purposes;
Share Alike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
The Author represents and warrants that:
the Work is the Author’s original work;
the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
the Work is not pending review or under consideration by another publisher;
the Work has not previously been published;
the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
the Work contains no libel, invasion of privacy, or other unlawful matter.
The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.
