Assessing Criterion Validity of Using Internet Searches as a Measure of Public Attention

Authors

  • Elizabeth A. Stiles John Carroll University
  • Patrick E. Grogan-Myers

DOI:

https://doi.org/10.15763/issn.2374-779X.2018.36.2.1-18

Abstract

We examine the criterion validity of using internet searches as a measure of public attention to United States Supreme Court (USSC) cases.  First, we construct a measure of public attention to three cases by comparing relevant search terms in Google Trends to one top search terms of the year, then sum the measure week by week during the period of the research design.  To test the measure’s criterion validity, we replicate Scott and Saunders’ (2006) models using their dataset (created by conducting phone interviews of a national sample using random digit dialing) that was designed to assess awareness of USSC decisions. We find that public attention as measured by Google Trends data is predictive of public awareness of USSC decisions for two of their three models. We conclude that using free, publically available big data to measure public attention to USSC cases has criterion validity, and is a valuable tool for researchers studying public policy and process. 

References

Barrett, L. C. (2006, June 2). Jim Lehrer on Billy Bob, Reports of Rain and Stenography As Journalism. Retrieved June 9, 2016, from http://www.cjr.org/behind_the_news/jim_lehrer_on_billy_bob_report.php

Bartels, Larry M. 1996. Uninformed Votes: Information Effects in Presidential Elections. American Journal of Political Science, Vol. 40, No. 1. (Feb., 1996), pp. 194-230. https://doi.org/10.2307/2111700

Baumgartner, Frank R. and Bryan D. Jones. 1993. Agendas and Instability in American Politics. Chicago: University of Chicago Press.

Biggers, D. R. (2012). Can a Social Issue Proposition Increase Political Knowledge? Campaign Learning and the Educative Effects of Direct Democracy. American Politics Research, 40(6), 998-1025. https://doi.org/10.1177/1532673X11427073

Campbell, Andrea. 2012. "Policy Makes Mass Politics," Annual Review of Political Science 15: 333-51. https://doi.org/10.1146/annurev-polisci-012610-135202

Campbell, Angus, Phillip E. Converse, Warren E. Miller and Donald E. Stokes. The American Voter. New York: John Wiley & Sons, Inc.

Carneiro HA, Mylonakis E. (2009), Google Trends: A Web-Based Tool for Real-Time Surveillance of Disease Outbreak. Clinical Infectious Diseases, 49:1557–1564. https://doi.org/10.1086/630200

Choi, H. and Varian, H. (2012), Predicting the present with Google Trends. Economic Record, 88: 2–9. https://doi.org/10.1111/j.1475-4932.2012.00809.x

Chung-pin Lee, Don-yun Chen and Tong-yi Huang. 2013. The Interplay Between Digital and Political Divides: The Case of e-Petitioning in Taiwan. Social Science Computer Review 2014, Vol. 32(1) 37-55. https://doi.org/10.1177/0894439313497470

Converse, Philip E. 1964. "The Nature of Belief Systems in Mass Publics." In David E. Apter, ed., Ideology and Discontent. Glencoe, IL: Free Press. Pages

Dahl, Robert. 1957. "Decision Making in a Democracy: The Supreme Court as a National Policy Maker." Journal of Public Law 6:279-95.

Dancy, Logan and Geoffrey Sheagley. 2012. "Heuristics Behaving Badly: Party Cues and Voter Knowledge." Journal? Pages? Delli, Carpini, Michael and Scott Keeter. 1996. What Americans Know About Politics and Why It Matters. Yale University Press. New Haven.

Downs, Anthony. 1957. An Economic Theory of Democracy. New York: Harper.

Eaves, Ronald C. and Suzanne Woods-Groves. 2007. Criterion Validity in Encyclopedia of Measurement and Statistics, edited by Salkind, Neil J. and Kristen Rasmussen. Sage. Pages

Ellis, Ripberger, and Swearingen. 2017. "Public Attention and Head-to-Head Campaign Fundraising: An Examination of U.S. Senate Elections." American Review of Politics, Vol. 36, No. 1, pp. 30-53. https://doi.org/10.15763/issn.2374-779X.2017.36.1.30-53

Fiske, Susan and Shelley Taylor. 2008. Social Cognition from Brains to Culture. New York: McGraw-Hill.

Fowler, Anthony and Michele Margolis. 2014. "The Political Consequences of Uninformed Voters." Electoral Studies 34(100-110). https://doi.org/10.1016/j.electstud.2013.09.009

Franklin, Charles H. and Liane C. Kosaki. 1989. Republican Schoolmaster: The US Supreme Court, Public Opinion and Abortion. American Political Science Review. Volume 83, Issue 3, pp. 751-771. https://doi.org/10.2307/1962059

Gibson, James L. and Gregory A. Caldeira. 2009. Knowing the Court? A Reconsideration of Public Ignorance of the High Court. The Journal of Politics, Volume 71, Issue 02. April 2009, pp 429-441. https://doi.org/10.1017/S0022381609090379

Gibson, James L., Gregory A. Caldeira, and Lester Kenyatta Spence. 2003. "Measuring Attitudes toward the United States Supreme Court." American Journal of Political Science 47:354-67. https://doi.org/10.1111/1540-5907.00025

Goidel, Robert K. (Ed.) 2011. Political Polling in The Digital Age: The Challenge of Measuring and Understanding Public Opinion. Baton Rouge, LA: Louisiana State University Press.

Google. (n.d.). Google Press Center: Zeitgeist. (2005). Retrieved November 12, 2014, from http://www.google.com/press/zeitgeist2005.html

Hanley, John, Michael Salamone and Matthew Wright. Reviving the Schoolmaster: Reevaluating Public Opinion in the Wake of Roe V. Wade. Political Research Quarterly June 2012 vol. 65 no. 2 408-421 https://doi.org/10.1177/1065912911404564

Hoekstra, Valerie J. 1995. The Supreme Court and Opinion Change: An Experimental Study of the Court's Ability to Change Opinion. American Politics Research. January 1995 vol. 23 no. 1, 109-129 https://doi.org/10.1177/1532673X9502300106

Hoekstra, Valerie J. Public Reaction to Supreme Court Decisions. 2003. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511509827

Leeper and Bolson. 2013. Self-Interest and Attention to News among Issue Publics. Political Communication Volume 30, issue 3, pp. 329-348.

Lowi, Ted. 1964. "American Business, Public Policy, Case Studies, and Political Theory." World Politics 61(4):677–715 https://doi.org/10.2307/2009452

McCarthy, John D. and Mayer N. Zald. 1973. American Journal of Sociology, Vol. 82 No. 6, pp. 1212-1241. Stable URL: ` https://doi.org/10.1086/226464

Mondak, Jeffery J. 1992. "Institutional Legitimacy, Policy Legitimacy, and the Supreme Court." American Politics Research 20:457-77. https://doi.org/10.1177/1532673X9202000406

Morris, D.S. and Morris J.S. Digital Inequality and Participation in the Political Process: Real or Imagined? Social Science Computer Review October 2013 vol. 31 no. 5 589-600 https://doi.org/10.1177/0894439313489259

Neumann, W. Russell. 1986. The Paradox of Mass Politics: Knowledge and Opinion in the American Electorate. Cambridge: Harvard University Press.

Newig, J. 2004. Public Attention Political Action: The Example of Environmental Regulation. Rationality and Society, 16(2), 149-190. doi:10.1177/1043463104043713. https://doi.org/10.1177/1043463104043713

Olson, Mancur. 1965. The Logic of Collective Action: Public Goods and the Theory of Groups. Cambridge: Harvard University Press.

Page, Benjamin I., and Robert Y. Shapiro. 1992. The Rational Public: Fifty Years of Trends in Americans' Policy Preferences. Chicago: University of Chicago Press. https://doi.org/10.7208/chicago/9780226644806.001.0001

Popkin, Samuel L. 1991. The Reasoning Voter, Communication and Persuasion in Presidential Campaigns. Chicago: University of Chicago Press.

Popkin Samuel L. and Michael A. Dimock. 1999. "Political Knowledge and Citizen Competence" in Citizen Competence and Democratic Institutions, Stephen L. Elkin and Karol Soltan, eds. State Park PA: Pennsylvania State University Press.

Prior M. 2007. Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections. New York: Cambridge Univ. Press https://doi.org/10.1017/CBO9781139878425

Reuning, Kevin, and Nick Dietrich. 2015. "TheEffect of the Media on the Invisible Primary." http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2709208.

Richey, Sean. 2013. Random and Systematic Error in Voting in Presidential Elections. Political Research Quarterly 66(3): 645-657. https://doi.org/10.1177/1065912912459565

Ripberger, Joseph T. 2011. Capturing Curiosity: Using Internet Search Trends to Measure Public Attentiveness. Policy Studies Journal, Vol. 39, No. 2, pp. 239-259. https://doi.org/10.1111/j.1541-0072.2011.00406.x

Schattschneider, EE. 1935. Politics, Pressure and the Tariff. New York: Prentice Hall

Scott, K. and Saunders, K. L. , 2006-04-20 "Courting Public Opinion: Supreme Court Impact on Public Opinion Reconsidered" Paper presented at the annual meeting of the The Midwest Political Science Association, Palmer House Hilton, Chicago, Illinois PDF accessed 03/31/2013: http://citation.allacademic.com/meta/p139252_index.html

Singh, Shane P. and Jason Roy. 2014. Political knowledge, the decision calculus and proximity voting. Electoral Studies, volume 34(89-99). https://doi.org/10.1016/j.electstud.2013.11.007

Skocpol, Theda. 1992. Protecting Soldiers and Mothers: The Political Origins of Social Policy in the United States. Cambridge, MA: Harvard University Press

Soroka, Stuart N. 2002. "Issue Attributes and Agenda-Setting by Media, the Public, and Policymakers in Canada." International Journal of Public Opinion Research 14: 264–85. https://doi.org/10.1093/ijpor/14.3.264

Stiles, Elizabeth and Patrick Grogan. 2014. The Dynamics of Public Attention to Policy Decisions by Various Means: Same-Sex Marriage and the Courts, the Legislatures, and Initiatives (2004-2014). Paper presented at the Annual Meeting of the Ohio Association of Economists and Political Scientists, September 2014.

Swearingen C. Douglas and and Joseph T. Ripberger. 2014. Google Insights and U.S. Senate Elections: Does Search Traffic Provide A Valid Measure of Public Attention to Political Candidates? Social Science Quarterly, copyright 2014 by the Southwestern Social Science Association DOI: 10.1111/ssqu.12075. https://doi.org/10.1111/ssqu.12075

Ura, Joseph Daniel. 2014. Backlash and Legitimation: Macro Political Responses to Supreme Court Decisions American Journal of Political Science 58(1): 110-126.

Downloads

Published

2018-06-03

Issue

Vol. 36 No. 2 (2018)

Section

Articles

License

Authors who publish with American Review of Politics agree to the following terms:

  1. The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.

  2. Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.

  3. The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:

    1. Attribution: other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;

    2. Non-Commercial: the materials may not be used for commercial purposes;

    3. Share Alike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.

  1. The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.

  2. Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.

  3. Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.

  1. The Author represents and warrants that:

    1. the Work is the Author’s original work;

    2. the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;

    3. the Work is not pending review or under consideration by another publisher;

    4. the Work has not previously been published;

    5. the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and

    6. the Work contains no libel, invasion of privacy, or other unlawful matter.

  2. The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.