Artificial Intelligence Tools and Bias in Journalism-related Content Generation: Comparison Between Chat GPT-3.5, GPT-4 and Bing

  1. Castillo-Campos, Mar 1
  2. Varona-Aramburu, David 2
  3. Becerra-Alonso, David 1
  1. 1 Universidad Loyola Andalucía
    info

    Universidad Loyola Andalucía

    Sevilla, España

    ROR https://ror.org/0075gfd51

  2. 2 Universidad Complutense de Madrid
    info

    Universidad Complutense de Madrid

    Madrid, España

    ROR 02p0gd045

Journal:
Tripodos

ISSN: 2340-5007,  1138-3305

Year of publication: 2024

Pages: 06

Type: Article

DOI: 10.51698/TRIPODOS.2024.55.06 GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Tripodos

Abstract

This study explores the biases present in artificial intelligence (AI) tools, focusing on GPT-3.5, GPT-4, and Bing. The performance of the tools has been compared with a group of experts in linguistics, and journalists specialized in breaking news and international affairs. It reveals that GPT-3.5, widely accessible and free, exhibits a higher tendency rate in its word generation, suggesting an intrinsic bias within the tool itself rather than in the input data. Comparatively, GPT-4 and Bing demonstrate differing patterns in term generation and subjectivity, with GPT-4 aligning more closely with expert opinions and producing fewer opinative words. The research highlights the extensive use of generative AI in media and among the general populace, emphasizing the need for careful reliance on AI-generated content. The findings stress the risks of misinformation and biased reporting inherent in unexamined AI outputs. The challenge for journalists and information professionals is to ensure accuracy and ethical judgment in content creation to maintain the quality and diversity of content in journalistic practices.

Bibliographic References

  • Abbott, Andrew (2010). Chaos of disciplines. University of Chicago Press.
  • Bailer, Werner; Thallinger, Georg; Krawarik, Verena; Schell, Katharina, and Ertelthalner, Victoria (2022). AI for the media industry: Application potential and automation levels. In International Conference on Multimedia Modeling (pp. 109-118). Springer International Publishing. https://doi.org/10.1007/978-3-030-98358-1_9
  • Barrio, David Alonso del and Gatica-Pérez, Daniel (2023). Framing the news: From human perception to large language model inferences. arXiv preprint arXiv:2304.14456.
  • Bosque, Ignacio (2004). Redes: Diccionario combinatorio del español contemporáneo. Las pala­bras en su contexto. Editorial SM.
  • Brennen, J. Scott and Nielsen, Rasmus Klein (2018). An industry-led debate: How UK media cover artificial intelligence. Reuters Institute for the Study of Journalism. https://doi.org/10.60625/risj-v219-d676
  • Carabantes, David; González-Geraldo, José L., and Jover, Gonzalo (2023). ChatGPT could be the reviewer of your next scientific paper: Evidence on the limits of AI-assisted academic reviews. Profesional de la Información, 32(5).
  • Clerwall, Christer (2017). Enter the robot journalist: Users’ perceptions of automated content. In The Future of Journalism: In an Age of Digital Media and Economic Uncertainty (pp. 165-177). Routledge.
  • Dale, Robert (2021). GPT-3: What’s it good for? Natural Language Engineering, 27(1), 113-118. https://doi.org/10.1017/S135132492
  • Dalen, Arjen van (2012). The algorithms behind the headlines: How machine-written news redefines the core skills of human journalists. Journalism Practice, 6(5-6), 648-658. https://doi.org/10.1080/17512786.2012.667268
  • Dhiman, Bharat (2023). Does artificial intelligence help journalists: A boon or bane? https://dx.doi.org/10.2139/ssrn.4401194
  • Donk, André; Metag, Julia; Kohring, Matthias, and Marcinkowski, Frank (2012). Framing emerging technologies: Risk perceptions of nanotechnology in the German press. Science Communication, 34(1), 5-29. https://doi.org/10.1177/1075547011417892
  • Dwivedi, Yogesh K.; Kshetri, Nir; Hughes, Laurie; Slade, Emma L.; Jeyaraj, Anand; Kar, Arpan K.; Baabdullah, Abdullah M.; Koohang, Alex; Raghavan, Vishnupriya; Ahuja, Manju; Albanna, Hanaa; Albash­rawi, Mousa A.; Al-Busaidi, Adil S.; Balakrishnan, Janarthanan; Barlette, Yves; Basu, Sriparna; Bose, Indranil; Brooks, Laurence; Buhalis, Dimitrios,... Wright, Ryan (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. Interna­tional Journal of Information Management, 71, 102642.
  • Gonçalves, Adriana and Melo, Paulo Victor (2022). Inteligencia artificial y periodismo: Una aproximación al contexto portugués. Fonseca, Journal of Communication, (25), 23-34. https://doi.org/10.14201/fjc.29682
  • Goyal, Tanya; Li, Junyi Jessy, and Durrett, Greg (2022). News summarization and evaluation in the era of GPT-3. arXiv preprint arXiv:2209.12356.
  • Grail, Quentin; Pérez, Julien, and Gaussier, Eric (2021). Globalizing BERT-based transformer architectures for long document summarization. In Proceedings of the 16th conference of the European chapter of the Association for Computational Linguistics: Main volume (pp. 1792-1810). http://dx.doi.org/10.18653/v1/2021.eacl-main.154
  • Gupta, Anushka; Chugh, Dishka; Anjum, and Katarya, Rahul (2022). Automated news summarization using transformers. In Sustainable advanced computing (pp. 249-259). Springer. https://doi.org/10.48550/arXiv.2108.01064
  • Gutiérrez-Caneda, Beatriz; Vázquez-Herrero, Jorge, and López-García, Xosé (2023). AI application in journalism: ChatGPT and the uses and risks of an emergent technology. Profesional de la Información, 32(5).
  • Hassan, Abdulsadek and Albayari, Akram (2022). The usage of artificial intelligence in journalism. In Future of organizations and work after the 4th industrial revolution: The role of artificial intelligence, big data, automation, and robotics (pp. 175-197). Springer. http://dx.doi.org/10.1007/978-3-030-99000-8_10
  • Hurlburt, George (2023). What if ethics got in the way of generative AI? IT Professional, 25(2), 4-6. https://doi.ieeecomputersociety.org/10.1109/MITP.2023.3267140
  • Kherwa, Pooja and Bansal, Poonam (2019). Topic modeling: A comprehensive review. EAI Endorsed Transactions on Scalable Information Systems, 7(24). https://doi.org/10.4108/eai.13-7-2018.159623
  • Knight, Megan and Cook, Clare (2013). Social media for journalists: Principles and practice. Sage.
  • Kohring, Matthias and Matthes, Jörg (2002). The face(t)s of biotech in the nineties: How the German press framed modern biotech­nology. Public Understanding of Science, 11(2), 143. https://doi.org/10.1088/0963-6625/11/2/304
  • Leippold, Markus (2023). Sentiment spin: Attacking financial sentiment with GPT-3. Finance Research Letters. https://doi.org/10.1016/j.frl.2023.103957
  • Liu, Sengjie and Healey, Christopher G. (2023). Abstractive summarization of large
  • document collections using GPT. arXiv preprint arXiv:2310.05690.
  • Llorca-Abad, Germán and López-García, Guillermo (2020). Communication flows in the European elections: Amid populism and Euroscepticism. Tripodos, (49), 9-12.
  • Nerdynav (23 November 2023). 107 up-to-date CHATGPT statistics & user numbers [Nov 2023]. Nerdynav. https://nerdynav.com/chatgpt-statistics/
  • Niederman, Fred and Baker, Elizabeth White (2023). Ethics and AI issues: Old container with new wine? Information Systems Frontiers, 25(1), 9-28.
  • Noain-Sánchez, Amaya (2022). Addressing the impact of artificial intelligence on journalism: The perception of experts, journalists, and academics. Communication & Society, 35(3), 105-121. https://doi.org/10.15581/003.35.3.105-121
  • Rai, Nishant; Kumar, Deepika; Kaushik, Naman; Raj, Chandan, and Ali, Ahad (2022). Fake news classification using transformer based enhanced LSTM and BERT. International Journal of Cognitive Computing in Engineering, 3, 98-105.
  • Rathje, Steve; Mirea, Dan-Mircea; Sucholutsky, Ilia; Marjieh, Raja; Robertson, Claire, and Bavel, Jay J. van (2023). GPT is an effective tool for multilingual psychological text analysis. https://doi.org/10.31234/osf.io/sekf5
  • Rodríguez de Luis, Eva (2023). Chatgpt vuelve a Italia: Qué medidas ha implementado OpenAi para levantar el veto. Genbeta. https://www.genbeta.com/actualidad/chatgpt-vuelve-a-italia-que-medidas-ha-implementado-openai-para-levantar-veto
  • Scheufele, Dietram A. (1999). Framing as a theory of media effects. Journal of Communication, 49(1), 103-122. https://doi.org/10.1111/j.1460-2466.1999.tb02784.x
  • Schütz, Mina; Schindler, Alexander; Siegel, Melanie, and Nazemi, Kawa (2021). Automatic fake news detection with pre-trained transformer models. In Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10-15, 2021, Proceedings, Part VII (pp. 627-641). Springer International Publishing. https://doi.org/10.1007/978-3-030-68787-8_45
  • Stray, Jonathan (2019). Making artificial intelligence work for investigative journalism. In Algorithms, Automation, and News (pp. 97-118). https://doi.org/10.1080/21670811.2019.1630289
  • Taboada, Maite (2016). Sentiment analysis: An overview from linguistics. Annual Review of Linguistics, 2, 325-347. https://doi.org/10.1146/annurev-linguistics-011415-040518
  • Tejedor, Santiago and Vila, Pere (2021). Exo journalism: A conceptual approach to a hybrid formula between journalism and artificial intelligence. Journalism and Media, 2(4), 830-840. https://doi.org/10.3390/journalmedia2040048
  • Türksoy, Nilüfer (2022). The future of public relations, advertising, and journalism: How artificial intelligence may transform the communication profession and why society should care? Türkiye İletişim Araştırmaları Dergisi, (40), 394-410. https://doi.org/10.17829/turcom.1050491
  • Verma, Pranshu (5 November 2023). AI fake nudes are booming. It’s ruining real teens’ lives. The Washington Post. https://www.washingtonpost.com/technology/2023/11/05/ai-deepfake-porn-teens-women-impact/
  • Vincent, James (29 March 2023). Elon Musk and top AI researchers call for pause on “giant AI experiments”. The Verge. https://www.theverge.com/2023/3/29/23661374/elon-musk-ai-researchers-pause-research-open-letter
  • Whittaker, Jason Paul (2019). Tech giants, artificial intelligence, and the future of journalism. Taylor & Francis. http://library.oapen.org/handle/20.500.12657/25879
  • Wodecki, Ben (3 February 2023). UBS: ChatGPT may be the fastest growing app of all time. AI Business. https://aibusiness.com/nlp/ubs-chatgpt-is-the-fastest-growing-app-of-all-time
  • Zhai, Xiaoming (2023). ChatGPT for next generation science learning. XRDS: Crossroads, The ACM Magazine for Students, 29(3), 42-46. https://doi.org/10.1145/
  • Zohny, Hazem; McMillan, John, and King, Mike (2023). Ethics of generative AI. Journal of Medical Ethics, 49(2), 79-80. http://dx.doi.org/10.1136/jme-2023-108909