社交媒体倾听平台的人工智能可信？检验Crimson Hexagon(现Brandwatch消费者研究公司)人工智能驱动分析的准确性（摘要）

从业者和学者越来越多地采用人工智能驱动的社交媒体倾听平台（SMLPs），从大量的社交媒体数据中提取可操作的见解，为研究问题和品牌战略提供信息。由于其专有性，SMLPs内的AI工具是 "黑匣子"，迫使用户盲目地接受结果，这引起了业界和学术界的关注。本研究试图通过评估领先的SMLP Crimson Hexagon（现Brandwatch Consumer Research）基于AI的分析结果与标准的人类内容分析及使用LIWC进行的分析的结果进行对比，从而更好地了解SMLPs的优势和弱点。选择以Colin Kaepernick为主角的耐克 "Dream Crazy"广告为例，通过随机选择关于该广告的10000条帖子作为样本进行分析，发现Crimson Hexagon的人工智能工具在品牌识别以及对帖子和品牌情感极性、特定情感和品牌结果的检测方面都非常不可靠，这表明了盲目依赖通过黑盒社交媒体倾听平台得出的结论的危害。研究结果强调了研究人员需要检查算法文档和训练数据集，以及在研究模型和决策中使用这些结论之前，需要对AI生成的数据进行评估。

Practitioners and scholars increasingly employ social media listening platforms (SMLPs) driven by artificial intelligence (AI) to extract actionable insights from large amounts of social media data informing research questions and brand strategy. Due to their proprietary nature, AI tools within SMLPs are “black boxes” that force users to accept results on blind faith, a source of concern in industry and academia. This study seeks to provide greater understanding of the strengths and weaknesses of SMLPs by assessing the AI-based results of leading SMLP Crimson Hexagon (now Brandwatch Consumer Research) against those of a standard human content analysis and an analysis conducted using Linguistic Inquiry and Word Count (LIWC). Analyzing a random 10,000-post sample of the conversation around the Nike “Dream Crazy” ad featuring Colin Kaepernick, findings reveal Crimson Hexagon’s AI tools to be woefully unreliable in terms of brand identification as well as detection of post and brand sentiment polarity, specific emotions, and brand outcomes, demonstrating the hazards of blindly relying upon conclusions drawn from black-box social media listening platforms. Findings highlight the need for researchers to examine algorithm documentation and training data sets, as well as assess AI-generated data prior to use in research models and decisions.

论文原文：Jameson L. Hayes, Brian C. Britt, William Evans, Stephen W. Rush, Nathan A. Towery & Alyssa C. Adamson. (2021). Can Social Media Listening Platforms’ Artificial Intelligence Be Trusted? Examining the Accuracy of Crimson Hexagon’s (Now Brandwatch Consumer Research’s) AI-Driven Analyses. Journal of Advertising, 50(01), 81-91.