GenAI Content Detection

Workshop on Detecting AI Generated Content

Keynote

Title: Detectability of Language Model Generated Content: Myths, Challenges, and Opportunities

Abstract: Large language models (LLMs) have captured the imagination of researchers and users worldwide, and led to a capacity race. We start with some myths on the detectability of LLM generated content. We then consider LLMs through the alignment-utility tradeoff perspective and reveal some surprising consequences of aligning LLM models for different purposes, such as 1) ability to mimic humans on natural language tasks and 2) safety..Throughout the talk, we shall highlight the challenges and opportunities for research with respect to detecting LLM generated content for the different purposes of alignment.

Speaker: Rakesh Verma, Professor, University of Houston

Rakesh Verma



Bio: Rakesh Verma is a Professor of Computer Science at the University of Houston, where he teaches a course on security analytics. He has made research contributions in equational logic programming, algorithm design and analysis, computer security, and data science. He is the author of Cybersecurity Analytics (CRC Press, 2019) (Verma and Marchette, 2019); a cybersecurity section associate editor of the Frontiers in Big Data journal; and co-organizer of the ACM Annual International Workshop on Security and Privacy Analytics (IWSPA) since 2015.

Workshop Schedule

You can view the program plan at the following link: Program Spreadsheet. Also listed below.

Saturday, January 18, 2025

14:00 – 19:00 - Registration (ADNEC)

Sunday, January 19, 2025

07:30 – 16:30 - Registration (ADNEC); Workshop: Capital Suite 9, ADNEC

Opening

09:00 - 09:10 - Opening Remarks: Firoj + Preslav
09:10 - 09:50 - Keynote 1: Detectability of Language Model Generated Content: Myths, Challenges, and Opportunities
Speaker: Rakesh Verma, Professor, University of Houston

Session 1 (Chair: Firoj Alam)

09:50 - 10:35

  • 09:50 - 10:05 - GPT-4 is Judged More Human than Humans in Displaced and Inverted Turing Tests
    Ishika M. Rathi, Sydney Taylor, Benjamin Bergen, and Cameron Jones
  • 10:05 - 10:20 - SilverSpeak: Evading AI-Generated Text Detectors using Homoglyphs
    Aldan Creo and Shushanta Pudasaini
  • 10:20 - 10:35 - Mirror Minds: An Empirical Study on Detecting LLM-Generated Text via LLMs
    Josh Baradia, Shubham Gupta, and Suman Kundu

10:35 - 11:00 - Coffee Break

Session 2 (Chair: Artem Shelmanov)

11:00 - 12:00

  • 11:00 - 11:15 - Human vs. AI: A Novel Benchmark and a Comparative Study on the Detection of Generated Images and the Impact of Prompts
    Philipp Moeßner and Heike Adel
  • 11:15 - 11:30 - GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human
    Yuxia Wang, Artem Shelmanov, Jonibek Mansurov, Akim Tsvigun, Vladislav Mikhailov, Rui Xing, Zhuohan Xie, Jiahui Geng, Giovanni Puccetti, Ekaterina Artemova, Jinyan Su, Minh Ngoc Ta, Mervat Abassy, Kareem Ashraf Elozeiri, Saad El Dine Ahmed El Etter, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Nurkhan Laiyk, Osama Mohammed Afzal, Ryuto Koike, Masahiro Kaneko, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov
  • 11:30 - 11:45 - Cross-table Synthetic Tabular Data Detection
    G. Charbel N. Kindji, Lina M. Rojas Barahona, Elisa Fromont, and Tanguy Urvoy
  • 11:45 - 12:05 - Benchmarking AI Text Detection: Assessing Detectors Against New Datasets, Evasion Tactics, and Enhanced LLMs
    Shushanta Pudasaini, Luis Miralles, David Lillis, and Marisa Llorens Salvador

12:05 - 13:30 - Lunch Break

Session 3 (Chair: Georgios Mikros)

13:30 - 15:25

  • 13:30 - 13:50 - GenAI Content Detection Task 2: AI vs. Human – Academic Essay Authenticity Challenge
    Shammur Absar Chowdhury, Hind Almerekhi, Mucahid Kutlu, Kaan Efe Keleş, Fatema Ahmad, Tasnim Mohiuddin, George Mikros, and Firoj Alam
  • 13:50 - 14:10 - GenAI Content Detection Task 3: Cross-Domain Machine Generated Text Detection Challenge
    Liam Dugan, Andrew Zhu, Firoj Alam, Preslav Nakov, Marianna Apidianaki, and Chris Callison-Burch
  • 14:10 - 14:25 - DAMAGE: Detecting Adversarially Modified AI Generated Text
    Elyas Masrour, Bradley N. Emi, and Max Spero
  • 14:25 - 14:40 - Your Large Language Models are Leaving Fingerprints
    Hope Elizabeth McGovern, Rickard Stureborg, Yoshi Suhara, and Dimitris Alikaniotis
  • 14:40 - 14:55 - I Know You Did Not Write That! A Sampling Based Watermarking Method for Identifying Machine Generated Text
    Kaan Efe Keleş, Ömer Kaan Gürbüz, and Mucahid Kutlu
  • 14:55 - 15:10 - Text Graph Neural Networks for Detecting AI-Generated Content
    Andric Valdez and Helena Gomez-Adorno
  • 15:10 - 15:25 - The Consistent Lack of Variance of Psychological Factors Expressed by LLMs and Spambots
    Vasudha Varadarajan, Salvatore Giorgi, Siddharth Mangalik, Nikita Soni, Dave M. Markowitz, and H. Andrew Schwartz

15:30 - 16:30 - Break + Poster Session

Session 4 (Chair: Preslav Nakov)

16:30 - 17:30

  • 16:30 - 16:40 - SzegedAI at GenAI Detection Task 1: Beyond Binary - Soft-Voting Multi-Class Classification for Binary Machine-Generated Text Detection Across Diverse Language Models
    Mihaly Kiss and Gábor Berend
  • 16:40 - 16:50 - IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometry
    Mohammad ALSmadi
  • 16:50 - 17:00 - BBN-U.Oregon’s ALERT system at GenAI Content Detection Task 3: Robust Authorship Style Representations for Cross-Domain Machine-Generated Text Detection
    Hemanth Kandula, CHAK FAI LI, Haoling Qiu, Damianos Karakos, Hieu Man, Thien Huu Nguyen, and Brian Ulicny
  • 17:00 - 17:10 - Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking
    German Gritsai, Anastasia Voznuyk, Ildar Khabutdinov, and Andrey Grabovoy
  • 17:10 - 17:20 - CNLP-NITS-PP at GenAI Detection Task 3: Cross-Domain Machine-Generated Text Detection Using DistilBERT Techniques
    Sai Teja Lekkala, Annepaka Yadagiri, Mangadoddi Srikar Vardhan, and Partha Pakray
  • 17:20 - 17:30 - Leidos at GenAI Detection Task 3: A Weight-Balanced Transformer Approach for AI Generated Text Detection Across Domains
    Abishek R. Edikala, Gregorios A. Katsios, Noelie Creaghe, and Ning Yu

Closing Ceremony

17:30 - 17:45 - Best Paper Award + Closing Ceremony
Chairs: Firoj Alam and Preslav Nakov