CheckThat!: Lab on Subjectivity, Fact-Checking, Claim Extraction & Normalization, and Retrieval

The eighth edition of the CheckThat! lab at CLEF presents a diverse set of challenges aimed at advancing technology to support and enhance the journalistic verification process. This edition revisits core tasks in the verification pipeline while also introducing auxiliary tasks such as subjectivity identification, claim normalization, and fact-checking numerical claims, with a particular emphasis on scientific web discourse. These tasks pose complex classification and retrieval problems at both the document level, including in multilingual contexts.

Tasks:

  • Task 1 - Subjectivity

    Given a sentence from a news article, determine whether it is subjective or objective. This is a binary classification task and is offered in Arabic, English, Bulgarian, German, and Italian for mono- and multi-lingual settings. Additionally, unseen languages like French and Spanish are considered for zero-shot settings.

  • Task 2 - Claim Normalization

    Given a noisy, unstructured social media post, the task is to simplify it into a concise form. This is a generation task, offered in 20 languages: English, Arabic, Bengali, Czech, German, Greek, French, Hindi, Korean, Marathi, Indonesian, Dutch, Punjabi, Polish, Portuguese, Romanian, Spanish, Tamil, Telugu, Thai.

  • Task 3 - Fact-Checking Numerical Claims

    This task focuses on verifying claims with numerical quantities and temporal expressions. Numerical claims are defined as those requiring validation of explicit or implicit quantitative or temporal details. Participants must classify each claim as True, False, or Conflicting based on a short list of evidence.

  • Task 4 - Scientific Web Discourse Processing (SciWeb)
    • Subtask 4.1: SciWeb Discourse Detection

      This task aims at classifying the different forms of science-related online discourse. Namely, given a tweet, this multilabel task aims at detecting if a tweet contains a scientific claim or scientific reference or is referring to science contexts or entities.

    • Subtask 4.2 - SciWeb Claim-Source Retrieval

      Given a tweet containing a scientific claim and an informal reference to a scientific paper, this task aims at retrieving the scientific paper that serves as the source for the claim from a given pool of candidate scientific papers.

Organizers

  • Firoj Alam, Qatar Computing Research Institute, HBKU
  • Julia Maria Struß, University of Applied Sciences Potsdam
  • Federico Ruggeri, Università di Bologna
  • Preslav Nakov, MBZUAI
  • Tanmoy Chakraborty, IIT Delhi
  • Vinay Setty, University of Stavanger
  • Stefan Dietze, GESIS - Leibniz Institute for the Social Sciences & HHU Düsseldorf

Contact

  • fialam@hbku.edu.qa
  • julia.struss@fh-potsdam.de
  • tanchak@iitd.ac.in