Replication Data for: Beyond the Link: Assessing LLMs’ Ability to Classify Political Content Across Global Media

Version 2.1
DE LA FUENTE CUESTA, ALEJANDRO; Alberto Martínez Serra; Nienke Visscher; Cardenal, Ana S., 2025, "Replication Data for: Beyond the Link: Assessing LLMs’ Ability to Classify Political Content Across Global Media", https://doi.org/10.82201/8UPPY6, BSC Dataverse, V2, UNF:6:j4VEdzl/g0wp+AX0/Pik1w== [fileUNF]
Learn about Data Citation Standards.
Dataset Metrics
24 Downloads
Description
This dataset and replication package accompany the paper: “Beyond the Link: Assessing LLMs’ Ability to Classify Political Content Across Global Media.” by Alejandro De La Fuente-Cuesta, Alberto Martínez-Serra, R. Nienke Visscher, and Ana S. Cardenal (2025). The materials include the data and code necessary to reproduce all analyses presented in the main text and the Supplementary Information. The study investigates whether large language models (LLMs) can accurately classify political versus non-political news content based solely on URLs, compared to full-text analysis, across five countries (France, Germany, Spain, the UK, and the US). Using web-tracking data and manually coded ground-truth labels, we benchmark multiple state-of-the-art LLMs (Gemma-3-27B, Mistral-3.1-24B, Qwen-32B, Llama-3.1-8B, and DeepSeek-R1-Distill-Qwen-7B) to assess their performance, precision–recall trade-offs, and sources of bias in URL-only classification. The dataset is derived from web-tracking records of news consumption across five democratic countries. A subset of 1,140 URLs was manually coded by human annotators as either Political (POL) or Non-political (NON) content to serve as the gold standard. LLM predictions were then compared against these human labels to compute accuracy, F1, precision, recall, and Cohen’s Kappa metrics. All personal data were anonymized before analysis, and all procedures complied with GDPR and institutional ethical guidelines. Replication Instructions Open the .Rmd files in RStudio (R ≥ 4.2). Install the packages listed in the setup section. Knit the documents to reproduce the corresponding .html outputs. All analyses use open-source R packages and can be fully reproduced on any standard machine. If you use these data or materials, please cite: Martínez-Serra, A., De la Fuente-Cuesta, A., Visscher, R. N., & Cardenal, A. S. (2025). Beyond the Link: Assessing LLMs’ Ability to Classify Political Content Across Global Media. (2025-10-22)
Subject Social Sciences
Keyword Large Language Models, Media analysis, Political Content, URL
Notes Related Material: https://arxiv.org/pdf/2506.17435
License/Data Use Agreement

CC BY 4.0

Filter by
1 to 7 of 7 Files
metadata_llm_classification.tab
Tabular Data - 2.0 MB
Published Oct 23, 2025
3 Downloads
11 Variables, 11400 Observations UNF:6:3JOQw070RCv+caIx4qwiRg==
The data needed for the replication of the main text.
Preview "metadata_llm_classification.tab"
README.md
Markdown Text - 1.9 KB
Published Oct 23, 2025
3 Downloads
MD5: 39a9e67ad79cdce6c661c9af7f69f655
Preview "README.md"
Replication_Material.html
HTML - 1.3 MB
Published Oct 30, 2025
4 Downloads
MD5: 3694dd5d1c666107ec4fffac5a18f17d
Replication HTML of all the tables and figures in the paper.
Preview "Replication_Material.html"
Replication_Material.Rmd
R Notebook - 12.1 KB
Published Oct 30, 2025
4 Downloads
MD5: 0405fe9e2d5ae014486340785fc0eab6
Replication Code of all the tables and figures in the paper.
Replication_SI.html
HTML - 2.3 MB
Published Oct 23, 2025
3 Downloads
MD5: 4f69b463bd23ba3695c0b5a27f1addbc
The HTML with the code and results of the Supplementary Information.
Preview "Replication_SI.html"
Replication_SI.Rmd
R Notebook - 10.0 KB
Published Oct 23, 2025
4 Downloads
MD5: 92e7b981f6cc7ff7388c754daa3eb042
The R code for the replication of the Supplementary Information.
SI_metadata_llm_classification.tab
Tabular Data - 6.9 MB
Published Oct 23, 2025
3 Downloads
13 Variables, 2280 Observations UNF:6:VV9+42PqeCoCKyXOS7EEPQ==
The data needed for the replication of the Supplementary Information.
Preview "SI_metadata_llm_classification.tab"
Edit File

This file has already been deleted (or replaced) in the current version. It may not be edited.

Restrict Access

Restricting limits access to published files. People who want to use the restricted files can request access by default. If you disable request access, you must add information about access to the Terms of Access field.

Learn about restricting files and dataset access in the User Guide.

Enable access request
You must enable request access or add terms of access to restrict file access.
Save Changes
Edit Embargo

The selected file or files have already been published. Contact an administrator to change the embargo date or reason of the file or files.
Edit Retention Period

The selected file or files have already been published. Contact an administrator to change the retention period date or reason of the file or files.
Delete Files

The file will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Select File(s)

Please select one or more files.

Share Dataset

Share this dataset on your favorite social media networks.

Continue
Dataset Citations

Citations for this dataset are retrieved from Crossref via DataCite using Make Data Count standards. For more information about dataset metrics, please refer to the User Guide.

Sorry, no citations were found.
Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired or the files can only be transferred via Globus.

You may request access to any restricted file(s) by clicking the Request Access button.

Ineligible Files Selected

The selected file(s) may not be transferred because you have not been granted access or the file(s) have a retention period that has expired or the files are not Globus accessible.

You may request access to any restricted file(s) by clicking the Request Access button.

Download Options

The files selected are too large to download as a ZIP.

You can select individual files that are below the 500.0 MB download limit from the files table, or use the Data Access API for programmatic access to the files.

Select File(s)

Please select a file or files to be downloaded.

Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired.

Click Continue to download the files you have access to download.

Ineligible Files Selected

Some file(s) cannot be transferred. (They are restricted, embargoed, with an expired retention period, or not Globus accessible.)

Click Continue to transfer the elligible files.

Delete Dataset

Are you sure you want to delete this dataset and all of its files? You cannot undelete this dataset.

Delete Draft Version

Are you sure you want to delete this draft version? Files will be reverted to the most recently published version. You cannot undelete this draft.

Unpublished Dataset Preview URL

Preview URL can only be used with unpublished versions of datasets.

Unpublished Dataset Preview URL

Are you sure you want to disable the Preview URL? If you have shared the Preview URL with others they will no longer be able to use it to access your unpublished dataset.

Delete Files

The file(s) will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Compute

This dataset contains restricted files you may not compute on because you have not been granted access.

Deaccession Dataset

Are you sure you want to deaccession? This is permanent and the selected version(s) will no longer be viewable by the public.

Deaccession Dataset

Are you sure you want to deaccession this dataset? This is permanent an it will no longer be viewable by the public.

Version Differences Details

Please select two versions to view the differences.

Version Differences Details
 
Version:
Last Updated:
Version:
Last Updated:
Select File(s)

Please select a file or files for access request.

Select File(s)

Embargoed files cannot be accessed. Please select an unembargoed file or files for your access request.

Edit Tags

Select existing file tags or create new tags to describe your files. Each file can have more than one tag.

Request Access

  You need to Sign Up or Log In to request access.

Dataset Terms

Please confirm and/or complete the information needed below in order to request access to files in this dataset.

This dataset is made available under the following terms. Please confirm and/or complete the information needed below in order to continue.

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation shown on the dataset page.

CC BY 4.0

Preview Guestbook

Upon downloading files the guestbook asks for the following information.

Account Information

Package File Download

Use the Download URL in a Wget command or a download manager to download this package file. Download via web browser is not recommended. User Guide - Downloading a Dataverse Package via URL

https://dataverse.bsc.es/api/access/datafile/

Compute Batch
Clear Batch
Dataset Persistent Identifier Change Compute Batch
Compute Batch
Submit for Review

You will not be able to make changes to this dataset while it is in review.

Publish Dataset

Are you sure you want to republish this dataset?

Select if this is a minor or major version update.

Publish Dataset

This dataset cannot be published until COMPASS - COMplex Political And Social Simulations is published by its administrator.

Publish Dataset

This dataset cannot be published until COMPASS - COMplex Political And Social Simulations and Computational Social Sciences & Humanities Dataverse are published.

Return to Author

Return this dataset to contributor for modification. The reason for return entered below will be sent by email to the author.

Curation Status History
StatusDateAssigner
No records found.
Add/Edit a Version Note
Styled Citation