Skip to main content Skip to journal navigation menu Skip to site footer
contact@ukscip.com
Article

Epistemological Asymmetries in the Digital Archive: Cultural Bias in LLaMA‑1 and CommonCrawl‑Based Large Language Models and Its Implications for Digital Humanities Practice

Mayuree Pal ORCID ,  C.P. Rashmi ORCID
    Received: 11 February 2026; Revised: 1 May 2026; Accepted: 13 May 2026; Published: 25 June 2026

    Abstract

    Given the increasing integration of large language models (LLMs) in the digital humanities (DH) for textual analysis, archival research, and computational interpretation, an urgent need to decolonize LLM knowledge structures has emerged. This study conducts a secondary data analysis to highlight profound epistemological asymmetries embedded within LLaMA-1 and other documented CommonCrawl-based large language models and digital archives. The quantitative analysis reveals a stark representational disparity: approximately 82% of LLaMA-1's training set is explicitly English-filtered, and 49–56% of top-website content is in English, despite only 25.9% of global internet users speaking English as a first language. After more than a decade of systematic funding and infrastructure development, the digitisation of cultural heritage within European institutions remains stagnant at merely 16–23%. Crucially, no equivalent systematic surveys of cultural heritage digitisation exist for institutions in the Global South, indicating a severe global structural deficit. To mitigate these cascading disparities (from biased training data to uneven archival digitisation), this paper proposes the Epistemological Asymmetry Framework (EAF). The EAF serves as a structured, verifiable tool designed for DH practitioners and research institutes to audit, measure, and actively address cultural and linguistic biases. Ultimately, establishing equitable digital practices requires both rigorous multilingual corpus auditing and significant infrastructural investments in global heritage digitisation.

    Keywords

    References

    1. 1. W3Techs. Historical Trends in the Usage Statistics of Content Languages for Websites. Available online: https://w3techs.com/technologies/history_overview/content_language/ms (accessed on 30 April 2026).
    2. 2. Bender, E.; McMillan-Major, A.; Shmitchell, S.; et al. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Online, 3–10 March 2021; pp. 610–623. DOI: https://doi.org/10.1145/3442188.3445922
    3. 3. Blodgett, S.L.; Barocas, S.; Daumé III, H.; et al. Language (Technology) is Power: A Critical Survey of "Bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 5–10 July 2020; pp. 5454–5476. DOI: https://doi.org/10.18653/v1/2020.acl-main.485
    4. 4. Touvron, H.; Lavril, T.; Izacard, G.; et al. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint 2023, arXiv:2302.13971. DOI: https://doi.org/10.48550/arXiv.2302.13971
    5. 5. Dodge, J.; Sap, M.; Marasović, A.; et al. Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic, 7–11 November 2021; pp. 1286–1305. DOI: https://doi.org/10.18653/v1/2021.emnlp-main.98
    6. 6. Luccioni, A.; Viviano, J. What's in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online, 1–6 August 2021; pp. 182–189. DOI: https://doi.org/10.18653/v1/2021.acl-short.24
    7. 7. Internet World Stats. Internet World Users by Language: Top 10 Languages Used in the Web; Internet World Stats: Bogotá, Colombia, 2020.
    8. 8. OBDILCI. English@Web: An Historical Bias. Available online: https://www.obdilci.org/projects/main/englishweb/ (accessed on 8 May 2026).
    9. 9. Baack, S. Training Data for the Price of a Sandwich: Common Crawl's Impact on Generative AI; Mozilla Foundation: San Francisco, CA, USA, 2024. Available online: https://assets.mofoprod.net/network/documents/Common_Crawl_Mozilla_Foundation_2024.pdf
    10. 10. Raffel, C.; Shazeer, N.; Roberts, A.; et al. Exploring the Limits of Transfer Learning with a Unified Text-To-Text Transformer. arXiv preprint 2019, arXiv:1910.10683. DOI: https://doi.org/10.48550/arXiv.1910.10683
    11. 11. Joshi, P.; Santy, S.; Budhiraja, A.; et al. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 5–10 July 2020; pp. 6282–6293. DOI: https://doi.org/10.18653/v1/2020.acl-main.560
    12. 12. Ahuja, K.; Diddee, H.; Hada, R.; et al. MEGA: Multilingual Evaluation of Generative AI. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6–10 December 2023; pp. 4232–4267. DOI: https://doi.org/10.18653/v1/2023.emnlp-main.258
    13. 13. Lai, V.D.; Ngo, N.; Veyseh, A.P.B.; et al. ChatGPT beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 6–10 December 2023; pp. 13171–13189. DOI: https://doi.org/10.18653/v1/2023.findings-emnlp.878
    14. 14. Europeana Pro. ENUMERATE Results. Available online: https://pro.europeana.eu/page/results (accessed on April 30 2026).
    15. 15. Sheldon, Z. The Archived Web: Doing History in the Digital Age. Inf. Commun. Soc. 2019, 23, 309–311. DOI: https://doi.org/10.1080/1369118X.2019.1682636
    16. 16. UNESCO. Charter on the Preservation of the Digital Heritage; UNESCO: Paris, France, 2009. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000179529.locale=en
    17. 17. UNESCO. Advancing Access to Information and Digital Preservation: IFAP's Impact on Digitizing Documentary Heritage. Available online: https://www.unesco.org/en/articles/advancing-access-information-and-digital-preservation-ifaps-impact-digitizing-documentary-heritage (accessed on 28 April 2026).
    18. 18. Risam, R. New Digital Worlds: Postcolonial Digital Humanities in Theory, Praxis, and Pedagogy; Northwestern University Press: Evanston, IL, USA, 2018.
    19. 19. Noble, S.U. Algorithms of Oppression: How Search Engines Reinforce Racism; New York University Press: New York, NY, USA, 2018.
    20. 20. Couldry, N.; Mejias, U.A. Data Colonialism: Rethinking Big Data's Relation to the Contemporary Subject. Televis. New Media 2019, 20, 336–349.
    21. 21. Fiormonte, D.; Chaudhuri, S.; Ricaurte, P. Global Debates in the Digital Humanities; University of Minnesota Press: Minneapolis, MN, USA, 2022.
    22. 22. Cao, Y.; Li, Z.; Lee, S.; et al. Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), Dubrovnik, Croatia, 5 May 2023; pp. 53–67. DOI: https://doi.org/10.18653/v1/2023.c3nlp-1.7
    23. 23. Trott, S. Large Language Models and the Wisdom of Small Crowds. Open Mind 2024, 8, 723–738. DOI: https://doi.org/10.1162/opmi_a_00144
    24. 24. Tao, Y.; Viberg, O.; Baker, R.S.; et al. Cultural Bias and Cultural Alignment of Large Language Models. PNAS Nexus 2024, 3, e346. DOI: https://doi.org/10.1093/pnasnexus/pgae346
    ×