Skip to content

Why we turn to AI systems and why cognitive integrity is at stake

Governing AI for cognitive integrity part one

This is part one of “Governing AI for cognitive integrity”, a report examinig AI’s impact on human autonomy through a cognitive lens. Read the other chapters here

Introduction

AI-mediated technologies[1] are now part of daily life because they address real human needs. People rely on generative AI chatbots and social media to find and make sense of information, maintain relationships and communities, communicate and create, and increasingly to manage emotional distress.

The challenge many users now face is how to fulfil those needs while preserving their autonomy and agency, and continuing to exercise the cognitive capacities, on which those needs ultimately depend. Public debate has tended to focus on the most visible manifestations of harm: compulsive use and “brain rot”[2], suicide-related cases[3], AI-induced psychosis[4], or claims that an entire generation is becoming addicted to social media[5]At the other end sit concerns about a future in which societal systems no longer depend on human participation, and so stop being answerable to it[6]. These cases and scenarios matter, but they also emerge from the gradual and cumulative ways in which humans interact with AI systems.

The governance challenge is therefore to preserve the benefits these technologies can provide while becoming more attentive to effects that are pervasive, cumulative and unevenly distributed. Recent recommendations from the European Commission’s Special Panel on Child Safety Online mark an important step in this direction[7]. The Panel recognises both the needs these technologies can help fulfil and the importance of supporting children in learning to use them, rather than relying on access restrictions alone. At the same time, it acknowledges that children are particularly sensitive to certain properties of these systems because key cognitive capacities — including attention, social cognition, emotional regulation and behavioural control — are still developing. In practice, this means that some design features can exploit vulnerabilities that are especially pronounced at particular stages of development.

But the Panel’s mandate is necessarily bounded around children. The vulnerabilities it identifies do not disappear at the age of majority. A governance framework capable of addressing cognitive harm therefore needs to recognise heightened vulnerability where it exists, without treating vulnerability as a category into which only some users fall.

A vocabulary that doesn’t reach

Across political, regulatory, and scientific debates in this space, a proliferation of terms describe what is at stake when people interact with these systems: brain health, brain skills, mental health, mental wellbeing, mental integrity, cognitive liberty, neurorights.

Some protect the neurological substrate, others protect states and outcomes, whether a clinical condition, a state of wellbeing, or a skill outcome, each of which registers only once the processes producing it have run their course. Some others protect the contents of the mind and who may access it, locating the wrong in what ends up in someone’s head — a belief, for example — or whether they consented to the source.

The dominant framing in policy discourse remains mental health. The problem is not the concept but its operationalisation, which proceeds through diagnostic categories such as depression, anxiety, addiction, and psychosis. These extremes are how the harms entered public and political awareness, through headlines and bestselling books. Such operationalisation is convenient because diagnostic criteria provide the threshold conditions, measurable outcomes, and causal claims that regulatory arguments require. But it systematically fails as a governance target in two respects.

First, it locates harm at the point of clinical presentation, after the conditions producing it have operated for months or years. Attentional erosion, emotional dependency, the progressive substitution of autonomous judgement by AI-generated outputs: these cause real harm long before any diagnostic threshold is crossed. A governance framework that waits for clinical diagnosis is reactive by design.

Second, psychiatric conditions are multifactorial. Depression emerges from genetic predisposition, family circumstances, developmental stage, and social and economic environment —all of which can be amplified or triggered when interacting with these technologies. Making pathological outcomes the target allows platform executives to invoke causal complexity most effectively. A mechanism-based target changes what the evidence must show: not whether a platform caused a condition, but whether a specific feature operates on an identified risk pathway in a demonstrable way.

The WHO definition of mental health points the same way. Mental health is defined not as the absence of disorder, but as a state of wellbeing that enables people to cope with the stresses of life, realise their abilities, learn and work well, and contribute to their community[8]. It exists on a continuum, experienced differently across individuals and circumstances, with clinical conditions as extremes and a wide range of mental states associated with distress or impaired functioning in between. That definition already implies a governance standard calibrated to gradual and uneven effects, attentive to erosion of capacity well before it becomes clinically legible. The problem is that without adequate operationalisation, policy instruments cannot reach there.

What follows from these failures is an attempt to build a governance framework adequate to what the mental health tradition, at its most rigorous, already implies. Such a framework would need to measure effects on capacity and functioning before clinical thresholds are crossed, operate with an evidentiary standard based on contributory mechanisms rather than exclusive causation, and track a continuum rather than a binary. Cognitive integrity, developed in the following section, is proposed as a framework that satisfies those requirements.

Cognitive integrity as normative standard

If the relevant mental health harms are sub-clinical, gradual, and mechanism-specific, the normative standard must operate at the level of the cognitive processes that underpin mental health. Cognitive integrity is proposed here as that standard[9].

We define cognitive integrity as the capacity of individuals to engage autonomously in the cognitive processes that underpin their functioning — including attention, decision-making, critical thinking, and emotional regulation — and to act in ways that reflect their own goals rather than being steered by external systems. Cognitive integrity does not assume the absence of external influence, which is constitutive of human development. It is premised on preserving the conditions under which influence remains compatible with self-determination. Those conditions require, at a minimum, that influence operates through processes a person can in principle be aware of, evaluate against their own goals, and contest. They require, further, that the influencing system does not exploit structural asymmetries of knowledge and power — specifically, of and over a person’s cognitive vulnerabilities — in ways that bypass deliberation.

“Cognitive” reflects the specificity of the harm pathway through which AI-mediated technologies affect human functioning: attentional capture, substitution of autonomous judgement, dependency formation, among others, operate at the level of cognition before they manifest as mental health outcomes. “Integrity” captures two distinct qualities: capacities that remain the person’s own rather than being degraded, hijacked, or replaced by technologies, and that can be exercised and developed across the lifespan. Cognitive integrity therefore represents the baseline condition from which everything else the WHO definition envisions becomes possible.

Cognitive integrity is exercised through the everyday activities for which people increasingly rely on AI-mediated technologies. The analysis therefore starts with the needs that drive that reliance, and with the cognitive capacities involved in fulfilling them.

The “Five Cs”: needs that drive adoption of AI systems

Human beings increasingly adopt and rely on AI-enabled technologies to fulfil core psychological needs: to comprehend the world, connect and communicate with others, create knowledge, meaning and artefacts, express the self, and cope with life’s adversities. Fulfilling those needs cannot simply be outsourced. This framework protects that engagement and the cognitive capacities it requires. The Five Cs — Comprehend, Connect, Create, Communicate, Cope — map the domains where AI-mediated technologies are more deeply involved in how people exercise these capacities, and where losing autonomy over them carries the greatest cost[10] [11]. Autonomy is the cross-cutting condition: any of the five needs can be fulfilled to varying degrees of it, along a spectrum that runs from genuine human exercise to AI substitution. Psychological research bears this out: needs fulfilled through externally controlled or substituted means do not produce the same wellbeing, identity coherence, or intrinsic motivation as needs fulfilled through autonomous engagement[12] [13]. And fulfilling these needs draws on cognitive functions — attention, memory, critical thinking, decision-making, emotional regulation, social cognition, and intrinsic motivation among others — that are themselves subject to influence, capture, and degradation. Like physical capacities, cognitive ones are maintained through use: what is systematically bypassed gradually atrophies[14]. Cognitive integrity names the condition in which these functions are developed through use, oriented by the person’s own goals, and not subverted or exploited by systems optimised for other ends.

As the figure below shows, the pattern is one of displacement: the need appears to be met, while the cognitive capacities through which it would otherwise be fulfilled are exercised less. AI systems can support these capacities; the concern arises when AI systems instead displace the agency and autonomy[15] with which they are exercised, either by substituting for them or by steering them toward the system’s objectives rather than the person’s own.

Figure 2. The Five Cs under high and low human autonomy. For the sources, see the extended references section.  

Governance of constitutional rights for cognitive integrity

This ‘Five C’s’ framework reframes the protection of cognitive integrity as a structural governance challenge to regulate the conditions under which cognitive functions — indispensable to fulfil psychological needs — are developed, exercised and protected.

The EU Charter of Fundamental Rights and the UN Convention on the Rights of the Child (UNCRC) provide the constitutional foundation for this governance challenge, even without explicitly naming cognitive integrity (See Table 1 below for a summary of main articles and analysis) .

The Charter provisions on human dignity (Article 1), mental and physical integrity (Article 3), respect for private life (Article 7), protection of personal data (Article 8), freedom of thought (Article 10), freedom of expression (Article 11), and rights of the child (Article 24) collectively ground cognitive integrity.

Article 3 on mental and physical integrity suggests strong limits on practices that deliberately target and degrade cognitive functions for commercial purposes. Article 10 on freedom of thought is underused in existing doctrine but arguably the strongest foundation for protecting epistemic autonomy, since freedom of thought protects not just the holding of opinions but also guards against their illegitimate alteration and supports the conditions under which they are formed[16]. Article 11 on freedom of expression, which encompasses not only the right to communicate but the right to receive information, grounds protection of the informational conditions for cognitive integrity. Article 24 on children’s rights amplifies all of the above, since children’s capacities are still forming and therefore disproportionately vulnerable to cognitive harms. The UNCRC reinforces this specifically through its provisions on cognitive development, protection from manipulation, and the best interest principle.

Existing case-law has already begun operationalising these provisions in ways that implicitly protect cognitive integrity. The German Federal Constitutional Court’s Volkszählungsurteil (1983) established[17] the right to informational self-determination, reasoning that individuals who cannot know what information is held about them may adjust their behaviour and refrain from exercising their freedoms. This principle influenced European data protection law and is reflected in Article 8 of the EU Charter and the GDPR’s normative foundations. At the ECtHR, surveillance cases[18] from Szabó and Vissy v. Hungary to Big Brother Watch v. The United Kingdom recognise that modern technologies can generate detailed profiles of individuals’ private lives, implicitly acknowledging that the conditions under which people form their choices fall within the scope of Article 8 ECHR[19]. Together, these strands suggest that the core conditions for cognitive integrity are already embedded in European rights frameworks, even where the doctrine has not yet been explicitly articulated.

Constitutional rights then operate at three levels in governance design:

  1. They ground legislative and regulatory intervention, establishing that the protection of cognitive integrity is enforcement of fundamental rights, not overreach, which matters when commercial interests are invoked against proposed design restrictions.
  2. They guide proportionality and interpretation when operational rules are ambiguous: what counts as manipulative, when is profiling harmful, how should vulnerability be defined. These questions gain legal content when assessed against dignity and mental integrity rather than purely against consumer harm or market distortion.
  3. They enable claims where existing frameworks don’t clearly reach: technical compliance with data or competition rules is no guarantee that mental integrity and human dignity remain intact.

This means that operational governance layers —market interventions, data protections, design rules, content moderation— are the instruments through which constitutional rights are implemented and the first place where the gap between constitutional principles and their implementation becomes visible.

Constitutional grounding is necessary but several structural gaps constrain its practical impact.

Doctrine remains underdeveloped: courts have not yet translated principles of dignity and mental integrity into enforceable standards for the harmful impact of AI-enabled systems on cognitive integrity, including cognitive manipulation[20] and attentional capture, though authoritative signals are emerging. The European Commission’s preliminary findings against TikTok for addictive design in February 2026 represent the first administrative operationalisation of DSA systemic risk obligations in relation to mental health[21].

The Charter rights most central to cognitive integrity, namely dignity, mental integrity, and freedom of thought, do not directly bind private actors in the way that operational frameworks do; legislative and regulatory translation remains necessary, and that translation is always incomplete and subject to sustained industry pressure.

Even where rights clearly apply, enforcement authorities frequently lack the technical expertise, resources, or political will to act. And when they do, economic interests of tech companies often push back: arguments that cognitive protection stifles innovation or restricts expression must be weighed against rights claims, creating persistent tension between protection and permissiveness.

Conclusion

Constitutional grounding is only the starting point. The practical protection of cognitive integrity depends on how those principles are translated into the governance of AI-mediated systems.

The next layers of the framework examine how business incentives shape what systems are optimised for, how those incentives are translated into practice through data and design choices, and how far existing governance frameworks reach each of those layers.

EU governance of cognitive integrity: An overview of regulatory reach and gaps

Instrument
Governance layer
What it requires
Where it stops
Enforced by

EU Charter Arts 1, 3, 7, 8, 10, 11, 24

Dignity, mental integrity, freedom of thought, expression bind EU law and its interpretation across all layers.

No operational standard; binds states, not platforms directly; untested for cumulative, sub-perceptual influence.

Courts (CJEU, national); regulators invoking Charter

DMA Arts. 5(2), 6(5), 6(9), 7

Contestability and conduct obligations for designated gatekeepers of core platform services; interoperability; data portability; self-preferencing restrictions; prohibition on cross-service data combination.

Designated gatekeepers only; most platforms and emerging AI business models outside scope; competition lens leaves the engagement model itself untouched. Portability meaningful only if genuine alternatives exist.

Commission

CSDDD

Identify, prevent, mitigate and account for adverse human-rights impacts, explicitly including mental health, across value chains

Weakened before implementation: largest companies only, harmonised civil liability removed, enforcement delayed to 2029

National supervisory authorities

GDPR Arts 4(1-4-14-15), 5(1)(b)-(c), 6, 7, 9, 22, 25

Lawful basis for processing; profiling limits; special-category safeguards; data protection by design; consent requirements; purpose limitation and data minimisation.

Inferred mental states unprotected and outside Art. 22 scope; consent not meaningful in practice; purpose limitation circumvented; enforcement too slow for system-scale harms.

National DPAs; EDPB coordination

AI Act Arts 5, 6–7, 10, 50, 51, 53; Annex III

Prohibits manipulative and subliminal techniques and vulnerability exploitation; conformity assessment for high-risk systems; transparency; GPAI systemic-risk duties; training data governance and transparency.

Intent and awareness thresholds hard to prove; companion and engagement systems fall outside high-risk classification; Art. 53 training data summaries lack granularity; implementation delayed to December 2027 (Annex III).

AI Office + national market surveillance authorities

Digital Fairness Act draft, expected Q4 2026

Expected to target dark patterns and deceptive design falling outside AI-system definitions.

Definitions unsettled; undetermined coverage of interface features specific to generative AI.

To be determined

UCPD

Prohibits unfair, misleading and aggressive commercial practices

Average-consumer standard poorly suited to personalised influence; consumer-protection lens only

National consumer authorities coordinated through the CPC network

DSA Arts 14-16, 28, 34–35, 37, 38, 40

VLOPs and VLOSEs assess and mitigate systemic risks to mental wellbeing stemming from design and functioning and offer recommender systems not based on profiling; notice-and-action mechanisms for illegal content; age-appropriate design for minors; independent audits; researcher data access.

Built for intermediaries; above threshold generative AI captured only via search functionality; no benchmarks for wellbeing risk; non-profiling recommender system is not default; lawful-but-harmful content unresolved. Notice-and-action logic mismatched to cumulative harm; Art. 37 audits assess process not risk; Art. 40 data access slow and contested.

Commission + national DSCs

Product Liability Directive; implementation Dec 2026

Covers software and AI systems; recognises medically relevant psychological harm as compensable damage; defectiveness assessed against expected safety.

Claimants must prove defect, damage, and causal link; “defect” legally unresolved for engagement-optimised systems; individual causation mismatched to cumulative harm.

National courts

Representative Actions Directive

Qualified entities can bring collective claims for consumer harm, enabling injunctions, damages, or other remedies.

High evidentiary barriers without platform data; no causation standard for cognitive harm; no framework for cumulative population-level claims.

National courts; qualified entities

Please note: Note. This table provides a summary of the policy analysis across the different governance layers. Please refer to the main text for the complete analysis and to the Governance map for a description of all relevant provisions of these and other instruments.

Acknowledgements

We sincerely thank the following people for reviewing this report and for their invaluable comments and feedback:

  • Alejandro Tlaie Boria, AI Policy Advisor at Pour Demain
  • Andrea Palumbo, Researcher at Centre for IT & IP Law, KU Leuven
  • David de Segovia Vicente, Researcher at Media, Innovation and Communication Technologies, Ghent University
  • Mike X Cohen, Neuroscientist and independent educator
  • Stefano Palminteri,  Professor of Cognitive Artificial Intelligence, École Normale Supérieure Paris; Research Director at Institut National de la Santé et de la Recherche Médicale

[1] “An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.”(OECD, 2024). For the purposes of this publication, the analysis focuses on two categories of AI systems: social media’s recommender systems and generative AI chatbots.

[2] Macchi, F., et al., “The effects of ‘brain rot’: How junk content is damaging our minds,” El País, 2024, english.elpais.com/technology/2024-12-26/the-effects-of-brain-rot-how-junk-content-is-damaging-our-minds.html, accessed December 12, 2025

[3] de Guzman, C., “AI Chatbots Can Be Manipulated to Provide Advice on How to Self-Harm, New Study Shows,” TIME, 2025, time.com/7306661/ai-suicide-self-harm-northeastern-study-chatgpt-perplexity-safeguards-jailbreaking, accessed accessed December 12, 2025

[4] Wei, M., “The Emerging Problem of ‘AI Psychosis’,” Psychology Today, 2025, psychologytoday.com/us/blog/urban-survival/202507/the-emerging-problem-of-ai-psychosis, accessed December 12, 2025

[5] Haidt, J., The Anxious Generation: How the Great Rewiring of Childhood Is Causing an Epidemic of Mental Illness, Penguin Press, 2024.

[6] Kulveit, J., et al., “Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development,” arXiv, 2025, arxiv.org/abs/2501.16946, accessed 8 March 2026.

[7]  European Commission, “Child Safety Online: Protecting and Empowering Minors in a Digital World,” Report by the Co-Chairs of the Special Panel on Child Safety Online, Prof. Dr. Jörg M. Fegert and Dr. Maria Melchior, 2026, commission.europa.eu/document/download/d833504d-5ec3-4fac-945f-38e7d0bd5326_en, accessed 13 July 2026.

[8] World Health Organization, “Mental health: strengthening our response,” World Health Organization, 2023, who.int/news-room/fact-sheets/detail/mental-health-strengthening-our-response, accessed 15 February 2026.

[9] The standard proposed here converges with existing accounts, notably Zuk’s conceptualisation of mental integrity and Chappuis’s cognitive integrity. Both accounts locate the object of protection at the level of processes rather than contents or outcomes. Our framework supplies the operationalisation, specifying the pathways through which cognitive integrity is affected and locating shared cognitive vulnerabilities as the entry point.

Zuk, P., ‘Mental integrity, autonomy, and fundamental interests’, Journal of Medical Ethics, 50(10), 2024, jme.bmj.com/content/50/10/676, accessed 20 July 2026.
Chappuis, I., quoted in Giussani, B., La mente sotto assedio. Come non lasciarsi manipolare nell’era dell’intelligenza artificiale, Casagrande, 2026, p. 22, edizionicasagrande.com/libri_dett.php?id=2994, accessed 1 August 2026
.

[10] Deci, E.L. and Ryan, R.M., Intrinsic Motivation and Self-Determination in Human Behavior, Springer, 1985, link.springer.com/book/10.1007/978-1-4899-2271-7, accessed 25 April 2026.

[11] The Five Cs are derived from Self-Determination Theory (STD), one of the most empirically validated frameworks in motivational psychology, grounded in substantial cross-cultural literature on human needs and development. STD identifies competence, relatedness, and autonomy as universal psychological needs whose fulfillment is constitutive of wellbeing and identity development. The Five Cs disaggregate competence into four distinct modes through which people build their competence in the interaction with the environment: understanding reality (Comprehend), producing meaning and artefacts from one’s own generative impulse (Create), negotiating reality through self-expression and exchange (Communicate), and managing the cognitive and emotional demands it places on the individual (Cope). Relatedness maps onto Connect, with Communicate and Cope carrying their relational dimension as well. The more significant adaptation concerns autonomy, which is repositioned as a cross-cutting condition which qualifies how all five needs are fulfilled; namely, through genuine exercise of a person’s own capacities, oriented by their own goals. In other words, any of the five needs can be fulfilled to varying degrees of autonomy, along a spectrum that, in this case, runs from human autonomy to AI system’s autonomy.

[12] Deci, Edward L., et al., “The ‘What’ and ‘Why’ of Goal Pursuits: Human Needs and the Self-Determination of Behavior”, Psychological Inquiry, 2000, doi.org/10.1207/S15327965PLI1104_01 , accessed 13 November 2025.

[13] Ryan, Richard M., et al., “Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being”, American Psychologist, 2000, pubmed.ncbi.nlm.nih.gov/11392867/ , accessed 18 November 2025.

[14] Von Bernhardi, Rommy, et al., “What Is Neural Plasticity?”, in von Bernhardi, Rommy, et al. (eds), The Plastic Brain, 2017, doi.org/10.1007/978-3-319-62817-2_1 , accessed 12 November 2025.

[15] For the purposes of this report, agency refers to the capacity to be the initiating cause of one’s own behaviour. Autonomy requires not just that one acts but that one’s actions genuinely express one’s own goals.

[16] Bublitz, J. C., “Rethinking the Right to Freedom of Thought: A Multidisciplinary Perspective,” Human Rights Law Review, 2022, academic.oup.com/hrlr/article/22/4/ngac028/6809071, accessed 20 March 2026.

[17] freiheitsfoo, “Census Act (Translation of the German Constitutional Court Judgment),” freiheitsfoo, freiheitsfoo.de/census-act/ , accessed 20 March 2026.

[18] Information Law & Policy Centre, “Not So Grand: The Big Brother Watch ECtHR Grand Chamber Judgment,” Information Law & Policy Centre Blog, 2021, infolawcentre.blogs.sas.ac.uk/2021/05/28/not-so-grand-the-big-brother-watch-ecthr-grand-chamber-judgment, accessed 20 March 2026.

[19] ECtHR jurisprudence is cited here as interpretive authority rather than as a separate constitutional instrument. Under Article 52(3) of the EU Charter, Charter rights corresponding to ECHR rights must be interpreted as offering at least the same level of protection as the Convention. ECtHR case law on surveillance, privacy, and psychological autonomy therefore directly informs how Charter provisions, particularly Articles 7 and 8, should be read in the context of cognitive integrity governance.

[20] United Nations Human Rights Council Advisory Committee, “Impact of disinformation on the enjoyment and realization of human rights,” United Nations, 2026, docs.un.org/en/A/HRC/61/68, accessed 23 March 2026.

[21] A March 2026 US jury verdict finding Meta and YouTube negligent for addictive platform design, the first of its kind, further signals that product liability frameworks are beginning to reach design-induced psychological harm, though under a different legal tradition and with appeal proceedings pending.

Extended references

Read part two of “Governing AI for cognitive integrity”. 

Subscribe

Stay up to date on the research, debates, and governance of climate intervention technologies

A view into the world of neurotech, covering new tech, policy, research, and the debates that matter



Centre for Future Generations
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.