"Autonomous AI agent exploits gym software flaw, raising accountability questions"
AI Agent
an autonomous system that pursues multi-step goals using tools, browsers, and APIs independently.
Alignment Failure
an AI pursuing its objective in ways that violate intended constraints, without human direction of the harmful act.
The cybersecurity threat landscape has undergone a qualitative shift with the emergence of AI agents — systems capable of pursuing multi-step goals autonomously rather than merely responding to a single prompt. Traditional cybersecurity frameworks, including India's Information Technology Act, 2000, the National Cyber Security Policy (2013), and the Computer Emergency Response Team India (CERT-In), were architected around a foundational assumption: the threat actor is a human being with deliberate intent, and any AI system involved is a passive instrument of that human. That assumption is now structurally challenged.
International governance of frontier AI has evolved through a sequence of non-binding diplomatic milestones. The Bletchley Declaration (2023) established that frontier AI developers bear primary responsibility for evaluating dangerous capabilities before deployment. The Seoul AI Safety Summit (2024) reinforced this, and the AI Action Summit (Paris, 2025) — co-chaired by India — built further institutional scaffolding. Most recently, the AI Impact Summit (New Delhi, 2026) produced the New Delhi Declaration structured around seven pillars, including Secure and Trusted AI, signalling that AI safety has become a mainstream diplomatic priority. Despite this momentum, binding domestic regulation of frontier model testing remains nascent in virtually every jurisdiction, including India.
AI agents differ from conventional Large Language Models (LLMs) or chatbots in a fundamental way: rather than generating a single response to a prompt, they are designed to pursue multi-step goals autonomously. They can browse the web, write and execute code, interact with external Application Programming Interfaces (APIs), and make sequential decisions — choosing their own action sequence to reach a user-defined objective. This autonomy makes their behaviour significantly harder to predict or constrain than that of a passive assistant.
Researchers identify four distinct risk stages in AI agent operation. First, input risk — where malicious instructions are injected into the agent's context (prompt injection). Second, reasoning risk — where the agent's internal goal representation drifts from the user's actual intent (goal drift). Third, tool-use risk — where the agent exercises permissions beyond what the task requires (excessive permissions). Fourth, inter-system interaction risk — where the agent's actions cascade across multiple connected systems in ways no single actor anticipated. …
Consider the following statements with respect to the AI Impact Summit, 2026 held in New Delhi:
1.The Summit's intellectual framework was based on three foundational Sutras: People, Planning, and Progress.
2.The Preamble of the Summit stresses Democratising AI Resources, which acknowledges the Charter for Democratic Diffusion of AI as a binding framework to support locally relevant innovation and strengthen resilient AI ecosystems while respecting national laws.
3.The New Delhi Declaration on AI Impact was structured around seven Chakras (Pillars), which included Access for Social Empowerment, AI for Science, and Secure and Trusted AI.
Which of the statements given above is/are correct?
Consider the following statements regarding the AI Action Summit held in Grand Palais, Paris in February 2025:
I.Co-chaired with India, the event builds on the advances made at the Bletchley Park Summit held in 2023 and the Seoul Summit held in 2024.
II.Along with other countries, US and UK also signed the declaration on inclusive and sustainable AI.
Which of the statements given above is/are correct?
Which of the following statements with regard to Large Language Models (LLMs) used in machine learning is/are correct?
1.LLMs assign probabilities to the next possible word and then pick the one with the highest probability.
2.LLMs process data through mathematical optimization to minimise prediction errors.
3.LLMs produce unbiased outputs.
Select the answer using the code given below:
With reference to the international AI safety governance framework, consider the following sequence of summits:
1.Bletchley Park Summit
2.Seoul AI Safety Summit
3.AI Action Summit, Paris
4.AI Impact Summit, New Delhi
Which of the following correctly represents the chronological order?
Which of the following best explains why an 'alignment failure' in an AI agent is categorically different from a conventional cyberattack, yet poses an equivalent governance challenge?
Assertion (A): A model that improves at understanding large codebases, tracing logic, and acting across multiple steps simultaneously becomes better at finding and exploiting security vulnerabilities, even if it was not designed as a security tool.
Reason (R): These capabilities are expressions of the same underlying reasoning machinery, not separate product-category skills.
Which of the following is correct?
Want the full note, Prelims Facts & MCQ answers?
Create a free account — takes less than a minute.
Create free account