Introduction

AI-driven OT security is becoming increasingly important as industrial organisations seek faster ways to detect, investigate, and respond to cyber threats.. AI and machine learning can help security teams analyse large volumes of telemetry, identify abnormal behaviour, correlate events, prioritise alerts, and investigate incidents faster than traditional manual approaches.

In Operational Technology environments, however, the use of AI introduces an additional challenge. OT systems interact directly with physical processes. A cybersecurity response may therefore affect not only networks and endpoints, but also equipment availability, production continuity, reliability, and safety.

This creates an important question for the future of AI-driven OT cybersecurity: if an AI system can identify an attack, under what conditions should it also be allowed to respond automatically?

Current cybersecurity and AI risk-management guidance suggests that the answer should not be based only on the accuracy of the AI model. Safe autonomous response also depends on operational context, clearly defined authority, human oversight, explainability, system criticality, and the potential consequences of an incorrect action.

For projects such as AICOT, these considerations are particularly relevant because the responsible use of AI in OT requires careful attention to operational constraints, human oversight, and the potential consequences of automated decisions.

What Current Guidance Says About AI Decision-Making

The NIST Artificial Intelligence Risk Management Framework provides a useful starting point for designing AI systems that operate in sensitive environments. The framework focuses on characteristics including reliability, safety, security, resilience, accountability, transparency, explainability, and interpretability.

One important principle is that the roles of humans and AI systems should be clearly defined. NIST notes that AI systems can operate across a spectrum ranging from fully autonomous decision-making to systems that defer decisions to human experts.

This means that human oversight should not simply be added at the end of development. The level of autonomy should be considered during system design.

For OT cybersecurity, this raises practical questions. Which decisions can the AI make independently? Which actions should require approval? Who is responsible for reviewing the recommendation? What happens when the AI does not have enough information to determine whether an action is safe?

OT Security Changes the Risk Model

OT cybersecurity has requirements that differ from conventional IT environments, particularly because industrial systems interact directly with physical processes. NIST’s OT security guidance highlights performance, reliability, and safety as essential considerations when protecting these environments, which means that a security response cannot be evaluated solely on whether it successfully blocks an attacker.

Consider an AI system that detects suspicious activity from an engineering workstation and automatically isolates it. From a cybersecurity perspective, this may appear to be an appropriate containment action. From an operational perspective, however, the same workstation could be supporting an active production process, providing engineering supervision, or participating in an authorised maintenance procedure. In that situation, the AI may be correct in identifying the threat while still choosing a response that creates unnecessary operational risk.

This distinction is fundamental when designing automated defence mechanisms for OT environments. Effective response logic must consider not only the likelihood of malicious activity, but also the potential impact of the action taken to contain it.

Best Practice 1: Separate Threat Confidence From Response Confidence

A useful design principle is to distinguish between two different forms of confidence: threat confidence and response confidence. Threat confidence reflects how certain the system is that the observed behaviour represents malicious or unauthorised activity, while response safety reflects how certain it is that the proposed action can be performed without causing unacceptable operational consequences.

These two values should not be treated as equivalent. An AI system may be highly confident that a device has been compromised while still lacking enough information to determine whether disconnecting that device is operationally safe. For example, a 99% confidence that an engineering workstation has been compromised should not automatically imply a 99% confidence that the workstation should be isolated immediately.

Keeping these two forms of confidence separate can help prevent automated security controls from turning a cyber incident into an operational incident. It allows the system to evaluate not only whether a threat is likely to be real, but also whether the proposed response is appropriate for the specific OT environment.

Best Practice 2: Use Human-in-the-Loop Controls for High-Impact Actions

Human oversight remains one of the most important safeguards for AI systems operating in environments where incorrect decisions can create significant consequences.

NIST’s AI RMF specifically highlights the importance of clearly defining human responsibilities when AI systems are used in operational decision-making.

For OT security, the objective should not simply be to place a human somewhere in the workflow. The system should define exactly when human approval is required and who has the authority to provide it.

For example, collecting additional telemetry may be fully automated. Blocking a suspicious remote-access session may require cybersecurity approval. Isolating an engineering workstation may require both cybersecurity and OT approval. Interrupting communications with a critical controller may require an even higher level of operational authorisation.

The level of human involvement should therefore increase as operational impact increases.

 

 

 

Best Practice 3: Build OT Context Into the AI Decision

AI response decisions should not rely solely on network indicators or anomaly scores. To support a meaningful response, the system needs to understand as much as possible about the environment surrounding the event and the role of the affected assets.

A network-level alert may only show that one IP address communicated with another, while protocol-aware analysis can reveal that the communication involved a PLC configuration operation. Asset awareness can add further context by identifying the destination as a production controller, and identity information may show that the activity originated from a vendor account through remote access. Operational context can then determine whether the activity occurred during an authorised maintenance period and whether the controller was supporting active production at the time.

Each additional layer of context improves the quality of the security decision by helping distinguish legitimate operational activity from potentially malicious behaviour. The goal should be to move away from alert-driven response and toward context-aware response, where decisions consider the event, the asset, and the operational environment.

Best Practice 4: Start With Low-Risk Automation

Autonomous response does not need to begin with high-impact actions such as device isolation or traffic blocking. A safer approach is to first automate activities that improve visibility and accelerate investigation without directly interfering with industrial operations.

These actions can include gathering additional telemetry, preserving logs, starting packet capture, correlating related alerts, retrieving asset information, checking vulnerability exposure, enriching incidents with threat intelligence, increasing monitoring around affected devices, and escalating incidents to the appropriate response team. This type of automation can provide many of the speed and efficiency benefits associated with AI-driven response while avoiding unnecessary changes to the production environment. What qualifies as a low-risk action will vary between organisations and should be defined according to asset criticality, operational requirements, and established security policies.

For an AI-driven OT security platform, this creates a more controlled path toward greater automation. Initial capabilities can focus on evidence collection, enrichment, and decision support, while more active containment actions are introduced only when the necessary operational context, safeguards, and approval mechanisms are in place.

Best Practice 5: Use Predefined and Approved Response Playbooks

High-impact security responses should not be improvised by an AI model. Instead, automated actions should operate within predefined response playbooks that have been reviewed and approved by cybersecurity teams, OT engineers, operators, and other relevant stakeholders.

These playbooks should clearly define the conditions under which an action is permitted, the systems it may affect, the approvals required before execution, the expected operational impact, and the procedure for reversing the action if necessary. This creates clear boundaries around the authority of the AI system and supports constrained automation rather than unrestricted decision-making.

Best Practice 6: Apply Least Privilege to AI Agents

The principle of least privilege should apply to AI systems in the same way it applies to users, applications, and services. An AI component responsible for analysing telemetry, for example, does not automatically need permission to modify firewall rules, while a component that generates response recommendations does not necessarily require the authority to execute those actions.

Separating analytical capabilities from execution privileges helps reduce the potential impact of incorrect AI decisions and limits the consequences of a compromise affecting the AI infrastructure itself. This separation is becoming increasingly important as cybersecurity platforms adopt more agentic AI capabilities that can interact directly with other security technologies and potentially influence defensive actions across the environment.

Applying least privilege to AI therefore helps ensure that each component has only the level of access required for its specific function, reducing unnecessary authority and creating stronger boundaries around automated decision-making.

Best Practice 7: Make AI Recommendations Explainable

Explainability becomes especially important when AI is involved in high-impact security decisions. A generic output such as “Threat score: 96%” provides limited value to an OT operator because it does not explain why the activity was classified as suspicious or what evidence influenced the result.

A more useful system should present the factors behind its assessment. For example, it might show that a vendor account established remote access outside an authorised maintenance window, connected to an engineering workstation it does not normally use, and initiated a configuration write against a high-criticality PLC. Providing this context allows the operator to understand the reasoning behind the alert rather than relying only on a numerical score.

The same principle should apply to response recommendations. If the AI suggests isolating a system, it should also present the evidence supporting that recommendation and indicate the expected operational consequences of the action. This gives the operator the information needed to validate the decision and make an informed response rather than simply trusting the AI output.

Best Practice 8: Ensure Every Automated Action Is Auditable

AI-driven response should produce a complete and traceable audit trail that allows security teams to reconstruct how a decision was reached and what happened afterwards. This should include what the system observed, which data influenced the assessment, what recommendation was generated, which action was proposed, whether human approval was required, who provided that approval, what action was ultimately executed, and the outcome of that response. This traceability also supports accountability by making it possible to determine how and why a particular response decision was made.

Maintaining this level of traceability is important for incident investigation, governance, model evaluation, and continuous improvement. It also creates a valuable feedback mechanism for development teams, particularly when human operators repeatedly reject or modify specific AI recommendations. Such patterns may indicate weaknesses in the model, missing operational context, or response rules that do not reflect real-world OT conditions.

Human overrides should therefore be treated as useful operational feedback rather than simple exceptions. Analysing these decisions can help identify recurring issues in AI recommendations, highlight missing context, and support the continuous improvement of AI-assisted response processes.

Best Practice 9: Design for Rollback and Fail-Safe Behaviour

Automated response mechanisms should be reversible wherever technically possible so that unexpected consequences can be contained quickly. If an automated security action affects an industrial process in an unintended way, operators should have a clearly defined mechanism for restoring the previous state and recovering normal operations.

Fail-safe behaviour is equally important. When the AI lacks sufficient information, loses access to required contextual data, or cannot reliably assess the consequences of an action, the appropriate response may be to reduce its level of autonomy rather than proceed with a high-impact decision. In these situations, uncertainty should act as a constraint on automation.

This leads to a useful design principle for AI-driven OT security: the less the system understands about the environment and the likely consequences of its actions, the more likely the decision should be escalated to a human operator. This approach helps ensure that automation supports resilience without introducing unnecessary operational risk.

 

Best Practice 10: Test AI Response Before Production Deployment

AI-driven response should be evaluated in controlled, simulated, or representative OT environments before it is trusted with production systems. Testing should assess not only whether the AI can correctly identify malicious activity, but also whether the recommended or automated response produces the intended operational outcome without introducing unnecessary risk.

A comprehensive evaluation should include scenarios involving correct detections, false positives, unusual but legitimate operational behaviour, missing telemetry, conflicting contextual information, and failures in supporting systems. These scenarios help determine how the AI behaves when the environment is incomplete, ambiguous, or different from expected conditions.

Testing should also examine how human operators interact with AI-generated recommendations, including whether they understand the reasoning, accept or reject suggested actions, and respond appropriately under pressure. The objective is to evaluate the complete human-AI response process, rather than focusing only on model accuracy.

AICOT Relevance: Designing AI That Knows When Not to Act

Research on trustworthy AI and OT security points to an important consideration for AICOT: effective AI-driven cyber defence is not only about improving the speed or accuracy of threat detection, but also about ensuring that automation is introduced responsibly and within clearly defined boundaries.

For AICOT, these principles provide an important reference point for considering how AI can support OT cybersecurity while respecting the safety, reliability, and operational requirements of industrial environments. Factors such as human oversight, explainability, operational impact, asset criticality, and clearly defined authority all contribute to a more controlled approach to AI-assisted security.

Rather than treating full autonomy as the default objective, AICOT can contribute to the broader industry discussion around responsible and context-aware use of AI in OT cybersecurity, where automation supports human decision-making without removing the safeguards required in critical environments.

Conclusion

AI has significant potential to improve the speed and effectiveness of OT cybersecurity, but autonomous response must be designed carefully.

Research and current guidance point toward several recurring principles: human responsibilities should be clearly defined, automated actions should operate within established boundaries, AI recommendations should be explainable, operational consequences should be considered separately from threat confidence, and high-impact actions should include appropriate human oversight.

For AICOT, these principles provide an important reference point for the responsible use of AI in OT cybersecurity.

The goal should be to combine the speed and analytical capabilities of AI with OT context, appropriate safeguards, controlled authority, and human operational knowledge.

For initiatives such as AICOT, progressive autonomy offers a useful model for thinking about how AI capabilities can be introduced while maintaining appropriate safeguards and human oversight.

AICOT — AI-Driven Cyber Defense for Operational Technology Environments in Critical Infrastructure.

Led by Logstail, the project builds on expertise in SIEM, data analytics, and security operations while extending these capabilities to the specific requirements and constraints of OT environments.

Follow the AICOT project on LinkedIn and X for project updates and practical insights into AI-driven OT cybersecurity.

AICOT is funded by the European Union’s Digital Europe Programme under Grant Agreement No. 101249826. Views expressed are those of the authors and do not necessarily represent the European Union or the European Cybersecurity Competence Centre.

Contact Our Experts  or Sign Up for Free