When the Chatbot Goes Off the Rails: A Governance Wake-Up Call for HR

Hello H.A.I.R. Community,

The headline was almost inevitable: "Eurostar’s Chatbot Goes Off the Rails." While this is the kind of pun I’d be proud of coming up with, the underlying story uncovered by security researchers at Pen Test Partners is a serious warning for any business leader overseeing digital transformation.

For those who missed the report, the findings were stark. Eurostar’s customer-facing AI chatbot, designed to assist travellers, contained significant security vulnerabilities. While the interface appeared to have safeguards to stop malicious use, the backend infrastructure was not consistently enforcing them. Hackers could bypass these checks to inject "prompts" - effectively tricking the AI into revealing internal instructions or rendering malicious code to other users.

You might be tempted to view this as a consumer travel issue, or perhaps a problem for the IT department to solve. However, for HR leaders (particularly those in Talent Acquisition) this incident highlights a critical vulnerability in our own backyard.

As we rush to deploy conversational AI to handle candidate/employee queries and screen applications, we are effectively widening the "attack surface" of our organisations. The Eurostar incident is not just a technical failure; it is a case study in why HR leaders must now become practitioners of robust AI governance.

Recruitment: The Open Front Door

In cybersecurity terms, a "front door" is any point where an external user interacts with your internal systems. Historically, HR’s front door was heavy and manual: a CV sent via email or a form on a static careers page. It was slow, but relatively easy to secure.

Today, AI-driven ATS and recruitment chatbots have replaced that heavy door with a digital concierge that is always on, always listening, and eager to help. These bots sit on public-facing career sites, inviting interaction from anyone on the internet.

The parallel with Eurostar is exact. Just as Eurostar’s bot was designed to help strangers book trains, recruitment bots are designed to help strangers apply for jobs. This openness makes them a prime target.

If a recruitment chatbot suffers from the same flaws found in the Eurostar system, the consequences could be severe:

1. Data Exfiltration: A "bad actor" could use prompt injection techniques (crafting specific inputs to confuse the AI) to trick the bot into revealing data about other candidates, such as names, contact details, or salary expectations.

2. Reputation Hijacking: The Eurostar researchers found they could inject "HTML" code into the chat. In a recruitment context, an attacker could theoretically make your chatbot display offensive content, fake job offers, or phishing links to genuine candidates, all under your brand’s banner.

3. Internal Access: Many recruitment bots are now integrated with Applicant Tracking Systems (ATS) to schedule interviews or check status updates. If the bot is compromised, it could act as a gateway into your wider internal network.

Recruitment is no longer just about branding and candidate experience; it is now a frontline cybersecurity function.

The Governance Gap: "Surface-Level" Security

The most distinct failure in the Eurostar case was the reliance on what we might call "surface-level" security.

The researchers noted that while the user interface (the part the customer sees) tried to filter out bad requests, the server (the brain of the operation) often accepted them without question.

In the HR technology market, we see a similar pattern. Vendors often demo tools with polished interfaces that seem secure. They show you how the bot politely refuses to answer inappropriate questions. However, true AI governance requires us to look deeper than the demo.

If the security checks only exist on the "client-side" (the user's browser), they are effectively useless against a determined attacker. It is akin to having a security guard who checks IDs at the entrance but leaves the side window wide open.

The Governance Question HR Must Ask: When evaluating a new recruitment tool, we must move beyond functional questions ("Can it schedule interviews?") to structural governance questions:

• "Where are the guardrails enforced?"

• "Does the backend API validate every single message, or just the first one?"

• "If I strip away the user interface, is the model itself secure?"

Why Process is Part of Security

Perhaps the most damaging aspect of the Eurostar story was not the code, but the culture.

When the security researchers first tried to report the vulnerability, they were met with silence. Weeks passed without a response. When they finally escalated the issue, Eurostar’s initial reaction was suspicion, with the researchers noting that the company suggested they were attempting blackmail.

This is a failure of governance.

Effective AI governance is not just about preventing valid attacks; it is about how an organisation listens to signals of risk. In an HR context, this is vital. Your candidates and employees are often the first people to notice when an AI is behaving strangely - whether it is showing bias, hallucinating facts, or acting erratically.

If your governance framework does not include a clear, psychological safe channel for reporting these issues, you are flying blind. A defensive reaction to a vulnerability report, as seen in the Eurostar case, turns a fixable technical bug into a public relations crisis.

Integrating AI Governance into HR Strategy

So, how do we move forward? We need to stop treating AI security as a feature we buy and start treating it as a discipline we manage. Here are three practical steps for HR leaders to tighten governance around their AI tools.

1. Demand "Red Teaming" Evidence

"Red Teaming" is the practice of ethically hacking a system to find its flaws before the bad guys do. The researchers who found the Eurostar flaws were essentially Red Teaming the system. Don't just ask your vendor if their tool is secure. Ask to see their latest Red Team report. If they haven't paid independent experts to try and break their chatbot, they are likely not ready to handle your candidate data.

2. The Principle of Least Privilege

The Eurostar bot was connected to internal systems that allowed it to be manipulated. In recruitment, we must apply the principle of "Least Privilege." Does your chatbot really need access to the entire candidate database to answer a question about company culture? Probably not. Ensure the AI only has access to the specific data it needs to do its immediate job. This limits the potential damage if the system is compromised.

3. Establish a "vulnerability intake" process

This is a core part of modern governance. If a candidate notices your chatbot is spouting gibberish or revealing private info, where do they send that screenshot? If it goes to a general "info@" inbox, it will be lost. HR operations teams need a specific protocol for AI incidents, ensuring they are flagged to IT security immediately and not dismissed as user error.

Final Thoughts

The Eurostar incident serves as a low-stakes warning with high-stakes lessons. No trains derailed, and the vulnerabilities were fixed before they could be maliciously exploited.

However, for HR leaders, it invalidates the assumption that we can simply "plug and play" with AI tools. As the guardians of people and data, we are introducing complex, public-facing software into our recruitment and management stacks.

We must accept that recruitment is now a "front door" for cyber risk. By adopting rigorous AI governance - asking the hard technical questions, demanding proof of security testing, and building processes that welcome feedback - we can ensure that door remains open to talent, but firmly closed to threats.

The State of AI in TA 2026 - Part 2!

Michael Blakley and Equitas are hosting me, Khyati Sundaram, Jamie Betts and Dan Gallagher for part 2 of our 'explosive' no-holds-barred conversation on AI in TA.

The response to Part 1 was incredible. Many of you even called it the best AI in TA session of 2025!!

Want to see what all the fuss was about? Register here.

Until next time,

Martyn

H.A.I.R. (AI in HR)

Putting the AI in HR. Safely.

Here's how H.A.I.R. can help you put the AI in HR:

1. H.A.I.R. Newsletter: get authoritative, pragmatic, and highly valuable insights on AI in HR directly to your inbox. Subscribe now.

2. AI Governance QuickScore Assessment: understand your organisation's AI governance maturity in minutes and identify key areas for improvement. Take your QuickScore here.

3. Advisory Services: implement robust AI Governance, Risk, and Compliance (GRC) with programmes designed for HR and Talent Acquisition leaders. Contact us for a consultation.

4. Measure Your Team's AI Readiness with genAssess: stop guessing and start measuring your team's practical AI application skills. Discover genAssess.