Success Factors for Implementing a Publicly Accessible AI Chatbot

Product-Oriented Phase Model

Three distinct phases can be identified in the development of an AI chatbot. Each phase has its own specific focus areas, but they also share many common issues that must be addressed accordingly. While costs and resource requirements vary by phase, they remain high even during the further development and operational phases.

The following diagram provides an overview.

The individual phases are examined in greater detail below.

PHASE 1 – Evaluation, Product Decision & Architecture

Chatbot use cases vary greatly: customer support, product advice, knowledge bases, rule-based, or even transactional use cases differ in complexity, required knowledge, technical implementation, and testing approach. Use cases such as investment advice or credit decisions are particularly complex and can lead to acceptance issues; they also require non-discrimination as well as transparent and justifiable recommendations. The potential benefits of these use cases vary greatly from bank to bank and should be carefully assessed before launch. Depending on the situation, it may make sense to allow a chat session with an AI chatbot to be handed over to a human. The number of supported languages also affects the implementation effort, and the testing effort increases almost proportionally. An AI chatbot is ideally implemented in phases: piloting, real-world user experience, and deliberate focus are more important than a broad range of features right from the start—less is often more.

Prudent management, focus, and phased implementation lead to success

In addition to AI technology, key aspects include business and process knowledge, use-case evaluation, data provision, security, legal/compliance, infrastructure decisions, system and process integration, training, change management, operational processes, and often provider management. AI projects in banks benefit equally from the expertise of generalists and specialists. Since much of this is uncharted territory for banks and there is a discrepancy between what they want to do, what they are allowed to do, and what they are capable of doing, external support, conservative planning, and sound risk management prove to be key success factors.

When it comes to sourcing, it’s worth conducting a detailed RFP with expert support and requesting comparable references. The focus should be on both the product features and the provider—specifically their expertise and experience. In the fast-paced world of AI, banks’ requirements may change, while providers must continuously invest significant effort to keep their products up to date with the latest technology. Switching providers may therefore well be a possible scenario down the line. One should prepare for a provider change, both contractually and, for example, by incorporating design-in-failures in the architecture.

Infrastructure & Data – the Foundation of Every AI Chatbot

Many banks rely on public cloud infrastructures for AI use cases. Legal, compliance, and security assessments for cloud-based AI solutions can be complex and time-consuming, and the costs of cloud infrastructures are often underestimated. A key success factor is an easy-to-use foundation, ideally focused on a single provider. It’s worth consciously addressing the risk of vendor lock-in and considering an abstraction layer—at least for accessing LLMs—since model changes will be required on a regular basis. The best way to gain initial experience is with simple use cases.

The quality of an AI chatbot depends largely on the data set. In banks, the required data is often scattered, of insufficient quality, or difficult to use. RAG systems reveal contradictions, duplications, and errors, as responses and references make these weaknesses visible. It is advantageous to address data cleansing and preparation promptly. Active data governance is crucial; the designated data owners must be held accountable on an ongoing basis. A key success factor is building a suitable data platform for AI and analytics use cases. Modern technologies such as MCP for accessing data and services are best utilized from the very beginning. Even after go-live, continuous content maintenance remains essential.

PHASE 2 – Initial Implementation

Many AI use cases assume that a human will validate the results. With a public chatbot, this is not possible. Users rely on an understandable, robust, and secure interaction, which significantly increases the requirements for reliability, response behavior, and error prevention. The primary risk is factually incorrect answers (hallucinations); this is addressed through grounding with source references, topic scoping, and output guardrails. The chatbot should be “dummy-proof” and thoroughly tested. Transparent user information, consent and opt-out mechanisms, and—where appropriate—the creation or maintenance of alternative channels are important.

Even small changes can significantly alter the behavior of AI applications: prompts, model parameters, and RAG and embedding configurations significantly influence the responses. The size and number of prompts also directly affect token consumption and response speed. Prompts and the entire configuration are therefore best treated as code and versioned. Different models, prompts, and configurations should be tested automatically and made comparable. Comprehensive monitoring is essential to be able to track and document application behavior at all times. In addition, it is recommended to establish a defined incident response process as well as a “kill switch”—ideally with a rollback to a verified configuration—to enable a rapid response in the event of malfunctions or security incidents.

Security efforts should not be underestimated

Public AI applications pose specific security risks, including volume attacks and prompt injection. Several measures have proven effective in mitigating these risks: penetration tests conducted by external experts and adversarial testing uncover vulnerabilities, while technical safeguards such as a WAF with DDoS protection, rate limiting, and other bot protection features fend off attacks. The LLM’s inputs and outputs can also be secured through PII masking and DLP.

PHASE 3 – Operation & Further Development

LLMs evolve rapidly and are frequently replaced by successor models; a change may be necessary almost annually. In Switzerland, depending on the cloud provider, models may become available later than in the EU or not at all. Successor models are not necessarily better for a specific use case, and locally operated LLMs currently do not yet match the performance of cloud-based LLMs, depending on the use case. After a model change, prompt and configuration adjustments are always required and should be thoroughly tested. In addition, security and compliance assessments may restrict or delay model usage. Therefore, established AI governance—including model approvals, risk classification, controls, and clearly defined roles and responsibilities—is critical to success. Of particular importance here are the revDSG, FINMA requirements (including outsourcing and cloud guidelines), and, where applicable, the EU AI Act.

Conclusion

Implementing a publicly accessible AI chatbot for a Swiss bank is a complex, security-critical, and operationally demanding undertaking. Starting with simple use cases, phased implementation, and a realistic approach to uncertainties, risks, and vendor/product limitations contribute significantly to success. Content and data maintenance, model changes and retesting, monitoring of performance metrics such as resolution rate, customer satisfaction, and cost per conversation, as well as regular security and compliance reviews, remain ongoing tasks. Banks are therefore advised to transition from a project-oriented to a product-oriented organization: a clearly designated product owner, a permanent operations team, and secured budgets enable continuous development and reliable operations.