Beyond RAG: Enterprise AI Agents Navigate Real-World Implementation Challenges (Panel)

Jun 8, 2025

Jun 8, 2025

The Brief: Agents are now in the features list of any IT solution, driven by demand for enhancing traditionally slow and uninspiring processes. Let's be candid—organizations are filled with tedious workflows, and introducing intelligence into these areas can transform the employee experience. This track offers a chance to explore serious, operational projects and tools that will inspire you to develop your own solutions.

Session: Track 2 - Automation
Date: June 4, 2025
Moderator: Raphaël Briner, Panoramai Curator

Key Takeaway: While agent frameworks promise transformational capabilities, Swiss AI leaders emphasize pragmatic deployment strategies focused on immediate business value over technological sophistication. The shift from proof-of-concepts to production systems reveals critical gaps in performance, data quality, and regulatory compliance that demand strategic prioritization.

The RAG Reality Check: From Startup Differentiation to Enterprise Deployment

The panel opened with a stark assessment of the current Retrieval Augmented Generation landscape. Simone Abbiati, Head of Training and Education at Squirro, addressed the fundamental challenge facing RAG startups: "So again, first struggle that we have is how do we differentiate what we do with all the rest? Because now we see the rise, from the rise of ChatGPT on, a lot of startups are emerging more or less presenting the same type or something that sounds the same."

Squirro's experience with highly regulated clients, including the European Central Bank, illuminates the complex reality of enterprise AI deployment. The 13-year-old Zurich-based company positions itself as "an enterprise ready platform that is able to ingest the databases from enterprises and that add an LLM layer to finally have a chatbot." However, Abbiati emphasized that successful implementations require far more than technical excellence, particularly regarding compliance: "So one other big topic that is not always mentioned is guardrailing. We need to be sure that both the input and the output is validated in terms of tone of the company and also regulations."

The knowledge graph integration approach represents Squirro's strategic response to persistent hallucination challenges. Abbiati explained the technical architecture: "Knowledge graph that includes taxonomies, ontologies, is a type of technology that includes networks of edges and nodes. So you could schematize a field of knowledge. In this way, you could prompt the LLM not just to retrieve data from the enterprise data, but also add, let's say, a contextual understanding to the LLM."

Agent Orchestration: The Performance vs. Promise Paradox

Clément Robin, AI and Innovation Lead at Mantu, provided insight from a $1 billion consultancy perspective across 60+ countries. His practical assessment of agent deployment challenges resonated throughout the discussion: "We already have so much difficulties to have just one reliable agent. Why play with other multiple agent coordination?"

Mantu's internal AI Olympics, involving 1,400 participants globally, produced a standout success story from Brazil. Their specialized support agents achieved dramatic efficiency gains, as Robin detailed: "So let's talk about the Support Space Search Agents, which are agents for internal support. So it's a lot of tickets, as you can see, and the running cost, it's not in the slide, but the running cost, honestly, it's ridiculous, something like 200 euros per month." The impact was transformative: "you could go from six business days to one minute to solve tickets."

However, Robin's reference to industry-wide performance challenges underscored a critical barrier: "For example, LangChain, so you guys know it's one of the main frameworks to do currently AI. They released a survey like six months ago, where the question was, why don't you put more agents in production? Number one answer, maybe half of the answer was like performance issues."

Conversational Analytics: The Dashboard Disruption

Timon Zimmerman, CEO of Magemetrics, articulated a compelling vision for enterprise software transformation. His central thesis challenges fundamental assumptions about user interfaces: "And the reality is that users don't want dashboards, they want answers."

Zimmerman's analysis of B2B SaaS competitive dynamics reveals a stark timeline: "we believe that most B2B SaaS, as you can think about your accounting software, CRM, ERP, have about 18 months to a year and a half to integrate AI agents, so advanced AI capabilities, or become obsolete and risk going under."

The strategic value proposition extends beyond user satisfaction to business intelligence. Zimmerman explained how conversational interfaces unlock new insights: "the cool thing about having your users using a conversation instead of navigating a dashboard is we can capture and analyze the conversations to surface product gaps, user needs, and targeted upsell opportunities potentially. Something that you cannot do by just tracking dashboard usage."

The competitive threat stems from user behavior patterns that traditional SaaS cannot address: "And when they cannot get answers, they export to Excel, so they essentially leave the SaaS for a side quest. They flood support with tickets, asking custom reports, custom visualizations, custom anything."

Open Source vs. Enterprise: The Motley Crew Perspective

Egor Kraev, former Wise machine learning leader and founder of Motley AI, offered a contrarian perspective on agent framework proliferation. His assessment was direct: "what I think is happening is that we are very near peak agent hype and quite possibly past it."

Kraev's analysis of current agent definitions revealed the marketing inflation around the term: "Really anything that's any kind of piece of software or automated workflow that happens to call a large language model somewhere inside it can be marketed as an agent. It often is."

He elaborated on the technical reality: "If you get to be slightly more technical, then maybe it's something that calls an LLM and then has a loop inside it. So the LLM does ask the outside world to do something, the results back into the LLM. And so you have this kind of a feedback loop, and that's called an agent. And really, that's all there is to it."

The transition from question-answering to storytelling represents a fundamental shift in AI application philosophy. Kraev referenced Robert Sheckley's work to illustrate current limitations: "Because to ask the right question, you must already know most of the answer. And so that's what we're trying to do at Motley AI, is to go from question answering to storytelling."

His philosophy on technology deployment emphasized pragmatism: "In fact, what I keep saying is you should always use the dumbest thing that works. If you can get things done without LLM, don't use it. If you can do without multi-agent, don't use multi-agent. But sometimes you really need them, and then you use them."

MCP Protocol: Standardization vs. Enterprise Constraints

The Model Control Protocol discussion revealed divergent perspectives on standardization benefits. Robin emphasized development efficiency gains: "The thing with MCPs is that it's going to simplify a lot our life. Because before, when we had to connect an LLM to any other software, API, everything, we have to build the connector. Now we have one single solution, so it's going to be fastest for us to do development that way."

However, Kraev expressed skepticism about enterprise applicability: "At the same time, I think MCP will matter a lot more for the consumer space than for the enterprise space. Because for the consumer space, there's much more demand for things that can do anything, or the way you have to discover your tools as you go. Whereas in the enterprise, generally, you want your agent or your workflow to do, you want to allow it to do certain things, and you want to constrain exactly, constrain as much as you want what it can do."

Zimmerman's forward-looking perspective identified strategic opportunities beyond internal tool consumption: "I think in the future, MCP or any kind of interoperability agent-to-agent framework would allow for new revenue streams. Because it would allow for enterprise companies or any kind of companies essentially to make internal processes, internal intelligence available externally through an agent potentially."

A critical implementation barrier emerged from Kraev's technical assessment: "Just in case you think of providing a tool via MCP, please, please make sure that dynamic registration is supported. Because right now, the one thing crippling MCP adoption is that to do dynamic authentication, like I just need to click a button to allow my agent to access this tool. Each tool needs to pre-register with authentication provided by OAuth."

European AI Advantage: Regulation as Competitive Moat

Abbiati's insights revealed how European companies transform regulatory complexity into strategic advantage: "Because here, what I am witnessing is using those regulations as a force. So many different companies, including us, to be honest, are trying to, let's say, play with the judges themselves, because the regulations are so hard to keep up with."

The European Central Bank case study demonstrates this approach in practice. Beyond efficiency metrics, Abbiati highlighted qualitative improvements: "For example, we saw that with the European Central Bank, instead of just evaluating the time that we were able to save up, also the quality of the activities that the employees were able to do augmented a lot."

Customer requirements often reflect these regulatory pressures, as Abbiati noted: "Because other times we had customers asking not to use OpenAI's models because of the training data." This constraint creates natural barriers for competitors lacking European compliance expertise.

The human-AI collaboration imperative remains central to regulated deployments: "So again, here we need always to remember that the human must be in the loop or in his presence or their presence needs to be discussed because just automating everything and just follow that dream of just, again, automatizing everything may bring us to some other hurdles."

Startup Pragmatism vs. Innovation Pressure

The startup perspective emerged clearly through Abbiati's resource allocation challenges: "For us, it's not one of the priorities at the moment, meaning that, again, the market is asking so much about agents that it's more profitable at the moment to just follow that rush instead of orchestrating something more difficult, let's say."

This tension between innovation and delivery creates strategic dilemmas: "I would say it's also a bigger problem here. So I was discussing with another of the attendees before, Every week, every day, we have something new in the AI environment. So it's really, really hard to keep up with everything that gets published, even just in terms of research papers. So we also have the risk, again, as a startup, to just follow that hype meta of keeping up with all that gets published instead of actually delivering something that works."

Trust-building in regulated industries compounds these challenges: "Also because just what on his point is that depending on the industry calling and making the user and the customer trust in LLM, it's very, very, very hard, very hard. Because of again, guardrailing, privacy, data access control, access control lists, it's very hard, especially with banks or governments or even with healthcare providers."

Data Infrastructure: The Foundation Layer Challenge

Guillaume Beauverd and Julien Groselle addressed fundamental data quality challenges. Groselle opened with a stark industry statistic: "So 80% of the AI deployment are failing because of bad data. And this is something really important, because with a bad data set, it's really, really hard to build something trustable."

Beauverd's organizational transformation framework suggests moving beyond traditional IT-driven implementations: "And I feel that now AI is reshuffling the cart. It's changing the way we should approach AI deployments, technology deployment within enterprises. And for that matter, my point is, for the first time, it may be possible for business users to choose and pick what tools augments them, enable them."

Dignow's approach combines AI automation with human verification through community validation. Groselle explained their methodology: "So we are using AI agents to collect and validate data through public API, private API, depends on the customer of course, of course internet, but also MCP servers. And then we developed a Trust Core."

The impressive results demonstrate the hybrid approach's potential: "We, let's say, collected and verified 13,000 data points in one month with this community. So the model is very good." Their achievement of 99% accuracy on crypto datasets while maintaining scale represents a significant breakthrough in data quality assurance.

Both companies emphasized AI-native operations. Groselle noted that "if we are talking about the product, I think 90% of the company was built with AI. So every document we have internally are done with three different AI we use." Meanwhile, Beauverd acknowledged both opportunities and limitations: "There are things that AI does extremely well, but at the moment there are things that AI does less well than it did a couple of months ago. And so we are revising some of the aspects of what we had automated with AI in order to keep the quality or our expectations at least where they are."

Strategic Implications: Navigating the Agent Reality

The panel discussion reveals several critical insights for enterprise AI strategy:

Prioritize Business Value Over Technical Sophistication: Successful implementations focus on immediate, measurable business impact rather than showcasing advanced capabilities. Mantu's dramatic cost reduction and Squirro's ECB deployment demonstrate the importance of solving specific, well-defined problems.

Embrace Regulatory Complexity as Competitive Advantage: European companies can leverage deep compliance expertise and regulatory relationships to create sustainable competitive moats against global competitors lacking local knowledge.

Invest in Data Infrastructure Before Agent Deployment: The foundational importance of clean, trustworthy data cannot be overstated. Organizations must address data quality systematically before expecting reliable agent performance.

Plan for Human-AI Collaboration, Not Replacement: The most successful implementations augment human capabilities rather than attempting full automation. This approach proves particularly critical in regulated industries where human oversight remains mandatory.

Consider Domain-Specific Solutions Over Generic Platforms: The shift toward specialized, industry-focused agents suggests that vertical solutions will outperform horizontal platforms in delivering measurable business value.

The Panoramai discussion ultimately reveals an industry in transition from experimental proof-of-concepts to production-ready systems. While the promise of autonomous agent orchestration captures imagination, the reality demands pragmatic focus on specific use cases, robust data foundations, and regulatory compliance.

More on the panelists

Simone Abbiati, Head of Training and Education, Squirro

Timon Zimmermann, co-founder MageMetrics

Guillaume Beauverd, co-founder Candice AI

Julien Groselle, CTO of DigNow.io

Egor Kraev, founder Motley Crew AI

See the full program of the day