#Germany #Berlin #Sovereign AI #Haystack AI #NVIDIA infrastructure

deepset's Innovative Reference Architecture for Sovereign AI Agents

On October 21, 2026, deepset, renowned for its open-source Haystack AI orchestration framework, announced a groundbreaking reference architecture designed for self-improving AI agents. This architecture is particularly distinctive as it operates on NVIDIA infrastructure and is crafted specifically for organizations seeking to maintain control over sensitive data and workloads in securely managed environments.

This new architecture marks an evolution in AI design, enabling self-improving agents that can function within air-gapped settings, ensuring that sensitive information remains within the organization's infrastructure without the necessity of external API interactions. This capability allows businesses to adapt their AI systems continuously, enhancing performance without compromising data security.

Collaborative Innovation with NVIDIA

The launch encompasses substantial collaboration between deepset and NVIDIA, integrating open models and software to refine how AI agents operate. At the NVIDIA GTC Berlin event, deepset demonstrated the architecture using a specialized cybersecurity agent that identifies vulnerabilities in confidential source code. This agent not only evaluates the existing vulnerabilities but also learns from its previous analyses, improving its performance through a robust feedback loop.

In essence, the architecture incorporates two Haystack agents working in tandem. The first agent focuses on task execution, specifically in the GTC demonstration, analyzing confidential code and vulnerability reports within a localized control system. The second agent functions as a quality control mechanism, analyzing the outcomes from the first agent's operation and proposing necessary adjustments to augment its performance. These proposed adjustments undergo evaluation against predefined objectives, ensuring that upgrades are systematically managed with adequate audit trails and rollback options.

Components of the Architecture

The reference architecture is constructed from four principal components:
1. Open Models: These provide foundational reasoning capabilities.
2. Modular Orchestration Layer: This layer connects these models to the organization's specific data and tools, enhancing adaptability.
3. Optimization Environment: Here, ongoing assessments of system improvements can occur.
4. Security Controls: These controls govern the actions and accesses of the agents, ensuring regulatory compliance and data integrity.

Utilizing NVIDIA's Nemotron open models, which are offered through NVIDIA NIM microservices, and NVIDIA's NeMo Gym optimization, the architecture showcases an array of capabilities that market leaders can rely upon. Notably, the framework employs NVIDIA NemoClaw to enforce policy-based control over files, networks, and tools used during agent execution.

Milos Rusic, CEO of deepset, remarked on the significance of powerful open models in establishing sovereign AI. “However,” he noted, “to fully unlock their potential, it is essential to engineer a complete system around them.” This sentiment encapsulates the philosophy underpinning the new architecture, enabling organizations not only to deploy autonomous agents but also to enhance them continually with user oversight.

Purpose and Implementation

For industries governed by stringent data regulations, this new architecture is a game changer. Organizations are no longer limited to the mere execution of models locally; they can oversee every layer, ensuring absolute control over what data agents utilize, which tools they can access, and the infrastructure supporting these operations.

As a hallmark of flexibility, deepset's architecture empowers organizations to adjust or swap components of the system as their needs evolve within the ever-changing technological landscape. This adaptability minimizes reliance on any singular model provider or closed application framework, a crucial aspect for various sectors.

Explore and Learn More

To further showcase the potential of this architecture, deepset and NVIDIA are scheduled to present their innovative work during the panel titled "Evolve Within Bounds: Secure, Self-Improving Agents for On-Prem AI" at NVIDIA GTC Berlin on October 22. Attendees can find deepset at booth #3071, where the company will provide insights and demonstrations of this revolutionary architecture.

For those eager to explore developing with this framework, a supporting repository is accessible on deepset's GitHub page, complete with sample code designed to guide developers through the adaptation of the reference architecture. Additionally, a technical walkthrough detailing a Haystack agent's utilization of local models for trace analysis and improvement recommendations is available.

In summation, deepset’s unveiling of its reference architecture signals a transformative step for organizations pursuing self-improving AI agents capable of navigating and evolving within securely managed environments while adhering to necessary regulations. Through this advancement, the combination of open-source solutions and controlled infrastructures exemplifies a forward-thinking approach to AI agent orchestration, reinforcing deepset's pivotal role in the AI landscape.