#USA #Redwood City #Zilliz #Milvus #VDBBench

Zilliz Launches VDBBench 1.0: A New Benchmarking Standard for Vector Databases

Zilliz has taken a significant leap in the world of database technology with the release of VDBBench 1.0, an innovative open-source benchmarking tool tailored specifically for vector databases. This pivotal platform aims to close the vast discrepancy that exists between benchmark expectations and real-world performance.

The advent of VDBBench comes at a crucial time, as the vector database market is witnessing immense growth and transformation since 2023. Organizations are often caught in a bind, making hefty investments based on inadequate benchmarks that fail to account for the complexities of actual production environments. James Luan, the VP of Engineering at Zilliz, highlighted this issue, stating that the existing benchmarking methodologies are not aligned with the current landscape of vector databases, which is why VDBBench 1.0 is a game-changer.

Features of VDBBench 1.0

Advanced Filtering Analysis

One of the standout features of VDBBench 1.0 is its advanced filtering analysis. This feature systematically tests metadata filtering across varying selectivity levels, ranging from 50% to 99.9%. This is pivotal because it uncovers the "hidden performance killers" that can adversely affect query speeds and recall accuracy in real deployment scenarios.

Streaming Read/Write Testing

Another key advancement is the inclusion of streaming read/write testing, which simulates the continuous data ingestion of queries. This reflects the conditions under which many databases falter, even when traditional benchmarks look promising. By visualizing performance in these scenarios, VDBBench provides a more authentic assessment of how systems will perform under stress.

Modern Datasets

The platform is equipped to use cutting-edge vectors sourced from state-of-the-art embedding models such as OpenAI and Cohere. With dimensions varying from 768 to 1,536, this allows organizations to gauge performance based on current AI workloads, which are becoming increasingly prevalent in various applications.

Custom Dataset Support

Understanding that every organization has unique requirements, VDBBench allows users to benchmark against their own production data and embeddings. This level of customization ensures that testing is relevant to specific industry needs.

Focus on Realistic Metrics

VDBBench prioritizes critical performance metrics such as P95/P99 tail latency and sustainable throughput under load. These insights are essential for realistic capacity planning and resource allocation. Instead of becoming bogged down by traditional metrics, organizations can now align their benchmarks with how databases are truly used in the field.

Interactive Visualizations

To enhance user experience, VDBBench packs a revamped dashboard with interactive visualizations. This feature enables engineers to swiftly identify performance gaps, fostering quicker response times and better optimization strategies.

Open Source Availability

As part of Zilliz’s commitment to transparency and community collaboration, VDBBench 1.0 is fully open-source and accessible via GitHub. Organizations looking to escape the trap of misleading benchmarks can get started with this platform immediately. Complete documentation and community support are also provided to assist in customizing the benchmark process further.

Access

About Zilliz

Founded in California, Zilliz is known for its innovative approach to database technologies. The company aims to help organizations effectively harness the potential of unstructured data and expedite the development of AI applications. With backing from notable investors, including Aramco's Prosperity7 Ventures and Pavilion Capital of Temasek, Zilliz continues to drive advancements in the data landscape. For more information, visit www.zilliz.com.