Many observers believe that this year should be a year of rapid development for Generative Artificial Intelligence (GenAI) in enterprises. One way this is happening is through Retrieval-Augmented Generation (RAG), a method that connects large language models with databases containing specific domain content, such as company documents.

However, RAG is a nascent technology and has its limitations.

Therefore, researchers from Amazon AWS propose in their latest paper to establish a series of benchmarks specifically designed to test RAG's ability to answer domain-specific content questions.

"Our approach is an automated, cost-effective, interpretable, and robust strategy for selecting the best components for RAG systems," wrote Gauthier Guinet, the first author of the paper, and his team in the preprint paper titled "Using Task-Specific Exams to Automatically Evaluate Retrieval-Augmented Language Models."

The paper will be presented at the 41st International Conference on Machine Learning in Vienna from July 21 to 27.