SRAG: Skill Retrieval Augmentation for Agentic AI

Source Excerpt

Overview

Skill Retrieval Augmentation (SRA) is a paradigm where agents dynamically retrieve, incorporate, and apply relevant skills from large external skill corpora on demand, rather than enumerating all available skills in the context window.

Problem

In existing agent systems, the dominant strategy for incorporating skills is to explicitly enumerate available skills within the context window. This fails to scale: as skill corpora expand, context budgets are consumed rapidly, and agents become less accurate at identifying the right skill.

The SRA Approach

1. Retrieval: Agent queries a large skill corpus to find relevant skills for the current task
2. Incorporation: Retrieved skills are loaded into context on demand
3. Execution: Agent applies the retrieved skills to complete the task

SRA-Bench

The paper introduces SRA-Bench — the first benchmark for decomposed evaluation of the full SRA pipeline:

• 5,400 capability-intensive test instances
• 636 manually constructed gold skills
• 26,262 total skills (including web-collected distractors)
• Covers skill retrieval, skill incorporation, and end-task execution

Key Finding

Retrieval-based skill augmentation substantially improves agent performance. However, a fundamental gap exists in skill incorporation: current Large Language Model agents tend to load skills at similar rates regardless of whether a gold skill is retrieved or whether the task actually requires external capabilities. The bottleneck lies not only in retrieval but in the base model's ability to determine which skill to load and when.

Why It Matters

Directly relevant to Hermes Agent's skill system. As skills scale beyond dozens, the current approach of listing all skills in context becomes unsustainable. SRAG provides a research-backed path for scaling skill systems.

Actionability

HIGH — Research-backed approach to scaling skill systems. The SRA-Bench benchmark provides a measurable framework for evaluating skill retrieval quality.

Status

Published April 2026, CC BY-SA 4.0. Authors: Weihang Su, Jianming Long, Qingyao Ai, Yichen Tang, Changyue Wang, Yiteng Tu, Yiqun Liu.

Sources

• https://arxiv.org/abs/2604.24594
• @dair_ai, April 28, 2026