Skip to content

Apr 8, 2025 · Presented at COLM 2026

SkillFlow: Scalable and Efficient Agent Skill Retrieval System

Fangzhou Li*, Pagkratios Tagkopoulos*, Ilias Tagkopoulos(* equal contribution)

Pagkratios Tagkopoulos beside the SkillFlow poster at COLM 2026
With the SkillFlow poster at COLM, October 2026.

Summary

SkillFlow is a retrieval pipeline that lets an agent find the right skill in a very large library, about 36,000 community-contributed skill definitions indexed from GitHub, by narrowing in four stages: dense retrieval, two rounds of cross-encoder reranking, and a final LLM selection.

On SkillsBench, retrieval lifted task success from 9.2% to 16.4% Pass@1, reaching 84% of what an oracle with the perfect skill achieves. On Terminal-Bench, agents used retrieved skills 70% of the time with no improvement. The paper concludes that retrieval helps when the library contains good, executable skills for the domain.

Presented at

  • 1st Agent Skills Workshop, ACM CAIS 2026, poster, May 26, 2026 (Presented by Fangzhou Li)
  • Conference on Language Modeling (COLM) 2026, poster, Oct 2026