MergeKit

Open-source toolkit for merging multiple fine-tuned LLMs into unified, specialized models

AI Development FrameworksFree

MergeKit is an open-source library for merging large language models, enabling practitioners to combine multiple fine-tuned models into a single model that inherits capabilities from all merged models. Techniques like SLERP, TIES, DARE, and passthrough merging enable different approaches to combining model weights. Model merging has emerged as a powerful alternative to continual fine-tuning, sometimes producing models that outperform individual fine-tuned models on multiple tasks simultaneously. The AI research community and practitioners use MergeKit extensively—many top-ranked models on Hugging Face's Open LLM Leaderboard are merge-based. MergeKit requires no additional training compute, making it a cost-effective way to create specialized models.

Key Features

• ✓Multiple merge methods
• ✓No training required
• ✓Open-source
• ✓Hugging Face compatible
• ✓Large model support