Reading Lists

Info

working in progress. Any suggestions are welcome.

I've been compiling a comprehensive list of papers and resources that have been instrumental in my research journey. This collection is designed to serve as a valuable starting point for those interested in delving into the field of deep model fusion. If you have any suggestions for papers to add, please feel free to raise an issue or submit a pull request.

Note

Meaning of the symbols in the list:

• Highly recommended
• LLaMA model-related or Mistral-related work
• Code available on GitHub
• models or datasets available on Hugging Face

Survey Papers

• E. Yang et al., “Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities.” arXiv, Aug. 14, 2024.
• W. Li, Y. Peng, M. Zhang, L. Ding, H. Hu, and L. Shen, “Deep Model Fusion: A Survey.” arXiv, Sep. 27, 2023. doi: 10.48550/arXiv.2309.15698.
• H. Zheng et al., “Learn From Model Beyond Fine-Tuning: A Survey.” arXiv, Oct. 12, 2023.

Model Ensemble

• Liu T Y, Soatto S. Tangent Model Composition for Ensembling and Continual Fine-tuning. arXiv, 2023.
• Wan F, Yang Z, Zhong L, et al. FuseChat: Knowledge Fusion of Chat Models. arXiv, 2024.

Model Merging

Mode Connectivity

Mode connectivity is such an important concept in model merging that it deserves its own page.

Weight Interpolation

• G. Ilharco et al., “Editing Models with Task Arithmetic,” Mar. 31, 2023, arXiv: arXiv:2212.04089. doi: 10.48550/arXiv.2212.04089.
• Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard, “Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models,” May 30, 2023, arXiv: arXiv:2305.12827. doi: 10.48550/arXiv.2305.12827.
• P. Yadav, D. Tam, L. Choshen, C. Raffel, and M. Bansal, “Resolving Interference When Merging Models,” Jun. 02, 2023, arXiv: arXiv:2306.01708. Accessed: Jun. 12, 2023. [Online]. Available: http://arxiv.org/abs/2306.01708
• E. Yang et al., “AdaMerging: Adaptive Model Merging for Multi-Task Learning,” ICLR 2024, arXiv: arXiv:2310.02575. doi: 10.48550/arXiv.2310.02575.
• L. Yu, B. Yu, H. Yu, F. Huang, and Y. Li, “Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch,” Nov. 06, 2023, arXiv: arXiv:2311.03099. Accessed: Nov. 07, 2023. [Online]. Available: http://arxiv.org/abs/2311.03099

Alignment-based Methods

• S. K. Ainsworth, J. Hayase, and S. Srinivasa, “Git Re-Basin: Merging Models modulo Permutation Symmetries,” ICLR 2023. Available: http://arxiv.org/abs/2209.04836
• George Stoica, Daniel Bolya, Jakob Bjorner, Taylor Hearn, and Judy Hoffman, “ZipIt! Merging Models from Different Tasks without Training,” May 04, 2023, arXiv: arXiv:2305.03053. Accessed: May 06, 2023. [Online]. Available: http://arxiv.org/abs/2305.03053

Subspace-based Methods

• Tang A, Shen L, Luo Y, et al. Concrete subspace learning based interference elimination for multi-task model fusion. arXiv preprint arXiv:2312.06173, 2023.
• X. Yi, S. Zheng, L. Wang, X. Wang, and L. He, “A safety realignment framework via subspace-oriented model fusion for large language models.” arXiv, May 14, 2024. doi: 10.48550/arXiv.2405.09055.
• Wang K, Dimitriadis N, Ortiz-Jimenez G, et al. Localizing Task Information for Improved Model Merging and Compression. arXiv preprint arXiv:2405.07813, 2024.

Model Mixing

• C. Chen et al., “Model Composition for Multimodal Large Language Models.” arXiv, Feb. 20, 2024. doi: 10.48550/arXiv.2402.12750.
• A. Tang, L. Shen, Y. Luo, N. Yin, L. Zhang, and D. Tao, “Merging Multi-Task Models via Weight-Ensembling Mixture of Experts,” Feb. 01, 2024, arXiv: arXiv:2402.00433. doi: 10.48550/arXiv.2402.00433.
• Zhenyi Lu et al., "Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging" 10.48550/arXiv.2406.15479
• Tang A, Shen L, Luo Y, et al. SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models. arXiv, 2024.

Libraries and Tools

Fine-tuning, Preparing models for fusion

• PyTorch Classification: A PyTorch library for training/fine-tuning models (CNN, ViT, CLIP) on image classification tasks
• LLaMA Factory: A PyTorch library for fine-tuning LLMs

Model Fusion

• FusionBench: A Comprehensive Benchmark of Deep Model Fusion.
• MergeKit: A PyTorch library for merging large language models.

Other Applications of Model Fusion

Applications in Reinforcement Learning (RL)

• (Survey Paper) Song Y, Suganthan P N, Pedrycz W, et al. Ensemble reinforcement learning: A survey. Applied Soft Computing, 2023.
• Lee K, Laskin M, Srinivas A, et al. “Sunrise: A simple unified framework for ensemble learning in deep reinforcement learning", ICML, 2021.
• Ren J, Li Y, Ding Z, et al. “Probabilistic mixture-of-experts for efficient deep reinforcement learning". arXiv:2104.09122, 2021.
• Celik O, Taranovic A, Neumann G. “Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts". arXiv preprint arXiv:2403.06966, 2024.