Meta open-sourced Llama 2 together with Microsoft, this time in contrast to Llama 1 free not just for research but also for commercial use.
- Free for commercial use for businesses with less than 700 Mio monthly active users
- Models with 70B, 13B, and 7B parameters
- Llama-2-70B model is currently the strongest open-source LLM (Huggingface leaderboard), comparable to GPT-3.5-0301, noticeably stronger than Falcon, MPT, and Vicuna
- Not yet at GPT-3.5 level, mainly because of its weak coding abilities
- RLHF fine-tuned
- Source code on GitHub, weights available on Azure, AWS, and HuggingFace
- Llama 2 paper
- 4K token context window
- Trained on 2 trillion tokens with training costs of about $20M
- Knowledge cut-off Dec 2022
- Testing on https://www.llama2.ai
Just 4 days after this announcement, on July 22, 2023, StabilityAI released FreeWilly1 and FreeWilly2 which are fine-tuned models based on LLaMA65B and Llama-2-70B. These models took over the leadership on Hugging Face (Huggingface leaderboard). However, both models have no commercial license and are just intended for research.