This is a pre-trained Score Entropy Discrete Diffusion (SEDD) model. Vocab size is 50256 and uses the GPT2 Tokenizer. Trained on OpenWebText.

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pre-trained and instruction tuned generative text models in 8B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, they took great care to optimize helpfulness and safety.

This repository contains the weights and the code of the Grok-1 open-weights model.