A small proof-of-concept language model (not an LLM) incorporating latent-space prediction, internal state using recurrent trace units, and byte-by-byte output, built with MLX.
| .gitignore | ||
| benchmark.py | ||
| LICENSE.md | ||
| main.py | ||
| README.md | ||
Hey! Thanks for being here.
Here's the video, if you came here from somewhere else -> Video
The train and benchmark scripts are provided (using MLX, but you can port to other platforms if you want).
Note that datasets are not included, and if you use a non-puretext dataset like wikipedia dump then feel free to write your own dataset extraction code or use an existing library.
The repo is MIT license, so feel free to fork the repo, I would be very happy to see that. Go ahead and explore!