AI/ML · Software Engineering · Databases & Data
Train a small language model from scratch with licensed data
Build an educational causal language model from randomly initialized weights using a licensed, provenance-tracked corpus. Train a dedicated tokenizer, define a roughly 34-million-parameter decoder architecture, establish random and unigram baselines, checkpoint and resume one bounded epoch, evaluate held-out loss and qualitative limitations, run local inference, and publish only an honest research artifact.