ShiftAI
A self-hosted studio for fine-tuning small language models on your own data, then scoring what they actually learned and retraining on the questions they got wrong.
Jun 2026 – Jul 2026 · Open source · Python, FastAPI, LoRA / PEFT, Hugging Face, Redis, React, Typescript, Ollama
Closes the loop: after training it auto-evaluates, finds the failed questions, and generates targeted data to retrain on.
Overview
ShiftAI takes documents, spreadsheets or a SQL database and turns them into training data with a teacher model, including reasoning traces, honest refusal pairs, and questions that need two passages to answer. A judge filter drops pairs that are not faithful to the source.
Training is LoRA / PEFT on any Hugging Face causal model, run as a background job with live loss curves, presets, and options like QLoRA, DoRA and answer-only loss.
After training, the model is automatically quizzed on the source. One click synthesises more data for the weak passages and retrains, so you can watch the score go up.
Results can be chatted with locally, compared side by side with the base model, or exported to GGUF and registered in Ollama.
Features
- Ingests CSV, JSONL, Excel, PDF, DOCX, Markdown, or any SQLAlchemy database
- LLM teacher with judge filtering for training-data quality
- LoRA / QLoRA fine-tuning with live progress and cancellation
- Auto-eval and an improve loop that retrains on failures
- Export to GGUF and Ollama
- Heavy ML libraries are lazy-loaded, so the API and UI run anywhere