I quantize and serve large models on hardware that isn't supposed to run them — 16 GB Blackwell GPUs, Jetson edge boards — and I reproduce every published number against its confidence interval before I call it done.
Serving a 50%-expert-pruned MoE coder model on a 16 GB consumer Blackwell GPU — REAP structured pruning + NVFP4A16 quantization, published on Hugging Face. SWE-bench Verified 52.0% (26/50, officially graded), HumanEval+ 89.0% / MBPP+ 90.5% reproduced on the shipped weights inside published confidence intervals, lossless sharding verified via SHA-256 across all 47,013 tensors. CI-checked reproduction pipeline.
The same pipeline on a different 35B-A3B MoE base (MIT) — 256→128 experts, MTP head and vision tower stripped, GPTQ-NVFP4A16, 12.47 GiB. SWE-bench Verified 44.0% (22/50, official harness, same 50-instance slice as the KAT run), HumanEval+ 84.2% / MBPP+ 89.2%. Three upstream vLLM gaps for this architecture patched locally. Model on Hugging Face.
4-bit weights and activations for an 8B MoE on a 16 GB RTX 5070 Ti, custom vLLM CUTLASS SM120 source build. HumanEval 72.6% pass@1 (95% CI [65.3, 78.8]) from a 6.02 GB checkpoint, matching or beating published full-precision results for Qwen 3 7B and Llama 3 8B, and apparently the first HumanEval figure for this model at any precision. 9.5 tok/s single-stream / ~74 tok/s batch-8. Also quantified the reasoning/latency tradeoff: disabling the reasoning trace is ~8.5x faster but costs 17 to 29 accuracy points. 2,400+ downloads on Hugging Face.
Security-first coding agent — async ReAct loop, 4-tier deny-first permission engine, SHA-256 hash-chained audit trail, 200+ LLM providers via LiteLLM. SWE-bench Lite (dev-23 split): 34.8% single-shot · 52.2% oracle best-of-5, free-tier drivers, $0 API spend.
Autonomous multi-agent personal intelligence on NVIDIA Jetson Orin Nano — 5 LangGraph expert agents, LiteLLM gateway, 3-tier ONNX intent router. Fully on-device, zero cloud dependencies. 415 tests.
Python · PyTorch · vLLM / CUTLASS · TRL / Unsloth · NVFP4 · GGUF · GPTQ / AWQ · CUDA · Docker