nyc-weather: An Hourly Forecaster That Learned to Look Upstream

An hourly forecaster for New York City — twelve surface variables over a 500-km box, with probability of rain as the product that actually matters to anybody. Gen1 looked respectable on temperature and hopeless on rain, and an adversarial review established that the fault lay with the experiment rather than with the model: five verified defects, running from a no-op precipitation transform to a deployed checkpoint that was effectively 5%-trained. Everything afterward followed from that one lesson, which is to fix the measurement before touching the model. Gen4 was then won on geometry rather than on scale, on the observation that CDS's request quota counts fields, not area — the same price therefore buys pressure-level data over a Midwest→Gulf→Atlantic box that actually contains tomorrow's weather. The one-shot test read confirmed it: calibration verified out-of-sample, temperature error down 28% at both 24h and 48h, and twenty-four-hour rain skill doubled.

The MIDI/Piano Generation Pipeline

Four related projects aimed at one goal: making dead-flat quantized MIDI — no rubato, no dynamics, no pedal — sound as though a real pianist had played it. midi-transformer gen-3, nicknamed PerformanceNet, is the ~10M-parameter model that worked; gens 4 through 7 chased it with bigger architectures and a custom velocity trajectory loss, almost none of which worked, and their checkpoints were deleted outright rather than hoarded. What survived was forked cleanly into piano-humanizer, which in turn exposed an upstream problem — the rough MIDI it was meant to run on was itself full of quantization noise — and spun off piano-normalizer, an ASAP-trained denoiser built around a deliberately constrained anchored edit-transducer. A fourth branch, piano-generator, trains a from-scratch generator on ARIA performance data instead. The datasets now live in one canonical ~/midi-corpus reached through directory junctions: one source of truth, and no more silent duplication.

yelp-gpt: A GPT-2 Trained from Scratch on Yelp Reviews

A GPT-2 Small trained from scratch, nanoGPT-style, on nothing but Yelp review text — 7M reviews held in five Parquet tables, in that format specifically because the working set then fits comfortably in RAM. The instructive part was the hardware. Naive intuition says a larger batch trains faster up to whatever the GPU can physically hold; on an 8GB RTX 3060 Ti, batch sizes of 8 and above spilled out of dedicated VRAM into shared system memory, and the spillover made throughput worse rather than better despite the configuration technically "fitting," so the fastest setup proved to be a small batch of 4 with gradient checkpointing and 32-step accumulation. The full run covered roughly 1.33B tokens, about one epoch across the 7M reviews, and reached a validation loss of 1.65 — by a wide margin the largest token budget of the three projects running that month.

▶ Try it live — generate a review in your browser

midi-transformer: From a Deterministic Regressor to Gen-3

The generational history behind PerformanceNet, and a case study in measuring the right thing. Gen-1 was a deterministic regressor returning exactly one performance per score, with no room anywhere in the architecture for interpretation; gen-2 rebuilt it as a conditional VAE whose heteroscedastic head worked and whose latent collapsed outright. That collapse was only visible because of a purpose-built eval harness, since validation NLL fell to 0.011 while the model quietly failed, and sweeping the correlation window located the real defect: no notion whatever of phrase-level rubato. Gen-3's most important fix then sat embarrassingly upstream of the network itself, because gen-2's timing target had been capped at ±half a grid step by construction, physically deleting phrase-level rubato from the training data before the model ever saw it. Re-representing timing as an unbounded local-tempo deviation, replacing the collapsed latent with five observed style descriptors, and selecting checkpoints on expression score instead of NLL is what produced gen-3. The eval viewer below plots a real held-out piece: gen-3's unbounded representation traces an actual phrase arc, while gen-2's bounded one is visibly flat by construction.