In June 2023 I joined Inferless as the owner of the Console — the web frontend developers use to deploy and monitor models on serverless GPUs. Two years and change later, that surface has gone from a pilot-stage app serving a handful of design partners to a GA product. This is the retrospective I wish I'd read before starting.

Pilot code is a promise you make to your future self

The Console I inherited was pilot code, and pilot code is correct pilot code: built fast, validated the idea, made no apologies. The mistake would have been treating it as a foundation. The other mistake would have been rewriting it on week one.

I did neither for three months. I shipped features inside the existing structure, and every time something hurt, I wrote it down. By month three the list had converged on the same four wounds: state management sprawl, no release process, inconsistent UI patterns, and zero visibility into what broke in prod. That list became the architecture roadmap — driven by evidence instead of new-joiner taste.

A design system nobody asked for (and everyone used)

We were four engineers. Nobody says "let's build a design system" at that size — you build the fifth slightly-different button instead. My compromise was a lightweight one: tokens, a dozen primitives, and one rule — new UI composes primitives, and if a primitive is missing, we add it, we don't inline it.

The payoff wasn't consistency for its own sake. It was speed. When GA prep landed and we needed dark mode across the entire product, it was a token swap, not a quarter. Developer tools get judged on feel — a console that looks stitched together makes people wonder what the infra looks like.

Cold starts are a frontend problem

Here's the thing nobody tells frontend engineers joining an ML-infra company: the hardest UI problem is waiting. A serverless GPU platform has cold starts, and a deploy can sit in a queue, pull a container, warm a model. If the Console shows a spinner for ninety seconds, users assume it's broken — because on every other website, it would be.

We rebuilt the deploy view as a narrated timeline: each stage of the lifecycle rendered as it happened, with real timestamps streaming in. Same latency, completely different perceived reliability. Support tickets about "stuck deploys" dropped to nearly zero without a single infra change. The lesson stuck with me: when you can't make it faster, make it legible.

The load-testing harness that paid rent twice

Mid-way through, I built an internal load-testing harness — fire traffic at a model endpoint, chart latency, throughput, error rate, and cold-start counts. It was meant for us, to validate releases.

Then we started screen-sharing it in customer calls. Watching your own model's p99 flatten out in real time answers the "but will it scale?" question better than any slide. Internal tools have a way of becoming product if you build them honestly. It also taught me more about our backend than any doc — auth, model lifecycle, job queues — which is how I ended up contributing there too, mostly to unblock onboarding flows the Console needed.

GA is a mindset shift, not a milestone

Pilot: ship it, we'll know in an hour if it broke, both users are in our Slack. GA: strangers with production workloads and no Slack channel. The transition demanded boring things, done thoroughly — a real release flow with staged rollouts, feature flags as standard practice, error tracking wired to alerts instead of vibes, and the discipline to keep a changelog humans can read.

None of it is glamorous. All of it is the difference between a product and a demo.

What I'd tell 2023 me

  1. Fix things in the order they hurt, not the order they offend you.
  2. Perceived reliability is a feature. Narrate slowness; never hide it.
  3. A primitive library beats a design system until well past the size you think it does.
  4. Build internal tools like someone will see them — someone will.
  5. Own a surface end to end. The Console forced me into the backend, into infra dashboards, into customer calls. "Frontend engineer" was the job title; the job was making the product trustworthy.

That last one is why I'd recommend a startup console team to anyone. You don't get to stay in your lane, and it turns out the lanes were fake anyway.