Building an AI Consultant Chatbot with Gemini — Lessons from Shipping One

If you're reading this on our homepage, you've seen the AI Project Architect widget. It's a live Gemini-powered chat that helps prospects scope an app idea. We shipped it in a weekend and it has been an outsized lead generator since.

Here's what we'd do the same again — and what we'd change.

The architecture

• Frontend: React + Vite + Tailwind (the same stack as this site).
• Streaming: Gemini's generateContentStream — the typing indicator makes a huge UX difference.
• System prompt: ~400 tokens that frame the model as a technical co-founder focused on scoping MVPs.
• Contact handoff: A "Contact Team" button that captures the chat transcript and emails it to us.

What we got right

1. Low-friction entry. No sign-in, no email gate. Users type first, then optionally share their email with the transcript.
2. Transcripts-as-leads. Every submission arrives with the full conversation attached. Our sales calls skip the "tell me what you want" phase entirely.
3. Streaming. Removing streaming in an A/B test dropped engagement 40%. It's a must.

What we got wrong

1. No rate limit at launch. Within a week someone wrote a loop that cost us ~$40 in a day. We added IP-based caps the next weekend.
2. Too-broad system prompt. We asked the model to "be helpful about anything software" — so it happily explained database indexes to people who wanted to hire us. A tighter prompt keeps the conversation oriented toward scoping, not tutoring.
3. No retry/backoff. Gemini occasionally 429s. A silent retry with small backoff eliminated 80% of observed failures.

The surprising usage pattern

The most common opener was not a polished app idea — it was some version of "I don't really know where to start." That told us the real job of this chatbot isn't to architect, it's to disarm and orient. We rewrote the system prompt around that insight.

Cost

For the traffic we get, monthly Gemini cost is under $8/month. The AI architect has paid for itself many times over in leads.

Would we do it again?

Yes, tomorrow. If you're a services company, an AI chatbot that scopes rather than pitches is one of the highest-ROI things you can ship to your site. We'd be glad to build one for you — start a chat with the actual architect.