Overview
ZoniSoft AI Chatbot is a production AI chat platform that removes the single biggest weakness in most AI products, which is total dependence on one AI provider. It is live at ai.zonisoft.io and serves both browser users and external developers through a public API.
Instead of connecting to one model provider, the platform manages a pool of eight of them and treats provider availability as an engineering problem to be solved rather than a risk to be accepted.
The Problem
Most AI applications are wired directly into a single provider API. That creates three recurring failures. Rate limits and outages take the entire product offline. Key exhaustion during traffic spikes causes visible errors for real users. And any pricing or policy change from that one vendor lands directly on the business.
On top of that, the surrounding platform work, meaning quotas, billing, abuse prevention, admin control and audit history, usually takes longer to build than the AI feature itself. Most teams underestimate it and ship late.
The Solution
The platform is built around a provider agnostic AI gateway. Every request goes through a single orchestration service that selects a provider key by priority, tracks the health of each key, and fails over to the next available option inside a fixed time budget.
Keys that hit a daily cap, a rate limit or a transient error are placed into cooldown automatically and skipped until they recover, so a degraded provider quietly stops receiving traffic instead of returning errors to users.
AI work never runs inside the web request. Prompts are dispatched to a dedicated queue and processed by background workers, while the browser polls for the result. That keeps the interface responsive and removes gateway timeouts entirely, even on slow model responses.
Key Features
Multi provider AI gateway. Eight providers in one prioritized pool: Gemini, SiliconFlow, Groq, OpenRouter, Cloudflare, Cerebras, Hugging Face and OpenAI, each supporting multiple keys with independent health tracking.
Automatic failover and key cooldown. Provider and key selection, retry estimates and cooldown states are coordinated through a dedicated key state service using cache locks, so parallel workers do not fight over the same failing key.
OpenAI compatible API. A drop in compatible endpoint for chat completions and model listing, so any existing tool, SDK or automation platform can connect by changing the base URL alone.
Native asynchronous API. A queued API that accepts a request, returns an accepted response with a polling URL, and tracks the full lifecycle of every request with stored messages and token accounting.
Live web search grounding. The system detects prompts that require current information and pulls context from Tavily, Serper or Brave before answering, which avoids stale training data on time sensitive questions.
Subscription billing. Stripe recurring checkout and PayPal orders, with plan management, webhook verification, activation, expiry and cancellation handled through a shared subscription layer.
Quotas and role based access. A four level hierarchy covering guest, user, admin and superadmin, with per session guest limits, per user prompt quotas, unlimited access for paid subscribers, and scoped account management for admins.
Secure API credentials. Keys are shown once at creation, stored as hashes with only a searchable prefix in plaintext, and resolved through dedicated authentication middleware on every request.
Audit logging. Every administrative action records the actor, target, structured details and client IP, giving a complete traceable history of account, key and plan changes.
Scheduled maintenance. Daily jobs expire lapsed subscriptions, send upcoming expiry notices and prune old conversation history according to a configurable retention policy.
Automation and Integration
Because the platform speaks the widely adopted OpenAI request format, it plugs into existing automation stacks without rewriting anything. Teams running n8n, Make, Zapier, LangChain or any OpenAI SDK based tool point their existing node at a new base URL and immediately gain provider failover, usage quotas, spending control and a full request audit trail across every workflow they already built.
This makes it especially useful for agencies and internal platform teams who want one governed AI endpoint instead of dozens of untracked provider accounts spread across projects.
Architecture Notes
The application is server rendered with a custom vanilla JavaScript chat client, backed by a service oriented backend where controllers stay limited to HTTP concerns and all business transitions live in dedicated services. Provider behaviour, quota calculation, subscription state, payment gateways, auditing and API request lifecycle each have a single owning service, which keeps the browser chat, the native API and the compatible API consistent with one another.
Persistence covers identity, sessions, queues, chat history, API activity, subscriptions, payments, audit logs and database backed settings. Deployment is automated through a CI workflow that installs production dependencies, warms framework caches and ships to the server over SSH.
Outcome
The platform is live in production and handles browser chat, external API traffic and paid subscriptions from a single codebase. Provider outages that would normally cause visible downtime are absorbed silently by the failover layer, and the surrounding commercial features that usually delay AI launches by months were delivered as part of the same build.
Skills Demonstrated
AI and LLM Integration. Designed a provider agnostic gateway across eight AI services with priority routing, key pooling, health tracking, cooldown states, response validation and time bounded failover.
Backend Architecture and System Design. Built a service oriented structure with clear ownership boundaries, asynchronous job processing, cache coordinated concurrency control and a single source of truth for AI, billing and quota logic.
API Design and Developer Experience. Delivered both a native asynchronous API and a standards compatible API surface, with documented authentication, polling, rate limiting and request lifecycle tracking.
Payment and Subscription Engineering. Implemented dual gateway billing with signature verified webhooks, subscription activation and expiry, payment records and automated renewal handling.
Application Security. Hashed credential storage, session regeneration at authentication boundaries, rate limited authentication routes, proxy aware IP handling, role enforcement middleware and complete administrative audit logging.
DevOps and Production Deployment. Automated deployment pipeline, scheduled maintenance commands, queue worker operation, environment driven configuration and a live production environment.
Product Thinking. Identified vendor lock in and platform overhead as the real blockers to shipping AI products, then solved both in one system rather than building another thin chat wrapper.