Threshold billing: Fraud reduction for AI at scale

The modern AI ecosystem is an ambitious race to push the boundaries of what we thought possible with software. Whether that’s through unbelievable programming assistants, solving the blank page problem, or helping us to imagine new futures, every AI company is looking to both advance research and provide user value as quickly as possible. 

There are two properties that feel true of every AI product today:

1. Seeing the magic requires playing with the product since the quality of AI output can’t be theoretically derived (no matter what the benchmarks say!). It’s hard to imagine a future today where AI software will be solved with a pure enterprise procurement process; users want to prompt and play with your product. They want to plug in their use case and see what happens.
2. Unlike traditional SaaS software, running AI has real marginal costs — perhaps to a third-party LLM, a vector database, or a cloud provider that runs the underlying architecture. Real intelligence comes at a real cost to the vendor, precisely because the work being done is so custom to the use case.

This creates quite a tension — companies are eager to open up their products for as many potential end-users as possible with as little friction as possible. Pay-as-you-go pricing is great for this, with no or little upfront payment required.

However, most companies traditionally also take on credit risk in the ‘try before you buy’ motion. The user doesn’t have to enter a proven up-front payment method on sign-up, and sometimes never an email at all. With very little COGS and dreamy SaaS margins, this system is completely manageable — worst case you’ve lost a few dollars of infrastructure spend.

Not so with the vast majority of AI companies. Not only does each request come with its marginal cost, the AI models exposed by platforms are an extremely attractive target for spammers because of the functionality they offer. Spam can be used to wrap and resell an application, steal a fine-tuned model, or to effectively engineer a harmful model response.

There’s always a cost to customer acquisition and conversion funnels are leaky, so it’s not that 100% of use needs to be legitimate. Solving this problem without compromising an effective PLG motion requires the concept of threshold billing, which allows companies to thoughtfully cap their downside. Threshold billing involves defining a set dollar threshold per customer, triggering a new invoice each time that value is hit over the course of a billing period. When paired with an auto-collection mechanism, a payment can be triggered very close to the target threshold value, and service is typically discontinued on a failed payment or insufficient funds.  

Threshold billing: the Orb way

We’ve built threshold billing into Orb with a unique approach that retains the scale and flexibility that define our core billing engine. Using the same underlying technology as our alerting infrastructure, Orb refreshes invoice amounts as usage events stream in. When the configured threshold is hit, a partial invoice is created for the subset of the period in as little as a few minutes and immediately collected against the payment gateway. The original invoice is annotated with the partial invoice, and each usage event in the billing period remains completely auditable. 

For established customer relationships you trust, you may not want to trigger a bunch of successive invoices. Orb also gives you the control to adjust or entirely remove thresholds, allowing you to reflect your increased confidence in those customer relationship.

Orb helps leading infrastructure companies like Replit tackle the fraud problem without an in-house lift:

At Replit, our mission is to simplify the process of turning ideas into reality with software. Orb plays a crucial role in this by providing real-time alerts and threshold-based billing, enabling us to deliver Replit on-demand to a large user base while mitigating risks from bad actors. - Ted Summer, Software Engineer

A legacy billing system like Stripe Billing does support threshold billing, but falls apart for usage-based use cases where it matters most, since it isn’t able to handle real-time event volume. In particular, Stripe’s billing engine — originally built for subscriptions and seats — isn’t meant to support ingesting or aggregating large amounts of data.

Threshold billing embodies two core product principles at Orb: configurability to align with your business goals, and real-time functionality to allow you to react to changes quickly. Orb equips you with the tools to combat spam by triggering charges, all with the confidence that you can do so before it creates a significant burden on your balance sheet. 

Especially now that AI products have such a powerful ‘aha’ moment, companies are looking to remove as much friction as possible in the buying process. Teams that have a tool to control the amount of risk they underwrite will bring AI products to market faster, and be able to invest in their core technology without the concerns that come with systemic fraud. Building features like threshold billing without a real-time system like Orb is particularly challenging; it requires years of engineering effort to build a system that can fluidly serve the infrastructure needs of engineering and accuracy needs of GTM and finance teams.