A new AI tool lands in your feed with a demo that looks like magic. You have a job to do and a limited amount of patience, and you can't afford to sink an afternoon into something that turns out to be a login wall wrapped around someone else's model. You also can't afford to be fooled by a video that was recorded on the one prompt that happened to work.
Here is a timed flow you can run yourself, minute by minute, before you commit real time or real data. It won't give you an objective score. It gives you an informed opinion, built from things you checked with your own eyes instead of things the landing page told you. Fifteen minutes is enough to reach a confident go, wait, or skip. It is not enough to clear a tool for high-stakes work, and this flow is not a substitute for a proper security or legal review when the stakes are real.
Set a timer. Open the tool's site and a fresh notes file. Go.
Minutes 0-3: Read the claim, not the animation
The demo is the sales pitch. Your first job is to translate it back into a plain sentence.
Write down, in your own words, what the tool actually claims to do. Not "AI-powered workflow intelligence." Something a coworker would understand: "it turns a meeting recording into a summary and action items," or "it generates React components from a screenshot." If you can't write that sentence in three minutes, that is itself a signal. Vague positioning often hides a thin product.
While you're reading, separate two things:
- What it does today, phrased in the present tense on the actual product pages.
- What it promises for later, usually in future tense, on a roadmap, or buried in a waitlist.
Marketing loves to blur these together so tomorrow's vision borrows credibility from today's shipped features. Keep them in separate columns in your notes. You are evaluating what you can use this week, not the founder's ambition.
One more quick move: find out whether the tool is doing the hard thing itself or wrapping a model you could call directly. That's not automatically bad — a good wrapper earns its keep through workflow, integrations, and taste. But you should know which one you're buying so you can judge the price fairly. If the whole product is a single prompt in a nicer box, you want to notice that now, not after you've paid.
Minutes 3-7: Find the limits, and the free-tier reality
Now go looking for where the tool stops. Every honest product has edges. The fast way to respect a tool is to find its edges quickly.
Open the pricing page first. You are checking two things:
- What the free tier actually allows versus what the demo showed. This is the single most useful check in the whole flow. Demos are almost always recorded on the paid plan with the limits switched off. The free tier is often a much smaller thing: a handful of generations a month, a watermark, no export, no API, a tiny context window, or a "free" plan that requires a card and converts to paid after a trial. Write down the real free-tier ceiling in concrete numbers.
- What the jump to paid costs, and whether pricing is per-seat, usage-based, or credit-based. Usage-based pricing can get expensive fast if the tool is doing something token-heavy, and "credits" are designed to make that cost hard to feel.
Then hunt for the failure modes. Skim the docs or a help center for words like limits, quota, supported, and known issues. A tool that documents what it can't do is usually more trustworthy than one that pretends it can do everything. If the docs are thin or missing, note that — thin docs on a shipping product often mean the product is younger than the marketing suggests.
If you can create an account inside your seven-minute window, do the smallest real test you can: one prompt, one file, one component — your actual use case, not the demo's. The gap between the staged demo and your first real attempt tells you more than any review.
Minutes 7-11: Data policy, where your input goes
This is the question every AI evaluation should ask and most people skip: when I paste something in, where does it go, and what happens to it?
You don't need a law degree. Find the privacy policy or a "trust" / "security" page and look for plain answers to four questions:
- Do they train on your inputs by default? Many tools do, unless you opt out or are on a paid tier. If you're pasting anything sensitive, this matters immediately.
- Is there a way to opt out of training, and is it on or off by default? Default-on is a meaningful red flag for anything work-related.
- Do they retain your data, and for how long? Look for a stated retention period versus vague "as long as necessary" language.
- Who's the underlying model provider, and does your data pass to them? A wrapper often forwards your input to a third party, which means their data policy applies too.
For anything involving customer data, source code, health or financial information, or anything under an NDA, treat a missing or evasive data policy as a hard stop. Not a "wait" — a stop, until someone qualified reviews it. This flow is for your own judgment; it does not replace your organization's security review, and nothing here is legal advice.
If the tool clearly states that it doesn't train on your data, retains it briefly, and offers an opt-out, that's a genuine point in its favor. Write it down.
Minutes 11-15: Proof of shipping
You've read the claims and the limits. Now check whether this is a living product or a launch-day shell.
Look for signs of real, ongoing work:
- A changelog or release notes with recent dates. Regular, specific updates ("fixed X, added Y") are one of the most reliable signals that a small team is actually shipping. A changelog that stops six months ago tells its own story.
- Independent signal, not just testimonials on the tool's own page. Search the tool's name alongside words like review, alternative, or vs. You're looking for people using it in the wild, discussing real friction, not a wall of five-star quotes the company selected.
- A findable team and a way to reach them. A named team, a support channel, a status page — these are cheap for a real product to have and awkward for a vaporware one to fake.
- Honest positioning about maturity. Products that say "beta," "early," or "we don't support X yet" are often more trustworthy than ones claiming to already do everything.
You will not verify a company's whole history in four minutes, and you shouldn't try. You're just checking for a pulse: is someone home, and have they been home recently?
The verdict: go, wait, or skip
Close the timer. You now have a page of notes built from things you checked, not things you were told. Turn it into one of three calls, and write down the reason:
| Verdict | What it means | Typical trigger |
|---|---|---|
| Go | Worth a real trial with real (non-sensitive) work this week | Clear claim, honest limits, sane data policy, signs of active shipping |
| Wait | Promising but not proven; revisit in a few weeks | Thin docs, brand-new, vague on limits, roadmap doing the selling |
| Skip | Not worth your time right now | Can't state what it does, evasive on data, dead changelog, pure demo |
The point of recording the reason is that verdicts expire. A "wait" today can become a "go" once the changelog fills in and the data policy gets clearer. Your note lets future-you re-check quickly instead of starting over.
A word on what this verdict is and isn't. It's an opinion backed by criteria you can see, not an objective rating you can hand to someone else as fact. Two careful people can run this flow and land in different places, and that's fine. What you've bought yourself is a judgment that stays ahead of the marketing instead of chasing it.
In Plain English
What this is: a 15-minute, minute-by-minute routine for sizing up a new AI tool — reading the real claim, finding the limits and free-tier reality, checking where your data goes, and looking for proof the thing is actually shipping.
Who it helps: anyone who tries a lot of AI tools and is tired of losing time to slick demos — builders, operators, and curious people who want to stay optimistic without being credulous.
When to use it: the moment a tool catches your eye and before you paste in anything real or reach for your card. Use the deeper review, not this, when the stakes are high.
What to do next: run the flow once on the next tool you're tempted by, write down your go / wait / skip and the reason, and keep the note.
This is the same posture we bring to the tools we're actually excited about: excitement is good, but it should survive a few honest questions. If you want to see the fuller process behind our listings, read how we vet an AI tool before we list it, and if you want a sharper eye for the failure case, here's what "AI slop" is and how we tell signal from it. You can browse everything that's cleared the bar in the directory or explore the wider AI and frontier software hub, and the standard behind every listing is set out in how we curate.
Grab the review checklist, keep it next to your browser, and run it on the next tool that promises the world. Fifteen minutes now saves you the afternoon you'd have lost to a demo that only worked once.