Production readiness checklist for AI-built apps: a scorecard

What "production-ready" means for AI-built apps

AI-built prototypes are great at getting something on screen fast, but they often break the first time real users arrive. The usual reason is simple: prototypes are built to look right, not to survive messy inputs, bad network days, unexpected traffic, and curious attackers.

"Production-ready" means the app can run safely and predictably for real people, with real data, under real conditions. It doesn't mean perfect or feature complete. It means you can launch without holding your breath every time someone signs up, pays, or refreshes the page.

This scorecard focuses on four non-negotiables:

• Security: secrets and user data stay protected.
• Observability: you can see what broke and why.
• Data integrity: your database stays correct even when users do weird things.
• Deployability: you can ship updates without chaos.

What this does not cover: whether your idea is good, whether your UX makes sense, or whether your app is fast under high load. It also won't replace a full security review for regulated industries. Treat it as a practical minimum for taking an AI-generated codebase from "demo" to "safe to run".

When you score each item, be strict:

• Pass: you can prove it's in place.
• Risky: it works on the happy path, but you have gaps that will show up in production.
• Fail: it's missing or unsafe. Don't launch until it's fixed.

If you inherited a messy prototype from tools like Bolt, v0, Cursor, Lovable, or Replit, this is also a good way to start: identify what's unsafe or unreliable first, then fix the highest-risk issues before adding more features.

How to use this scorecard (step by step)

Start by picking one target environment to judge. If you have staging that mirrors production, use that. If you don't, use production, but run checks during a quiet time and avoid risky changes.

Set a baseline so your answers match reality. Write down what "normal" looks like: how many users you expect in week one, what data matters most (payments, profiles, content), and what spikes might happen (a launch post, an email blast). A chat app for 20 testers has different needs than a checkout flow that handles real money.

Run checks in this order:

1. Pick the environment and freeze the version you're evaluating (commit, build, or release tag).
2. Do security first, then observability, then data integrity, then deployability.
3. For each check, capture evidence: a config value, a log line, a screenshot of a setting, or the exact command output.
4. Score each item as Pass, Risky, or Fail based on what you can prove, not what you assume.
5. Stop and fix anything marked Fail before scoring "nice to have" items.

Evidence matters because AI-built apps often seem fine until one missing setting breaks login or exposes a secret. If you can't show the proof in 30 seconds, treat it as Risky.

A practical example: you're launching a prototype built in Cursor and deployed quickly. You confirm secrets are stored only in environment variables (Pass), but there are no request logs or error tracking (Fail). That means you're shipping blind when users report "it doesn't work".

Security non-negotiables (fast ways to verify)

Security is the part you can't patch later. Treat these as launch blockers: secrets aren't exposed, auth is handled safely, permissions are minimal, inputs are validated, and dependencies aren't full of known holes.

Quick checks most teams can do in under an hour:

• Scan the repo for secrets: search for API_KEY, SECRET, TOKEN, PRIVATE_KEY, and long random-looking values. If you find one, assume it's compromised and rotate it.
• Confirm passwords aren't stored directly: the code should hash passwords (not "encrypt" them) and never log them. In production, session cookies should use secure settings.
• Test write endpoints with bad input: try quotes, long strings, and unexpected types on create/update forms. If the app crashes or stores weird data, you need input validation and safer queries.
• Check least privilege: don't use an admin database user for normal requests, and scope API tokens to only what the app needs.
• Run a dependency scan: use the standard audit tool for your package manager and note any high or critical issues, especially around auth, templating, and database drivers.

A common failure: a prototype uses a single environment variable like DATABASE_URL, then someone copies it into a config file "to make it work" and commits that file. Now anyone with repo access has full database control. The fix isn't just deleting the file. You rotate credentials, move secrets back to environment settings, and reduce permissions so leaked credentials can't drop tables.

Auth, secrets, and injection checks you can do quickly

Auth problems are a top reason an AI-built app feels fine in a demo but breaks in real use.

Start with one end-to-end test: log in, refresh the page, then fully restart the app (or redeploy) and try again. If the app logs you out on refresh, loses session state after a restart, or gets stuck in a redirect loop, you likely have token storage, cookie, or callback issues.

Then check access control. It's not enough that the UI hides buttons. Confirm the server blocks actions too. A quick test is to open a record that should be private (another user's project, invoice, admin page) and see if you can view or edit it by changing an ID in the URL or request.

Secrets are another fast win. If you see API keys, database URLs, or auth provider secrets inside the repo, config files, or client-side code, treat that as an emergency. They should live in server-side environment variables, and the client should only receive public keys meant for the browser.

Five-minute sweep:

• Login still works after refresh and after a full restart (no broken sessions).
• Non-admin users cannot access admin routes, even by typing the URL.
• No debug mode, test users, or "temporary" bypass flags are enabled.
• Secrets are not in code, logs, or front-end bundles (only env vars).
• No raw SQL built with string interpolation. Queries are parameterized.

For injection risk, search for raw queries and string-built WHERE clauses. If you're unsure, treat it as Risky until you can point to parameterized queries and input validation.

Observability non-negotiables (so you can debug in production)

When something breaks in production, you need answers fast: what failed, who it affected, and how often. For AI-built apps, this matters even more because the code can look fine in a demo but fail under real traffic or real data.

The basics are simple: you can see errors (with details), you can see latency (what is slow), and you can see key events (what users and background jobs are doing).

Quick ways to verify (without being an expert)

Run these checks on staging or right after a deploy:

• Force a known error and confirm it's captured with a stack trace. Example: hit an endpoint with bad input. You should see a clear error report that includes where it happened.
• Check logs for request IDs and basic context. Pick one request and make sure you can follow it end-to-end using a request ID. Logs should also include route or job name, plus user or session context.
• Confirm three basic metrics exist: uptime, latency, and error rate. You don't need fancy dashboards. You do need a way to answer: "Are we up?", "Are we slow?", "Are we failing?"
• Verify alerts exist for crashes and failed background jobs. Trigger a failing job (or temporarily break one) and confirm someone is notified quickly.
• Test one real user journey and make sure key events appear. For example: sign up, log in, complete the main action. You should be able to tell where users drop off.

A concrete example: users report "payments sometimes don't work". Without request IDs and error tracking, you're guessing. With them, you can find the failed requests, see the error message, and learn whether it's a timeout, a bad API key, or a logic bug.

Data integrity non-negotiables (protect your database)

If your data is wrong, everything built on it breaks: billing, reporting, permissions, and user trust. AI-built apps often look fine in demos but fail under real usage because the database is missing basic guardrails.

The goal: records stay correct, consistent across tables, and recoverable after mistakes.

Quick integrity checks you can verify today

• Constraints exist where they matter: confirm key tables have unique constraints (emails, external IDs) and foreign keys (orders belong to a user). A quick sign of trouble is duplicate rows that should never duplicate.
• Migrations are tracked and repeatable: you can rebuild the database from scratch using migrations only. If someone says "run this one-off SQL in prod," it's Risky.
• Backups are real, and restore works: a backup that has never been restored is a guess. Do a restore test to a throwaway environment and confirm the app can start and read expected data.
• Writes are safe under retries: if a client retries a request (common on slow networks), the app should not create duplicates. Look for endpoints that create records without a stable idempotency key or a unique constraint.
• Deletes are safe: don't hard-delete records needed for audit, billing, or support unless you have a clear retention plan.

Small example

A "Create subscription" endpoint inserts a new row every time it's called. In testing, it's fine. In production, a payment provider retries a webhook and you get two active subscriptions for one user. A simple fix is a unique constraint on (user_id, provider_subscription_id) plus update-on-conflict logic.

Deployability non-negotiables (can you ship reliably?)

If you can't deploy the same app twice and get the same result, it's not production-ready. AI-built apps often work only on the creator's machine because the build is fragile, config is hidden, and the deploy steps live in someone's head.

A deployable app has predictable builds, safe configuration, and repeatable deploys. That means a clean build step that always succeeds, a clear start step that runs in the target environment, and zero secrets baked into the repo or bundled files.

A quick sanity check is a cold start from scratch (new machine, new container, or a clean checkout). You should be able to do:

• One command to install and build
• One command to start the app
• The same result every time

If that simple flow breaks, the next deploy will break too.

Fast verification checks

Config should be explicit. There should be a short document or README listing required environment variables, which ones are optional, and safe defaults for local development. If you can't answer "what variables does this need in production?" in 2 minutes, the deploy will turn into guesswork.

Startup failures must be visible. A failed migration, missing env var, or crashed server should show up clearly in logs, and the app should expose a basic health check so you can tell if it's ready or stuck.

Finally, verify the app works after deployment: load the main page, confirm static assets load (CSS, JS, images), and hit a few critical API endpoints. A common AI app failure is a hardcoded localhost URL or missing base path that only shows up after deploy.

Add-on checks: performance, reliability, and basic compliance

Once the core items are solid, these add-ons decide whether your app feels trustworthy on day one.

Performance basics (fast enough for real users)

Start with the user's view: click through your slowest screens on a normal connection. If you regularly wait more than a couple seconds, users will bail.

Time three things: first page load, the slowest action (often search or checkout), and a cold start after the app has been idle. If anything hits timeouts or random delays, look for huge API responses, missing pagination, and database queries that scan entire tables.

Reliability basics (fails safely, then recovers)

Many AI-built apps fail in sharp, confusing ways: one flaky dependency takes down everything, or one user can spam requests and crash the server. Aim for graceful failure: clear errors, safe retries, and no data corruption.

A small set of checks:

• Trigger a failure on purpose (turn off a dependency or use bad input) and confirm the app shows a clear message and logs the error.
• Confirm retries are limited (no infinite loops) and timeouts are set for external calls.
• Add basic rate limiting on login and key endpoints so one bot can't flood you.
• Verify background jobs are idempotent (running twice doesn't double-charge or duplicate records).
• Keep a simple rollback plan if a release breaks something.

Compliance is "as needed," but don't ignore it if you touch personal data. Be able to answer: what PII do we store, where, for how long, and who can see it? If you need an audit trail, confirm key events (logins, permission changes, payments) are recorded without storing secrets.

Printable scorecard: quick checks and scoring

Use this as a quick production-readiness checklist. The point isn't a perfect number. It's a clear yes/no on what blocks launch.

Scoring (0 to 2 per area)

• 0 = Fail: missing, unknown, or clearly unsafe.
• 1 = Risky: partially in place, but gaps remain.
• 2 = Pass: in place, repeatable, and backed by evidence.

Mark an area as Pass only if it's a 2.

Quick pass/fail checks (one line each)

• Security: secrets are not in the repo, inputs are validated, and common injections are blocked.
• Observability: errors are captured with context, key actions are traceable, and you can find failures fast.
• Data integrity: migrations exist, constraints are used where needed, and destructive actions are protected.
• Deployability: one command or one pipeline deploys, configs are environment-based, and rollbacks are possible.

Record evidence as you go. For each check, write (1) where you verified it, (2) what you saw, and (3) a screenshot or snippet title you could find again later. Example: "Secrets: checked .env usage and repo scan, found API key in config.ts, needs move to env var."

Turn the scorecard into a 1-week fix list by sorting work like this: anything scored 0 first, then anything that blocks deploys, then anything that blocks debugging.

Common traps in AI-built apps

The hardest part about AI-built apps is that they often look finished before they are safe, stable, or easy to ship. A prototype can feel done because the UI loads and a happy-path demo works. Production fails in the boring details: configs, edge cases, and how the app behaves under real users.

Traps that show up over and over:

• "Works on my machine" equals deployable. Quick check: can a clean environment (new laptop or CI) run it using only a README and environment variables, with no manual steps?
• Debug settings shipped to the internet. Quick check: search for debug=true, permissive CORS like *, and placeholder admin accounts in config.
• Schema changes without migrations. Quick check: is there a migrations folder and a standard migration process?
• No monitoring until users complain. Quick check: can you answer "what broke?" from logs alone, without reproducing locally?
• Fixing symptoms instead of structure. Quick check: do you see repeated patches and duplicated logic across files?

A quick scenario: a founder demos a Lovable-built app flawlessly. On launch day, signups fail because the production callback URL for auth was never set, and the app logs full request bodies (including tokens) to the console. The fix isn't just "change the URL" or "hide logs". It's tightening config handling, secrets storage, and environment parity so the app behaves the same way in dev, staging, and prod.

Example: scoring a real prototype before launch

A founder brings a marketplace app built with Bolt and finished in Replit. The demo looks great: users can sign up, create listings, and pay. They want to launch to real customers, so they run a quick scorecard before spending money on ads.

What fails first is almost never the UI. It's the hidden stuff: login edge cases, secrets in the repo, unsafe database writes, and a deployment that only works on the creator's machine.

Here is what the scorecard can look like after 30 minutes:

• Security: 0/2 (Fail) - A .env file with API keys was committed, and a search box builds SQL with string concatenation.
• Observability: 0/2 (Fail) - Errors show in the browser console, but the server has no structured logs and no request IDs.
• Data integrity: 1/2 (Risky) - Deleting a user leaves orphaned listings; there are no foreign keys on key relationships.
• Deployability: 1/2 (Risky) - It deploys, but build steps are manual and environment variables are not documented.

They don't fix everything at once. They fix what can leak data or break payments first:

• Remove exposed secrets, rotate keys, and add a secret scanner
• Replace unsafe queries with parameterized queries and validate inputs
• Lock down auth cookies, add basic rate limits, and tighten permissions
• Add minimal logs (errors plus key events) and a simple health check

After a couple days, the app launches with fewer surprises: signups stop spiking error rates, failed payments are traceable, and deployments are repeatable.

Next steps: fix the gaps before you ship

Treat your scorecard results as a plan, not a verdict. Sort every failed check into three buckets: risks that can expose users (security), risks that can corrupt data (integrity), and risks that can break releases (deployability). Fix those first, even if it delays nice-to-have features.

A simple way to turn failures into action is to write each one as: "Problem -> user impact -> fix -> owner -> ship date." If you can't explain the impact in one sentence, it's probably not a top priority yet.

Suggested priority order:

• Stop-the-bleed: exposed secrets, broken auth, injection risks, unsafe file uploads
• Data safety: missing constraints, weak migrations, no backups, risky deletes
• Operability: no logs, no error tracking, no health checks
• Release safety: no staging, no rollback plan, builds depend on local config
• Cleanup: refactors, tests, code style

Know when to stop patching. If every fix creates two new bugs, the architecture is tangled, or the app has no clear boundaries (UI, API, data), a refactor or rebuild is often cheaper than weeks of whack-a-mole.

If you'd rather not debug this alone, FixMyMess (fixmymess.ai) specializes in taking broken AI-generated prototypes and making them safe to run in production, including codebase diagnosis, auth and logic repair, security hardening, refactoring, and deployment preparation. A quick audit can also give you a clear Pass/Risky/Fail list so you know what to fix first.