Before launching an AI-assisted app, map its data and trust boundaries, keep secret credentials out of the browser, verify server-side and database authorization, review code and dependencies, and test the release candidate as users would. The objective is evidence that important access paths behave as intended, not a claim that the app is perfectly secure.
For a customer portal, create two test accounts and try to read or change the other account’s records through both the interface and the direct API. Then inspect database policies, grants, browser responses, dependencies, and the deployed build. A clean scanner result cannot show that one customer is blocked from another customer’s data.
Two reported figures need careful context. Veracode found a detectable security flaw in 45% of the AI-generated coding tasks it tested. That describes its study tasks, not the percentage of insecure production code. WIRED reported that researchers identified more than 5,000 publicly accessible vibe-coded apps and estimated that about 40% exposed sensitive data. WIRED also said it could not independently verify all apparent data. See the Veracode report and WIRED’s account of the investigation.
What does a useful vibe coding security review need to prove?
Start with a one-page threat summary. Record who can use the app, what data it stores, which external services it calls, what actions change records or spend money, and which routes or tables are intended to be public. Use this summary to choose checks that match the app’s actual risks.
- Data and trust boundaries: identify sensitive fields, public endpoints, user-controlled inputs, and server-side operations.
- Credentials: confirm secret keys stay out of source, browser bundles, logs, screenshots, and prompts.
- Authorization: test server-side checks, database grants, policies, and the requests real users can make.
- Code and packages: run code analysis, dependency analysis, and secret scanning as separate checks.
- Release behavior: inspect the built app, network responses, uploads, and costly operations before deployment.
A security review is useful when every important claim about access, data, and deployment can be tied to an observed result or a specific configuration record.
Static analysis and AI-assisted review are evidence sources, not proof of security. Use them alongside direct behavior tests and human review of high-impact changes.
Set up a review loop that produces traceable findings
Put project-specific expectations in the agent’s instruction file, such as CLAUDE.md or AGENTS.md. Include user roles, sensitive data, deployment exposure, privileged operations, and rules such as “authorization is checked on the server.” The public security-rules example is a plain-language starting point. It is not executable enforcement or a complete security system.
After a feature commit, major fix, or release-candidate build, give the agent the threat summary, commit, changed files, schema, manifests, and role rules. Ask it to explain changed trust boundaries, identify plausible attack paths, and propose concrete tests. Keep its role bounded: the agent can suggest and explain, scanners and tests produce their own results, and a person decides whether the build can ship. If you are defining a bounded agent workflow, see ConsultEvo’s AI agent services.
Use a finding record that matches the observation
The following is an editorially proposed record for one finding occurrence. It is not a vendor schema. Its grain is one scanner observation in one scan run, linked to one commit and evidence location.
{
"scan_run_id": "run-2026-11-13-01",
"commit_sha": "a1b2c3d4",
"scanner_name": "example-scanner",
"scanner_version": "1.2.0",
"rule_id": "AUTHZ-001",
"normalized_file_path": "src/api/orders.ts",
"line_start": 42,
"line_end": 47,
"evidence_hash": "illustrative-hash",
"status": "open",
"reviewer": "assigned-person"
}
Keep an immutable occurrence for each scan. Use a separate fingerprint to group related observations across runs. A repository-plus-rule key is too broad because the same rule can flag multiple files or locations. If concurrent workers may record findings, enforce the intended uniqueness in the database and use an atomic upsert. A lookup followed by an insert can race.
For example, a proposed uniqueness rule for one scan occurrence could include the scan run, repository, commit, scanner, rule, normalized path, and evidence location. A persistent grouping fingerprint may omit the scan run so that related results can be tracked across scans. Choose the key from the declared row grain rather than copying a key from a different tool.
Deterministic checks are appropriate for parsing fields, allowed values, package existence, lockfile consistency, secret patterns, and expected HTTP status codes. Use AI for threat brainstorming, explaining data flows, and suggesting missing tests. Do not use an AI judgment as the gate that permits a sensitive write or production change.
Test authorization and database access with real identities
Make a role-operation matrix before writing policies. For each table or endpoint, state the expected read, create, update, delete, and export behavior for an anonymous visitor, an authenticated owner, an authenticated non-owner, and a server-side role. Then compare actual API responses and data changes with that matrix.
- Create User A and User B, each with a record that belongs to that account.
- As User A, try to read, update, delete, and export User B’s record by changing its identifier. Test through the interface and by calling the API directly.
- Repeat while logged out. Check that a denied request reveals no protected fields and makes no database change.
- Try changing client-supplied role, plan, owner ID, price, or record ID values. The server must validate identity, permissions, and authoritative values rather than trust those inputs.
A database policy can coexist with broad grants, and privileged server credentials may bypass row-level security. Review grants and policies, then test requests using the actual roles and paths the app uses. Save both the configuration evidence and the observed allow-or-deny result.
Supabase and Postgres
For Supabase and Postgres, review table grants as well as row-level security policies. Supabase recommends enabling RLS on exposed tables and defining policies for the operations the app needs. A table’s behavior still depends on grants, roles, policies, views, functions, request identity, and credentials.
Keep secret or service keys server-side because they can bypass RLS. Review the official guidance on Supabase RLS, API access, and the production security checklist. Test each required operation with both an allowed identity and a denied identity. An RLS policy is configuration evidence; the API result and database change are behavioral evidence.
Firestore
For Firestore, test the application’s real queries against its Security Rules. A query that cannot satisfy the rules may fail rather than return a filtered subset, so a policy that appears restrictive can also break a legitimate list. Check the Firestore rules and query conditions.
Keep credentials, dependencies, and sensitive work on the right side of the boundary
Keep secret credentials out of client code
Classify each credential by purpose, owner, required permissions, storage location, and rotation procedure. Secret keys belong in server-side configuration using the hosting platform’s documented environment-variable mechanism, not in source code, browser bundles, logs, screenshots, or AI prompts. During local development, keep secrets in an ignored local environment file and confirm it is excluded from version control.
A publishable key may be designed for browser use, but that does not make its access harmless. The provider’s authorization rules, scopes, database policies, allowed origins, and abuse controls determine what it can do. Review its intended exposure and permissions.
If a secret was committed, deleting it from the latest file does not revoke it. Follow the provider’s process to revoke or rotate it, update deployment configuration, and check relevant history and logs. Public-key handling is provider-specific, so do not apply secret-key rotation assumptions to every client-exposed key.
Verify every generated dependency
Before accepting a new package, compare the declared name, intended registry, resolved version, lockfile entry, and linked repository identity. Review relevant advisories and package history. Research documents package hallucination and the possibility that an attacker registers a malicious package under a name generated by a coding model. That establishes a supply-chain risk, not proof that a particular unfamiliar package is malicious.
Treat a lookalike name or unclear provenance as a reason to pause and investigate. Downloads, a recent release, or a low-activity repository are signals for review rather than automatic verdicts. Preserve the resolved package version and registry in the dependency review record.
Tools such as Socket may support dependency review, but inspect the configured product and results rather than assuming a tool detects every package risk. Code scanners also need to be enabled and configured. For example, GitHub CodeQL code scanning does not run merely because a repository is public. Check current setup and plan details in the Semgrep documentation or the Snyk plan information before relying on a specific feature or allowance.
Keep authoritative decisions on the server
Front-end code can be inspected and changed. The server should verify permissions and compute values that affect money, privileges, or protected records.
For a checkout, accept an order identifier, quantity, and coupon code as untrusted input. Load current prices and eligibility on the server, validate the request, calculate the total, and only then write the order. Add idempotency controls where a retried request could otherwise create a duplicate charge or record. Apply the same rule to subscription tiers, discounts, administrator flags, and ownership changes: recompute the authoritative value from trusted server-side data before writing it.
Use a repeatable implementation pattern for review examples
The following patterns are proposed operating designs, not documented modules supplied by a scanner or database vendor. Each separates the AI task from the deterministic validation and identifies where the result goes.
Feature-triggered review
- Trigger: a feature commit, major fix, or release-candidate build.
- AI job: summarize changed trust boundaries, identify plausible attack paths, and propose tests from the threat summary, schema, manifests, and role rules.
- Validation: run configured SAST, dependency, secret, and authorization checks against the actual commit or build. Confirm every finding points to the reviewed evidence.
- Action: attach immutable findings to a scan run and commit, assign remediation, and rerun relevant checks after the fix.
- Fallback: pause for a named human reviewer when authentication, authorization, RLS, migrations, secret handling, payment logic, or production infrastructure changes.
Two-account authorization test
- Trigger: a new authenticated route, record operation, role, or database policy.
- AI job: suggest missing cases from the role and ownership rules without producing the pass or fail result.
- Validation: record the actor, target owner, operation, HTTP status, response fields, and whether the database changed. Compare them with the access matrix.
- Action: store the integration-test result in the release review record and investigate any mismatch.
- Fallback: if the UI and API disagree, treat the direct API result and server-side data change as the priority investigation path. Do not approve until the discrepancy is explained.
Generated-dependency verification
- Trigger: an AI coding agent proposes or installs a new dependency.
- AI job: explain why the package is needed and identify evidence to check. The explanation is not proof of legitimacy.
- Validation: verify the package in the intended registry, confirm project identity, compare the resolved version with the lockfile, and review advisories and maintenance history.
- Action: approve the dependency only after the reviewed manifest and lockfile contain the expected package and version.
- Fallback: investigate or reject unclear provenance, lookalike names, registry mismatches, or unexplained install commands before installation or deployment.
Exposed-secret containment
- Trigger: a secret appears in source control, a browser bundle, logs, screenshots, prompts, or suspicious provider usage.
- AI job: help assemble a containment checklist and identify affected code paths. Never provide the secret value to the AI service.
- Validation: confirm revocation or rotation in the provider account, verify that the replacement is absent from the built client, and test the affected feature.
- Action: deploy the replacement configuration, preserve useful logs, monitor provider activity and billing, and record the incident owner.
- Fallback: if user data may have been accessed or an attack is ongoing, preserve evidence and escalate to an incident-response professional or legal counsel.
Test the running release candidate
Source review cannot reveal every exposure in deployed behavior. Test staging or the actual release candidate and record the expected status, response, and data change for each case.
- Inspect network responses: confirm APIs do not return hidden fields, internal notes, tokens, or other users’ records merely because the interface does not display them.
- Try invalid inputs directly: test type, range, length, and format boundaries through the API, not only through form validation.
- Check rendered content: verify user content is safely rendered as text, or sanitized for its intended rich-text context.
- Test uploads by content: try a file whose extension does not match its content. Validate type on the server rather than relying on the filename.
- Exercise costly endpoints: test rate limits and define what happens on HTTP 429. Quotas vary by provider and scope. Google Sheets, for example, documents 60 read requests per minute per user per project and a separate project-wide quota, and recommends exponential backoff for quota errors. Those limits are specific to that API, not a universal setting for app endpoints.
For paid API calls, payment totals, discounts, subscription decisions, and privileged writes, verify both who is allowed to perform the operation and where the authoritative decision is made. For high-volume unauthenticated endpoints, consider controls beyond a single-IP limit because distributed traffic can bypass it.
If a key or user data may be exposed
For a suspected exposed secret, identify the provider and credential, then revoke or restrict it using that provider’s controls. Preserve useful logs, deploy the replacement configuration, test affected features, and monitor provider activity and billing. Do not send the secret value to an AI service while investigating.
If user data may have been accessed, preserve relevant evidence and involve an incident-response professional or legal counsel to assess notification duties. The required response depends on the data, incident facts, contracts, sector, and applicable jurisdiction. Assign one incident owner to coordinate containment, investigation, and the decision about notification.
Use a release gate, not a security-score shortcut
Release only when a named reviewer can trace each critical check to the reviewed commit, a test result, or a configuration record, and the deployed build matches that commit. Record any unresolved finding with its owner, severity rationale, mitigation, and review date. Do not suppress a finding just to make a dashboard appear clear.
- Two-account and logged-out authorization tests match the documented access matrix.
- Exposed tables, grants, policies, views, functions, and privileged database paths have been reviewed.
- Secrets are absent from client artifacts; exposed credentials have been revoked or rotated.
- Dependencies, code-analysis results, and secret-scan results are reviewed against the release commit.
- The release candidate’s network responses and sensitive operations have been tested, and every high-impact exception has an owner.
A defensible release decision rests on specific evidence and named ownership, not a security score. If a suspected exposure involves sensitive data, credentials, or an ongoing attack, pause the launch and escalate to qualified human support.
