A useful technical SEO audit starts with a question, not a tool list. Use Google Search Console to understand Google’s reported indexing and search data, a crawler such as Screaming Frog to inspect URLs and links at scale, and PageSpeed Insights to examine performance. Add Siteliner for a limited duplicate-content review or Website Grader for a quick homepage baseline.
For example, a product page may return HTTP 200 in a crawl while Search Console reports that Google has not indexed it. Those are two valid observations from different systems. Save both, investigate the conflict, and do not treat either tool as the whole answer.
Technical SEO removes barriers to crawling, rendering, and indexing. It does not guarantee that a page will be indexed or rank. Before auditing, separate three questions: can Google fetch the URL, has Google reported it as indexed, and is the page competitive in search? Google’s technical requirements describe eligibility, not a promise of indexing or rankings.
Which technical SEO tools should you use?
Choose a tool by the evidence you need. Search Console is Google’s view of a verified property. A crawler reports what it found during a configured crawl. PageSpeed Insights combines Lighthouse lab diagnostics with real-user data when available. None of those outputs is a ranking prediction.
| Audit question | Best-fit tool | Evidence produced | Decision owner |
|---|---|---|---|
| Has Google indexed this URL? | Search Console URL Inspection | Google-reported URL and crawl information | SEO lead |
| What site-wide crawl issues exist? | Screaming Frog SEO Spider | Statuses, links, redirects, directives, canonicals, and metadata | SEO analyst with developer review |
| What does page performance look like? | PageSpeed Insights | Lighthouse diagnostics and available CrUX field data | Web performance owner |
| Are publicly accessible pages similar? | Siteliner | Limited internal duplicate-content and broken-link review | SEO and content owners |
| What is a quick homepage baseline? | HubSpot Website Grader | Broad homepage-level performance, SEO, mobile, and security signals | Site owner |
Match each tool to the evidence it can produce, then require a separate decision about what the evidence means.
Website Grader is a quick baseline, not a full-site crawler. Screaming Frog’s free edition is limited to 500 URLs per crawl, and edition and configuration affect available features. Siteliner Free analyzes up to 250 prominent pages and is generally limited to one scan per site every 30 days. These limits make the tools useful for different scopes, not interchangeable audit results.
Start with Google Search Console for Google’s view
Verify ownership of the relevant Search Console property. Then review indexing and issue reports, inspect representative URLs with URL Inspection, and check sitemap status. URL Inspection provides Google-specific information about an individual URL, but it is not a site-wide crawler for internal links, metadata, redirects, or every page template. Google’s Search Console overview describes the available reports and inspection workflow.
A sitemap helps Google discover URLs, but submitting one does not force crawling or indexing. Include URLs intended for search visibility rather than every generated URL. Use Google’s sitemap guidance when deciding what belongs in the file.
For an important URL, compare URL Inspection with the crawler’s status, canonical, robots observations, and sitemap membership. Preserve disagreements as conflicts to investigate. If you collect Search Analytics through the API, treat each result as an aggregate for the requested property, date range, dimensions, and filters. It is not a complete event-level export of every query or impression. Google’s Search Analytics reference documents query behavior and limits.
Use a crawler to find site-wide issues
A crawler follows URLs within a defined scope and can surface broken links, redirects and redirect chains, duplicate metadata, robots directives, canonical declarations, sitemap inconsistencies, and, when configured, JavaScript-rendered content. Screaming Frog documents these capabilities on its SEO Spider product page.
Before crawling, record the seed URL or sitemap, scope, exclusions, user agent, authentication settings, and rendering configuration. For complex pages, compare raw HTML with rendered findings where relevant. Export findings with a crawl ID so a later crawl becomes a separate observation rather than silently replacing the earlier one.
A discovered URL is not proof that Google indexed it. Similarly, a duplicate title is an audit signal, not evidence of a ranking penalty. Compare priority URLs with Search Console and investigate differences in status, robots directives, canonical signals, internal links, and rendered content.
Make 404, redirect, robots, and canonical decisions deliberately
Diagnose the intended outcome before changing a URL. If removed content has no suitable replacement, keep its 404 response or return 410. If the content has moved to a genuinely relevant replacement, consider a 301 redirect. Do not redirect every obsolete URL to the homepage. A custom 404 page also does not replace the correct HTTP status. Google’s crawling-error guidance explains these response choices.
If a page intended for search is blocked by robots.txt or carries a noindex directive, confirm the indexing policy before changing it. A robots.txt block controls crawling and is not the same as noindex. Canonical declarations are signals, not absolute instructions. When Google selects a different canonical, check content similarity, redirects, internal links, and other signals before deciding there is an error.
Duplicate content is not automatically a Google penalty. Practical concerns can include conflicting signals, inefficient crawling, and an unintended canonical selection. Use Google’s canonicalization troubleshooting guidance when reviewing the result.
A 404 is a diagnosis, not an instruction to redirect. Keep it when no relevant replacement exists, and request a redirect review only when the destination genuinely replaces the old resource.
Measure speed and mobile experience with current evidence
PageSpeed Insights combines Lighthouse lab diagnostics with Chrome User Experience Report, or CrUX, field data when sufficient data is available. Field data represents a trailing 28-day period and may be unavailable for a URL with insufficient coverage. Current Core Web Vitals are Largest Contentful Paint, Cumulative Layout Shift, and Interaction to Next Paint. Google evaluates relevant field-data aggregations at the 75th percentile.
A single lab run and field data are different observations. Field data may be page-level or origin-level, while lab results are produced under a test strategy and set of conditions. Use lab findings to investigate potential improvements and field data to understand real-user experience when available. Performance work can improve user experience and may support search performance, but a higher PageSpeed score does not guarantee a ranking gain.
For mobile audits, check that the mobile version retains important content, metadata, links, structured data, and robots directives. Google’s mobile-first indexing guidance explains why mobile content parity matters. Browser and device testing can reveal visual or interaction problems, but it does not reproduce Google’s entire rendering and indexing process.
- Record whether the test used mobile or desktop strategy.
- Save the test time, run identifier, and release or template identifier when relevant.
- Label missing CrUX field data as unavailable, not as a failure.
- Keep lab diagnostics separate from field metrics and note page-level or origin-level scope.
- Check mobile content, links, structured data, and indexing directives on priority templates.
Use duplicate-content tools for investigation, not verdicts
Siteliner scans publicly accessible internal pages and reports duplicate content and broken links. Its free service analyzes up to 250 prominent pages and generally allows one scan per site every 30 days. Premium supports larger scans, subject to account and billing conditions. The free scan is not a complete inventory of a large site and does not cover login-protected pages.
Review reported page pairs or clusters and ask whether they serve the same user need. Shared navigation and boilerplate do not prove that two pages are duplicates in purpose. Depending on the intended URL structure, the decision may be to retain both pages, consolidate content, redirect one URL, or use a canonical. Verify canonical signals with Search Console and Google’s guidance rather than treating a duplicate percentage as a Google metric.
Turn findings into a traceable audit workflow
Use an issue tracker, spreadsheet, database, or data warehouse as the system of record. The workflow below is a proposed implementation design, not a built-in integration or vendor-provided schema. AI is optional and should summarize collected evidence or suggest an issue category only after deterministic checks have produced the underlying facts.
| Signal or trigger | Bounded AI job | Validation and decision gate | Action and owner |
|---|---|---|---|
| 404 or 410 in a crawl | Rank possible replacements and explain the evidence | Confirm a materially relevant replacement and check internal links | SEO or content owner approves; developer deploys |
| Declared and selected canonical differ | Summarize possible causes from recorded evidence | Check content, redirects, internal links, and intended URL structure | SEO lead creates or closes a technical ticket |
| PageSpeed lab finding | Turn documented diagnostics into a developer-ready summary | Separate lab results from field data and retest after the change | Performance owner manages the release ticket |
| Similar page pair in a scan | Summarize overlap and distinct user intent with source URLs | Review purpose and canonical signals before consolidation | SEO and content owners choose the URL action |
The AI output is a draft explanation, not approval. Status codes, URL normalization, redirect-chain detection, duplicate metadata checks, robots directives, canonical comparisons, sitemap membership, and Core Web Vitals threshold labels should remain deterministic. Redirects, robots rules, noindex directives, canonicals, and sitemap exclusions require human approval before deployment.
Use an observation contract that preserves grain
Every record needs a declared measurement grain. A crawler observation is one URL in one crawl at one observation time. A PageSpeed observation is one URL, one run, one strategy, and one test configuration. A Search Analytics row is an aggregate for a property, date range, search type, dimension set, filter set, and returned dimension values. A citation is a separate source record attached to an observation or recommendation.
The following JSON is an illustrative implementation design, not a Screaming Frog schema. Its proposed key identifies one URL in one crawl. If the same URL appears in another crawl, it creates another observation rather than overwriting history.
{
"crawl_id": "crawl_2026-10-13_001",
"normalized_url": "https://www.example.com/old-guide/",
"observed_at": "2026-10-13T12:00:00Z",
"source_tool": "Screaming Frog",
"issue_type": "not_found",
"http_status": 404,
"source_report_url": "https://audit.example.com/reports/crawl_2026-10-13_001",
"recommended_action": "request_301_review",
"owner": "SEO lead",
"approval_status": "pending"
}For this record, the proposed unique key is (crawl_id, normalized_url). For PageSpeed, include the run ID, normalized URL, strategy, locale or test configuration, and analysis timestamp. For Search Analytics, retain the property, date range, search type, aggregation, dimensions, filters, and returned dimension values. For citations, use a separate proposed key such as (observation_id, source_url, source_role) so multiple citations do not collapse into one field.
When concurrent workers can process the same run, enforce uniqueness in the database and use a transactional upsert where supported. A lookup followed by an insert can race. Keep the original observation and source report link; store approval and resolution separately rather than overwriting evidence. Validate URL normalization, allowed status and strategy values, issue types, and timestamps before writing.
Measure completion by whether an approved change is deployed and the expected technical condition is verified in a later observation, not by a tool score alone. Teams formalizing ownership and evidence review can explore workflow and automation consulting.
Frequently asked questions about technical SEO tools
Which tool checks whether Google indexed a URL?
Use Search Console URL Inspection for Google-reported information about an individual URL. A crawler cannot prove that Google indexed it.
Is Google’s standalone Mobile-Friendly Test the current recommended workflow?
Do not rely on the legacy standalone test. Check the mobile version under current mobile-first indexing guidance, supported by browser and device testing.
Does missing PageSpeed field data mean a page failed Core Web Vitals?
No. CrUX data may be unavailable when there is insufficient coverage. Report it as unavailable, not as a failure.
Can Search Console API results represent every query and impression?
No. Search Analytics returns aggregated rows subject to documented limits, not a complete event-level export.
Should every 404 be redirected?
No. Keep a 404 or use 410 when no suitable replacement exists. Consider a 301 only when a relevant destination genuinely replaces the old resource.