What is a technical SEO audit and what problems does it detect?
A technical SEO audit is a systematic analysis of the technical factors that affect the crawling, indexation and ranking of a website. It detects robots.txt errors, canonicalisation problems, performance issues (Core Web Vitals), duplicate URLs and schema errors. According to Screaming Frog, 75% of audited sites have at least one critical indexation problem.
Key takeaways
- 75% of websites have critical indexation problems detectable with crawling tools such as Screaming Frog or Sitebulb.
- Core Web Vitals (LCP <2.5s, INP <200ms, CLS <0.1) are confirmed ranking signals by Google since May 2021.
- A complete SEO audit covers 5 areas: crawling and indexation, URL architecture, performance, structured data and security (HTTPS).
- Google Search Console is the most reliable free tool for detecting index coverage errors and Core Web Vitals problems in real field data.
- Web migrations are the highest-risk SEO event: without a prior audit, 35-40% of sites experience traffic drops exceeding 20%.
A technical SEO audit is not a report. It is a diagnostic process. The report is just what gets handed over at the end — the result of crawling, analysing and prioritising. What separates a useful audit from a list of automated warnings is knowing which of those warnings actually limit organic performance and which are noise.
Screaming Frog will surface 400 issues on a typical 5,000-URL site. Amongst them, three or four will be causing most of the damage. In practice, identifying those three or four requires understanding the relationship between crawl architecture, indexation, rendering and Core Web Vitals — not just knowing which tool to run.
This guide walks through a professional audit methodology from initial crawl to prioritised remediation, with tool recommendations and data interpretation at each stage.
What is a Technical SEO Audit and Professional Methodology
Fundamentals of an effective technical audit
A solid technical foundation is the basis of any successful ranking strategy:
- Makes it easier for search engines to discover and traverse all your important pages.
- Ensures the correct pages are included in search indexes.
- Improves user experience and search engine ranking.
- Identifies and corrects technical problems that could result in sanctions.
- Guarantees optimal performance on mobile devices, especially important in the mobile-first indexing era.
Tools and access required
To perform a complete technical audit, you will need:
Analysis tools
- Screaming Frog SEO Spider, Sitebulb, DeepCrawl
- Google Search Console, PageSpeed Insights, Mobile-Friendly Test, Rich Results Test
- Ahrefs or Semrush, GTmetrix, WebPageTest
Access required
- Google Search Console access
- Google Analytics access
- Server/hosting credentials
- CMS administration panel access
- Web server logs (especially useful)
Technical Analysis: Crawling, Structure and Performance
Crawling and indexation
Check how search engines explore your site:
-
robots.txt analysis
- Verify it does not block important content
- Review the file’s location and accessibility
-
noindex/nofollow directives review
- Identify pages with inadequate meta robots
- Check rel=“nofollow” attributes on important internal links
-
Server log analysis
- Observe bot crawling patterns
- Identify visit frequency and ignored pages
- Detect 4XX and 5XX errors during crawling
# Example server log line analysis for Apache
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
# Example line for Google bot
66.249.66.1 - - [02/Apr/2024:10:15:23 +0200] "GET /product/sports-shoes HTTP/1.1" 200 24716 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
URL structure analysis
Evaluate the structure and format of your URLs:
-
User-friendly URL pattern
- Descriptive and readable
- Appropriate use of keywords
- Appropriate length (preferably <100 characters)
-
URL parameters
- Identification of redundant parameters
- Management in Google Search Console
- Canonical implementation for duplicate versions
-
URL consistency
- Consistent use of www vs non-www
- HTTP vs HTTPS protocol
- Trailing slashes
Performance, Core Web Vitals and optimisation
Analyse key user experience metrics:
- Largest Contentful Paint (LCP): Identify the largest element and its loading time, solutions (preloading, image optimisation, CDN)
- First Input Delay (FID)/INP: Measure response to interactions, JavaScript optimisation and main thread tasks
- Cumulative Layout Shift (CLS): Detect elements causing visual shifts, implementation of explicit dimensions
Site speed optimisation
-
Load time by device
- Mobile vs desktop comparison
- Analysis by page type (home, category, product, etc.)
-
Resource optimisation
- Image compression
- CSS/JS minification
- Lazy loading
- Browser caching
- Removal of blocking resources
<!-- Example of lazy loading and size implementation for images -->
<img src="small-image.jpg"
data-src="full-image.jpg"
loading="lazy"
width="800"
height="600"
alt="Image description">
Architecture, navigation and duplicate content
Evaluate how pages connect to each other:
- Navigation depth: Pages more than 3-4 clicks from the homepage, distribution of internal link value
- Internal links: Quality and relevance, use of descriptive anchors, orphan pages
- Information hierarchy: Category/subcategory structure, topical siloing, breadcrumbs
Mobile optimisation
- Responsive design vs mobile site: Consistency between versions, correct technical implementation
- Mobile-first indexing: Content parity between mobile and desktop, optimised mobile experience
Duplicate content management
-
Internal: Pages with similar or identical content, taxonomies generating duplicates (tags, categories, etc.), filtering and sorting parameters
-
External: Duplicate content between domains, cross-domain implementation
-
Implemented solutions: Canonicals, Hreflang for languages/regions, appropriate redirects
Metadata, structured data and security
Review the implementation of meta elements:
- Title tags and meta descriptions: Duplicates, missing or inadequate, optimal length, inclusion of relevant keywords
- Schema.org: Correct implementation, appropriate types according to content, validation in Rich Results Test
<!-- Example of Schema.org implementation for a product -->
<script type="application/ld+json">
{
"@context": "https://schema.org/",
"@type": "Product",
"name": "Ultra-lightweight sports shoes",
"image": "https://example.com/images/shoes.jpg",
"description": "Perfect sports shoes for running, ultra-lightweight with maximum cushioning.",
"brand": {
"@type": "Brand",
"name": "BrandName"
},
"offers": {
"@type": "Offer",
"url": "https://example.com/product/sports-shoes",
"priceCurrency": "EUR",
"price": "89.99",
"availability": "https://schema.org/InStock"
},
"aggregateRating": {
"@type": "AggregateRating",
"ratingValue": "4.8",
"reviewCount": "47"
}
}
</script>
Security and HTTPS implementation
- SSL certificate: Validity and expiry, certificate type, mixed content problems
- Redirects: HTTP to HTTPS correctly implemented, redirect chains avoided
International configuration (if applicable)
- Hreflang implementation: Correct syntax and reciprocity, consistency between signals
- Domain/subdomain/subdirectory configuration: Structure chosen according to needs
JavaScript and rendering
- Critical vs non-critical JavaScript: Excessive dependency on JS for important content, rendering problems detected in GSC
- JavaScript links: Crawlability of dynamically generated links, pushState/replaceState implementation in SPAs
Implementation and Audit Process
Step-by-step implementation process
Configuration and discovery
- Configure Screaming Frog or Sitebulb with crawl limits adjusted to site size, filters to exclude non-relevant sections, and link-following rules.
- Run the full crawl, export the data and compare it against GSC coverage to surface anomalies — pages crawled but not indexed, or URLs in GSC that the crawler cannot find.
Data analysis
- Group findings by problem type, prioritise by potential impact and ease of resolution, and identify patterns by site section.
- For performance, run PageSpeed Insights on representative samples of each page type (home, category, product). Always prioritise field data (CrUX) over lab data.
Documentation and presentation
- Organise findings by category with concrete evidence: screenshots, example URLs, comparative data.
- The action plan should clearly distinguish between critical fixes (crawl blocks, 5xx errors), high-impact optimisations and incremental improvements.
Complete audit checklist
Crawling and indexation
- robots.txt file correctly implemented
- XML sitemap accessible and up to date
- Appropriate meta robots directives
- Important pages allowed for indexation
- Server log analysis performed
Performance
- Core Web Vitals within acceptable ranges
- Image optimisation verified
- CSS/JS resources minified
- Browser cache configured
- Above-the-fold content prioritised
Architecture
- Consistent and optimised URL structure
- Clear hierarchical navigation
- Breadcrumbs implemented
- Internal links optimised
- Orphan pages identified and corrected
Technical content
- Canonical tags correctly implemented
- Titles and meta descriptions optimised
- Heading hierarchy (H1-H6) logical
- Structured data implemented and validated
- Duplicate content identified and managed
Specific aspects
- HTTPS implemented without errors
- Hreflang (if applicable) verified
- JavaScript rendering evaluated
- Geotargeting configuration reviewed
- AMP pages (if applicable) correctly implemented
Case Study and Applied Methodology
To illustrate the practical application of this methodology, we present a real case (with data modified for confidentiality):
An e-commerce with 50,000+ pages saw a 35% organic traffic decline over six months, coinciding with a platform migration from Magento to Shopify.
Main findings
-
Crawling problems
- 42% of important URLs were blocked by robots.txt
- URL pattern changed without adequate redirects
-
Performance problems
- Average LCP of 5.8 seconds on mobile
- High CLS (0.35) due to dynamic banner loading
-
Architecture problems
- Category filters generating millions of crawlable URLs
- Pagination implemented with AJAX without crawlable links
Action plan
-
Phase 1: Critical corrections
- robots.txt reconfiguration
- Implementation of 301 redirects from old URLs
- Banner loading adjustment with reserved dimensions
-
Phase 2: Optimisations
- Implementation of rel=“next/prev” in pagination
- Image optimisation with WebP and appropriate dimensions
- Configuration of canonicalisation rules for filters
Three months later: 85% of lost traffic recovered, Core Web Vitals at optimal levels (LCP <2.5s, CLS <0.1) and a 32% increase in indexed pages.
Conclusions and next steps
A thorough technical audit is a complex but worthwhile process for identifying improvement opportunities and resolving problems that may be limiting your website’s performance in search engines.
The key is to maintain a systematic, data-driven approach prioritised according to potential impact. Remember that technical work is not a one-off task, but a continuous process of optimisation and maintenance.
If a crawl of your site has surfaced any of these issues, the next step is gauging their real scope. Contact our team for a no-obligation technical assessment.
Sources and references
-
Google Search Console Help - Coverage Report (support.google.com)
-
Core Web Vitals - web.dev (web.dev)
-
Google Search Central: Crawl Budget (developers.google.com)
-
Screaming Frog SEO Spider Documentation (screamingfrog.co.uk)
-
PageSpeed Insights (pagespeed.web.dev)
Share this article
If you found this content useful, share it with your colleagues.
Frequently Asked Questions
How often should a technical SEO audit be performed?
For sites with more than 10,000 pages: complete semi-annual audit + monthly monitoring with Google Search Console. For medium-sized sites (1,000-10,000 pages): an annual audit is sufficient. Always perform a complete audit before and after a web migration or redesign.
What is the most recommended tool for technical SEO audits?
Screaming Frog SEO Spider is the industry standard for local crawling (up to 500 URLs free). For large sites, Sitebulb offers advanced visual reports. Always complement with Google Search Console for real Googlebot crawl data and PageSpeed Insights for Core Web Vitals.
What is crawl budget and how does it affect SEO?
Crawl budget is the number of pages Googlebot crawls on a site in a given period. It is optimised by blocking low-value URLs in robots.txt (parameters, filters, duplicate URLs) and improving server response time. Sites with more than 100,000 URLs must prioritise its active management.