Pathtorch

Our Technical Findings From Auditing 100+ Sites In 2019

Most common technical SEO issues

Since January we've audited over a hundred sites across different industries and sectors... And we made sure to track every issue we've encountered.

At the start of this year, I made the mental note to keep track of every issue that I encountered on peoples sites.

This is useful for us as it helps us get a better idea of what items actually make a difference to peoples sites.

What correlates with positive improvement, etc.

As this is our very first blog post on the site, I wanted to share some of our findings with the wider community.

In the analysis I'll only be including issues that we encountered on over 20% of sites.

This is because due to our limited dataset, anything under that just isn't significant enough to warrant including.


🔦 Want to see how your site compares?

Request a quote

Over 80% Of Sites Affected

Issue We Encountered:Percentage Of Sites:
Internal Redirected URLs85.71%
Has Only One Followed Internal Linking URL85.71%
External Redirected URLs80.95%
Broken External URL80.95%

It's common knowledge to us that most SEOs don't take redirects seriously. So this should come as no surprise.

However, the "One Followed Internal Linking URL" is a bigger issue. As we only apply our crawlers to check this on important pages...

This means one of the most powerful things you can do to improve the rankings of a page is being ignored on certain pages by 80% of people!

Breakdown

Internal Redirected URLs

85.71%
Why this matters

Most internally redirected URLs are done so with a .301 redirect. While this isn't technically an error, it is an issue. When it comes to redirects you lose some of the link equity or "juice" that would pass between an ordinary link. Redirects also increase the load time of a page, and generally represent a crawl issue.

Has Only One Followed Internal Linking URL

85.71%
Why this matters

This represents an issue, but only in context... We're not talking about links to "special" pages. We're talking specifically about valuable indexed pieces of content that people want to rank. One internal link is not enough for most pages to rank with in competitive SERPs (Search Engine Ranking Pages).

External Redirected URLs

80.95%
Why this matters

Earlier this year I wrote up some findings from a case-study I did on my Medium blog... In this case-study the only thing we changed was switching external redirected URLs to "NoFollow". The result was over 1259% growth over the course of a few months. It's safe to say, this matters a lot and it's shocking that over 80% of sites are suffering from this issue in varying degrees.

Broken External URL

80.95%
Why this matters

It's often the case that you will link to a site, and they will eventually remove the URL from their site or the site may simply cease to exist. In these cases it's common that a redirect isn't setup by the linked site. So the link simply becomes broken.

Over 50% Of Sites Affected

Issue We Encountered:Percentage Of Sites:
Sitemap/s Missing From Robots.txt61.90%
No Image Compression In Use61.90%
Contains One Or More Point Of Failure57.14%
Broken Internal URLs52.38%
URL Is Orphaned52.38%

My main takeaways from this array of findings is that over half the sites still aren't compressing their images. I feel like this should be standard practice by now.

The next big issue is the points of failure issue... Most crawling tools can also show you this error, as it's relatively simple to check. So the issue here is that a big percentage of people don't take this issue seriously enough. Check the breakdown to see why it's so important.

Breakdown

Sitemap/s Missing From Robots.txt

61.90%
Why this matters

While it's not mission critical to include your sitemap in your robots.txt file, it seems like a good practice that should be taken. Interestingly, we most often find that people are forgetting to include ALL of their sitemaps in the robots.txt file.

No Image Compression In Use

61.90%
Why this matters

The single biggest thing that affects page size is images... The compression industry is huge, and is responsible for saving big businesses billions of dollars per year by improving the load times of pages. The fact that over 60% aren't doing this is absolute madness as there are lots of free compression tools on the market today.

Contains One Or More Point Of Failure

57.14%
Why this matters

This issue is one we encounter when a script or more than one script is failing to load on a page. This essentially means that parts of peoples sites are broken. In this case a significant 57% of peoples sites aren't working as intended. This is often due to plugin conflicts, broken requests to 3rd-party scripts etc.

Broken Internal URLs

52.38%
Why this matters

This is an issue that causes problems in multiple different ways. Perhaps most importantly, it's something that I firmly believe effects the overall quality score of your page (which probably goes into an overall site quality score).

URL Is Orphaned

52.38%
Why this matters

Orphaned URLs aren't good for anyone... This means that you have a page on your site that is not directly accessible via your site structure. Usually these are found in sitemaps. This represents a missed opportunity for improving the rankings on that page, or an issue with your sitemap not being properly maintained.

Over 20% Of Sites Affected

Issue We Encountered:Percentage Of Sites:
External URL Redirect Broken (4XX or 5XX)38.10%
URL Receives Both Follow And NoFollow Internal Links33.33%
Index Bloat33.33%
Crawler Traps33.33%
Redirect (3XX) URL In XML Sitemaps28.57%
Mixed Content Warnings28.57%
/ and non / conventions accessible28.57%
Has External Opener Links Vulnerable To Tabnapping28.57%

Each and every one of these issues is important, though none more so right now than the Mixed Content Warnings.

The writing has been on the wall for a while, but we actually got official confirmation that mixed content will soon be blocked on Google Chrome by default.

The issue here is that images, scripts and other important parts of your site could cease to work for visitors using Google Chrome.

This will definitely harm conversions and potentially rankings as well since Google are now using an Evergreen version of Chrome for their crawler. Meaning if the visitor can't see it, Google won't be able to "render" it either.

Breakdown

External URL Redirect Broken (4XX or 5XX)

38.10%
Why this matters

This issue often goes hand in hand with the External Redirected URL issue, though not in the case of every link. Our advice is always set NoFollow to external links that are likely to or change regularly as you'll always suffer from redirects and broken redirects in these cases.

URL Receives Both Follow And NoFollow Internal Links

33.33%
Why this matters

I consider this an issue as it's simply sending mixed messages to the search engines... You should be consistent with your internal linking directives.

Index Bloat

33.33%
Why this matters

This has become a bigger issue over the last few years, since the introduction of certain updates, we've become firm believers in an overall quality score for your site... The web pages you index definitely play a bigger part in this score. We've found it's become vital to only index pages that offer value to a searcher. So during our technical SEO audit we have been checking the index for junk pages that "bloat" your sites overall index. A total of a third of sites have one or more non-desirable pages indexed. We suspect this would be much higher if our data wasn't skewed by our higher than average percentage of SEO customers.

Crawler Traps

33.33%
Why this matters

Crawler traps are links on your site, usually stashed in the meta areas or head area of your site that get crawled but don't need to be there. These vary from platform to platform. WordPress for example has several crawler traps that we're aware of which aren't suitable for all types of sites. Remember, WordPress was designed as a blogging platform. Not as a CMS (Content Management System) for businesses and ecommerce sites. It's grown into that, but automattic are notoriously stubborn about implementing SEO related changes to the platform.

Redirect (3XX) URL In XML Sitemaps

28.57%
Why this matters

This represents an issue with your sitemap displaying URLs that should no longer be there. We've already discussed in this article some of the reasons why redirects aren't a great thing for your site.

Mixed Content Warnings

28.57%
Why this matters

The worrying thing here is that it's just been announced that Google Chrome will soon stop displaying Mixed Content altogether. Based on our own findings and more savvy than most customer base, we're anticipating this will mean a lot of sites are going to "break".

/ and non / conventions accessible

28.57%
Why this matters

This issue occurs when there is an issue in the server config or often in the .htaccess file (apache servers, etc)... While it's not often a huge issue due to the usage of canonicals - not all sites correctly use canonicals. In these cases, where internal linking is not strict and relative URL protocols are used. You're essentially creating an exact duplicate of your site. All it takes is a single link in this scenario to create this problem and allow a crawler in.

Has External Opener Links Vulnerable To Tabnapping

28.57%
Why this matters

This problem is still fairly common despite the development community being pretty firm on this... External opener links that aren't coded appropriately can cause major issues. It will almost certainly go into any kind of site quality score algorithm as it's easy to check this for a crawler.


🔦 Want to see how your site compares?

Request a quote

Conclusion

Even from a hundred sites, we've seen clear patterns arise regarding certain commonly missed issues.

We've also personally learned that despite the bravado we get in the SEO industry; that a lot of people are still making basic mistakes.

We believe that the fundamentals matter. Not just because of the results we've had with helping people fix up their sites... But because we believe that doing the right thing is important, even when it requires more effort.

It also just so happens that effort usually wins the day. No bad reward for the extra time it takes to perfect your technical foundations.


Thanks for reading.


Share this post