Sheet IV of VI

Canonical URLs and duplicate content

When the same content can be reached at more than one URL, Google groups the duplicates and picks one, the canonical URL, as the version to show and evaluate. You can tell Google which one you prefer with redirects, rel="canonical" or a sitemap, but your preference is “a hint, not a rule”.

General information from Google’s own documentation, not advice for your particular site. Google’s Search Central documentation is the place to check before you change anything, and it is updated over time.

On a surveyed plan, one acre has one number. Websites drift away from that: the same page answers at http and https, with and without www, with a tracking tag on the end. Canonicalisation is how Google puts one number back on the acre.

Where duplicates come from

Google lists the usual reasons a site ends up with duplicate content:

Region variants, such as separate US and UK pages with essentially the same content in the same language;
Device variants, a mobile and a desktop version of a page;
Protocol variants, the HTTP and HTTPS versions of a site;
Site functions, such as sorting and filtering a category page;
Accidental variants, such as a demo version left open to crawlers.

Language versions count as duplicates only when the main content is in the same language, for example when only the header and footer are translated.

It is not a penalty

Some duplicate content is normal and doesn’t break Google’s spam policies. Google’s starter guide answers the worry in its own words: if some content is reachable under multiple URLs, “it’s fine; don’t fret about it”, and it adds, “Copying others’ content, however, is a different story.” The real costs are a confusing experience, harder tracking of how content performs, and crawling spent on copies.

How Google chooses

When Google indexes a page it works out the page’s primary content. If several pages look the same or very similar, it clusters them and picks the one it judges most complete and useful for searchers as canonical. The canonical is crawled most regularly; the duplicates less often. Signals include whether a page is served over HTTP or HTTPS, redirects, presence in a sitemap, and rel="canonical" annotations.

A search result usually points to the canonical, unless a duplicate suits the searcher better: on a phone, for instance, the result will probably point to the mobile page even if the desktop page is canonical.

The signals, by strength

Redirect
A strong signal that the target should become canonical. Use it when retiring a duplicate.
rel="canonical"
A strong signal, in the HTML <head> or as an HTTP header.
Sitemap inclusion
A weak signal; every URL listed is suggested as a canonical.

Google’s order. The methods stack: using two or more that agree increases the chance your preferred URL is shown.

None of this is required. Google says most sites will likely do fine without stating a preference, because it will pick the best version itself. It is worth doing when you want to choose the URL people see, consolidate links pointing at variants into one URL, keep your own metrics in one place, or save crawling for new pages.

The methods side by side

Canonical methods compared, from Google’s documentation
Method	Good for	Watch for
`rel="canonical"` link element	Any number of duplicate HTML pages	Hard to maintain where URLs change often; only works in the `<head>` of HTML pages.
`rel="canonical"` HTTP header	Non-HTML files such as PDFs; doesn’t add page weight	Same maintenance burden as the element.
Sitemap	Large sites; simple to maintain	A weaker signal; Google still has to work out the duplicates.
Permanent redirect	Getting rid of a duplicate page	Use only when deprecating the duplicate.

A canonical element, on an invented page

<head>
  <title>Kangaroo paw</title>
  <link rel="canonical" href="https://www.example.com/plants/natives/kangaroo-paw">
</head>

That line goes on each duplicate, and Google recommends putting the same self-referencing line on the canonical page too. Use an absolute URL: relative ones are supported but can cause problems later, for instance if a test copy of the site gets crawled. Pick one of the element or the header, not both, since mixing them makes it easy to name two different URLs.

Google advises

Link internally to the canonical URL, not a duplicate.
Redirect HTTP to HTTPS, or point HTTP pages at their HTTPS versions with rel="canonical".
With JavaScript sites, put the canonical in the HTML source and don’t let scripts change it.
With hreflang, name a canonical in the same language.

Google advises against

Using robots.txt to canonicalise; disallowed URLs can still be indexed without their content.
Using the URL removal tool, which hides every version.
Naming one URL in a sitemap and another in rel="canonical" for the same page.
Using noindex to choose between pages on one site, or a URL fragment as the canonical.

HTTPS is preferred, unless something contradicts it

Google prefers an HTTPS page over its HTTP twin as canonical, except when the HTTPS page has an invalid certificate, loads insecure dependencies other than images, redirects to or through HTTP, or carries a rel="canonical" pointing to HTTP. Bad certificates and HTTPS-to-HTTP redirects make Google prefer HTTP very strongly, and HSTS can’t override that.

When Google picks a different page

See what Google chose. The URL Inspection tool shows the canonical Google selected. Ask whether its choice actually suits searchers better than yours.
Look for technical causes. Google names incorrect canonical elements from a CMS or plugin, misconfigured servers, missing hreflang on localised sites, and hacked sites injecting redirects or canonicals.
Make clustered pages genuinely different. If pages are grouped as duplicates but shouldn’t be, the fix is making them sufficiently different; clear, significant differences split out faster.
Allow time. Google may hold pages in a duplicate cluster for up to two weeks after content is fixed. You can then use Request Indexing, which has quotas, for the most important URLs.

For syndicated articles, Google says the canonical element isn’t recommended, because partner pages are often very different; the effective fix is for partners to block indexing of your content.

Redirects themselves are the subject of the sheet on migrations and redirects; sitemaps are in the sitemap sheet.