indexing

Staging Site indexing

So you got your staging site indexed? Happens to everyone. Here’s a rough guide on fixing it, and suggestions for preventing it. (thought I’d write this up somewhere) The fastest way to get the staging site removed from search is remove it via Search Console. For that, you need to verify ownership via Search Console [1] (ironically, this means you’ll likely have to make it accessible to search engines again, or figure out DNS verification, which isn’t that common but also not that hard).

Trailing slash or not

The “trailing-slash or not” question comes up from time to time, so I thought I’d write something short up. tl;dr: the slash after a hostname or domain name is irrelevant, you can use it or not when referring to the URL, it ends up being the same thing. However, a slash anywhere else is a significant part of the URL and will change the URL if it’s there or not. This is not SEO-specific, but just how websites work :).

429 or 503

Here’s one for fans of the hypertext HTTP protocol – should I use 429 or 503 when the server is overloaded? It used the be that we’d only see 503 as a temporary issue, but nowadays we treat them both about the same. We see both as a temporary issue, and tend to slow down crawling if we see a bunch of them. If they persist for longer and don’t look like temporary problems anymore, we tend to start dropping those URLs from our index (until we can recrawl them normally again).

503s

Dear webmasters, if something goes drastically wrong with your hoster, and you can’t host your website anymore, please return a “503 Service unavailable” HTTP result code. Doing so helps search engines to understand what’s up – they’re generally more than happy to give your site some time to catch up again. Returning an error page with “200 OK” will result in us indexing the change of content like that (and if all of your pages return the same error page, then we may assume that these URLs are duplicates).

Twitter indexing peculiarities

This post has one main reason: popular sites don’t always get it right. You can also turn that around: you don’t have to get everything right in order to be popular. Never do something on your site just because a large site does it like that. Combine web 2.0 with a search engines, what do you get? Lots of rel=nofollow links (archive.org) :), heh. You’d assume that they could get a few things right with regards to search engine optimization though.