SEO (Search Engine Optimization) is an ever changing art. Even the most well versed in SEO can feel overwhelmed trying keep pace with the changes. Luckily, there are a few things in SEO that never change and ensuring your site can be crawled is one of them.
According to the infallible source known as Wikipedia (their definition is actually correct this time) a Web crawler, sometimes called a spider, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites’ webcontent. In other terms it is what Google and a hundred other companies use to read and index your site.
Close your eyes when… NOT YET… You can’t read this blog with your eyes closed. Unless of course you have some sort of text to speech program, which would be really cool, but I digress. Close your eyes when I tell you to do so. Once your eyes are closed, imagine you were born without sight. You have never seen color or light. The only thing you know about the world is what you can sense without sight. Go ahead and close your eyes.
If you have fallen asleep or raged quit out of my blog… you probably will never understand why we just did that. The reason we closed our eyes was to try and understand how someone lacking a basic sense that we take for granted could understand the world around us, was so we could start to understand how web crawlers experienced our pages. If you handed a blind person a photograph and asked them to describe it they would only be able to describe the material it was printed on. They can’t see the photo so you would need to describe to them what the photo is of. The same thing rings true for web crawlers. Graphics and photos are perfect for creating dynamic looking pages, but to Google they are just a picture if you haven’t added a description to them.
Now imagine you have invited a blind friend to your house. They ask you where your bathroom is and you tell them it is 4 feet to their right. They go to the exact spot you told them and find nothing but a solid wall. They probably would laugh thinking you were either messing with them or you are just terrible at directions. So you apologize and tell them you meant it was 4 feet to their left. They do an about face and head 8 feet in the opposite direction only to find another solid wall. This time they are probably not going to be in the best of moods. Web crawlers are told where to go through your links…. enough broken links and the web crawlers start to dislike your website.
Now that you understand how these crawly thingies worky, we can start learning how to keep them happy. Here is a list of the do’s and dont’s when taking crawlability into consideration.
Just as I mentioned above, crawlers HATE broken links. It tells them that your site is not well maintained and you should be indexed below other sites who are actually up to date. And, just like a crawler… I also HATE broken links. If I visit your site and you have a broken link I will force you to watch the final episode of How I Met Your Mother… over and over and over.
NO DESCRIPTIONS –
Ever wonder why almost every single website creation tool on the planet has an area where you can describe the picture you just posted? It is because web crawlers need that description so they know what they are looking at. You could have the most amazing blog about penguins that has glorious photos of those tuxedo wearing birds plastered all over the place, but if you do not tell Google that it is looking at an adorable baby penguin having a flipper fight to the death with a bald beaver it would never know you have a picture of such prestige on your site.
PRIVATE PAGES –
Everyone is all excited that Karen is 7 months preggers and her baby shower is tomorrow. You sit politely listening to how her husband knows the sex of the baby, but she doesn’t want to know until she see’s it in her arms. You start to rock ever so slightly as you feel the rage building within. You are not angry that she is having a baby, you don’t even mind her excitedly telling everyone about every single detail related to her pregnancy…. no… the intense fury building deep in your gut stems from the fact that in your trunk, at this very moment, lies a pair of little booties purchased as a gift for a baby shower that you never received an invitation for.
Private pages are like baby showers. You have to be invited to get in. Web Crawlers are not fans of private pages. If you do have any private pages you need to set them to “nofollow”.
By the way Karen…. it is a boy.
BROKEN INTERNAL LINKS –
A chain is only as strong as its weakest link… blah blah blah… yadda yadda yadda… Guess what! If a long chain has a broken link it is now a short chain. It is no longer usable for whatever manly task you were using it for. Your website has internal linkage. If those links are broken it has a massive effect on how crawlers perceive your site. It makes it more difficult for them to navigate your site and they see your site as less useful.
SLOW LOADING PAGES –
Do you remember back in the old days when you were trying to view a picture of your favorite TV star in as little clothing as possible? Depending on your preference, you were either waiting for AC Slater’s oily hair to load line by line or you were on the edge of your seat while the fringe on Kelly Kapowski’s cutoff jean shorts was starting to take shape. Back in the 90’s this was our life… and it was a good life, but things have changed. There is nothing worth waiting for in today’s society. Everything must be delivered promptly. This includes your website. If your site is taking too long to load (we are talking seconds here folks) the web crawlers are going to get impatient. It will really hurt your rankings.
These are all basic things you can do to keep your website crawler friendly. There are many more things you can do to help, but you honestly don’t have the time to go over each one. There are many more technical things that can be done and we may cover those in a future blog, but for today we are sticking with the basics. If you can at least cover these basics you will have an edge over those who don’t. On a side note…. No Karen, I have not seen the pictures of your baby you posted on Facebook yesterday…. I blocked your posts on my Facebook ages ago!