This is a long rant about Google and the trouble they have caused ThereArePlaces. You really do not need to read the whole, sordid tale, but it made us feel much better writing it.
It’s been a tough couple of weeks here at ThereArePlaces. Last week Google rejiggered its search index to get rid of low quality, content houses that were masquerading as high quality content websites with the intent on capitalizing on advertising that their high index positions brought them. For some reason, unknown to us, ThereArePlaces was a casualty in the process and several of our listings dropped on Google’s search engine results pages for our most popular destinations, including Italy – one of our finest sections that rivals anything available anyplace else on the web.
We hope you know, that we spend a great deal of effort creating quality content for ThereArePlaces and spend a great deal of time and money field researching our destinations and our travel advice. Hopefully, over time, we will be able to reclaim our former top rankings, but with Google, you never know what will happen.
Since our site traffic was down significantly due to the Google rejiggering, we decided it was a good time to solve another problem that was related to Google and no other search engine.
Sometime after we launched ThereArePlaces in 2004 we learned that we would need to create a sitemap for Google and were told that it was an important component of Google’s site referencing. A sitemap is a document that includes the URL to every item on your website, indicating its location in the file structure, as well as the date it was last modified.
At the time, we did not have the software to create a sitemap, but learned that our web host did and asked them to create one for us. They did and we submitted it. About a week later, we began to see odd errors in our web logs. Apparently the software from our web host was improperly quality assured and had an error in it. We notified them and they immediately fixed the problem and generated a new, correct site map that we resubmitted to Google. Unfortunately, that began a multiple year game of Whack-A-Mole.
The errors in the original software resulted in an erroneous URL in the sitemap for a critical folder in our website structure where we store most of our destination guides. Part of the URL should have appeared as “/Guidebook/pdest/” , but, due to the error it appeared as “/Guidebook%5Cpdest/”. This form of the URL, which was used in Google’s search index, would show the correct page on our site, but the links to images and photos on it were broken, that is, they did not appear on the page. In addition, the links on the page to any other page in our website were also broken, in the sense that any pages linked to by a broken page would not show the photos designed to be displayed. Note that the broken URLS did not appear on our website, or on any page in our website, but only in the sitemap that was public only for a week or so, many years ago. Since that time, using new sitemap software that we validated as working correctly, we have submitted several hundred updated versions of our sitemap (each time we add or edit a part of our website) and all of the URLs (over 10,000 of them) have been correct for this period of time.
Those of you who are not web publishers probably do not know this, but there is no way for us to directly contact Google about problems such as these. The best you can do is leave a message at the public help forum for webmasters and hope that someone from Google might see it and take pity on you. We left many messages on the Google webmasters forum, but to no avail. Eventually the problem cleared up – or so we thought. However, several months later it reappeared, then disappeared and reappeared and has gone through this cycle for the last several years.
How is this possible?
We do not actually know. But we suspect that some other website (or search engine) copied the broken listing from Google and used it in their directory. Google then “discovered” it (again) when they spidered this directory on the other website and used this “newly discovered” broken link to identify the page in our website in the Google search listings. You would think they would use our sitemap, but no, apparently that was not authoritative enough for them; they preferred to use something they found on the Internet.
We recently viewed a video featuring Matt Cutts, one of Google’s Search engine experts, telling us that the sitemap was used a tie-breaker in situations where Google was not sure of the validity of the results it was evaluating. Sorry Matt, not is our case! Our sitemap and website consistently lost to the competition for the correct URL form to any place Google found any results about our website. Nope, the actual website and its verified sitemap specifically conveyed to Google were not enough to win the war.
As noted, the problems of the reappearance of the broken URLs happened on and off over the last several years. Unfortunately, in late 2010 and early 2011 all of our listings on Google were in this incorrect form, and Google, apparently, was unwilling to accept our sitemap as the truth. As you might guess, this impacted our traffic because the URL Google was providing was broken. This led us to put an ugly yellow text box on each page of our website with a note “Can’t See the Photos?” and a link that would correct the problem.
Often, people would see the broken images on the page, not read our yellow sign and use the back tab to return to Google’s index to select another website. In the end, we suspect that this backtabbing is what caused Google to reduce our ranking, even though they perpetuated the problem over the last several years when all they had to do to fix it was look at the sitemaps we had and continue to submit to them.
Other than referring to the sitemap, the next best way to solve this problem is to use what is called a 301 redirect, which works at the server-side of web hosting to intercept a request for a web page and send it to a new location where that page has been moved. Unfortunately, at least for a short time, this action further depresses your web rankings with Google and other search providers. However, since Google had reduced our rankings already, we decided it was the time to take the secondary hit and we redirected most of our destination guides from the “oldfolder” to a “newfolder”. Changing our folder structure from “Guidebook” to “newguidebook” seems to have resolved the broken image and link problem, but it may make it harder for you to find us for a while. Generally we can be found in the search engines by typing “best places to visit in COUNTRY NAME/CITY NAME”, but that really depends on Google. Well, if you want to find us without delay, just go to http://www.thereareplaces.com.
Now that we have that off our chest, we want you to know that we continue to develop new content and hope to push it out to you over the next few months, whether Google thinks it has value or not. We suspect we know more about travel than anyone at Google, how sad that they are our judge.
Thanks for reading this rant; it is just heartbreaking to spend so much time producing quality content that then gets trapped in your own, special Kobayashi Maru. We just had to tell someone about the insanity of trying to coexist with Google.
No Comments yet
Sorry, the comment form is closed at this time.