How Does Twitter Maintain High Google Search Rankings Using #! URLs?

, Wellesley, MA

A friend asked how to maintain high search rankings of a bunch of URLs that use Google’s #! AJAX URL convention. He cited Twitter as an example of a website doing it well. It’s true that Twitter uses #! in its URLs, and that Twitter pages rank highly in Google’s search results, but I notice that the search result URLs omit #!. Here’s how I think Twitter implements #! to maintain high search rankings:

  1. The pages that rank highly omit #!:

    http://twitter.com/matthewlmcclure rather than http://twitter.com/#!/matthewlmcclure

  2. http://twitter.com/matthewlmcclure is a static HTML snapshot rather than a real time feed.

  3. The HTML snapshot includes the following so that requesting it in a JavaScript enabled browser will redirect to the corresponding #! URL that loads a real time feed:

    <script type="text/javascript">
    //<![CDATA[
    window.location.replace('/#!/matthewlmcclure');
    //]]>
    </script>
    
  4. For every #! URL, there is a corresponding ?_escaped_fragment= URL like:

    https://twitter.com/?_escaped_fragment_=/matthewlmcclure
    
  5. The ?_escaped_fragment_= URL returns a 301 with Location: https://twitter.com/matthewlmcclure to close the loop and present the HTML snapshot to Google.

The content that Google sees for a #!/foo URL is different from what it sees for #!/bar so Twitter avoids any duplicate content penalty.

Enjoyed reading this post? Discuss it on Reddit, or follow me on Twitter.