Kingish.com

Here's an interesting tip that I'll share with you, just in case it saves you the hours and hours of research I did to come up with this solution!

Having made a number of sites, and written a stack-load of PHP code, I still could not find a solution on the web for creating a search friendly URLs. And, not just any urls, but creating search engine friendly urls for user generated content.

So what is a search engine friendly url?

That's simple. Search engines love static urls. That is, urls that do not contain parameters being passed to dynamic pages. You probably know the type. An example might be:

• www.example.com/page.asp?page=123&type=456&product=62

The above (made up) link could, for example, be used by a site to display product information for a specific type of Blue Widget. Search engines hate that! A search engine friendly version of that url could be something like:

• www.example.com/product-62.html
• www.example.com/blue-widget.html

A few essential rules to stick to when creating search friendly urls:

• Use a hyphen as a word-break character
• Don't use punctuation, if you can help it
• Definitely don't create pages that include either of the following characters: ? &

There are many good articles on the web explaining the details about what urls search engines like and don't like, and many sites dealing with ways to make search engine friendly urls. Check out this search for more info.

Ah, I hear you cry, what's the problem if there's so much information around?

User generated titles. That's the problem.

User Generated Pages and URLs

When you have a website that allows users to contribute content, you don't have any control over what they title their pages. Assuming that you wish to use the title of a page as its url, that can cause real problems. As these types of sites are database driven and the site creates links to the pages automatically, you can end up with some really long urls. What's more, they can contain any typed character in the title. Take, for example, a fictional article called "The grey line between what's right & what's wrong in politics - what's your view?". Potentially, an automatically generated url for this article might look like this:

• www.example.com/the-grey-line-between-what's-right-&-what's-wrong-in-politics---what's-your-view?.html
• www.example.com/resouces/articles/the-grey-line-between-what's-right-&-what's-wrong-in-politics---what's-your-view?.html

They're bad. Really bad. And really long.

When I've encountered this on my own sites before, I've usually given up and simply used an ID that represents that entry in the database, so the url looks more like:

• www.example.com/4561.html

That's good. It is short, and contains no bad punctuation or other unfriendly stuff. It's also bad. There are no keywords in it.

So how do you go about addressing this situation?

A PHP & MySQL Solution

Ok, let's start by specifying our aims and by setting out what we hope to achieve:

• We want short urls
• We don't want any punctuation
• We do want keywords
• We want to reverse-engineer the page title from the url

In other words, our code must be able to convert a title to a search engine friendly url so that it doesn't cause any problems for web crawlers/spiders. It must also be able to handle a visitor who arrives at our site via a search engine, and be able to correctly identify which page is to be shown from the referring URL. Lets tackle this in stages.

Short URLs & Punctuation

Below is a function that strips out punctuation and other fairly standard things:

This on it's own leaves our example url looking like this:

• www.example.com/the-grey-line-between-whats-right-whats-wrong-in-politics-whats-your-view.html

Keywords & Density

Our aim is to generate a url containing the main words in the title, but to keep the url short, we also would like to remove unnecessary words (stopwords) like the or and. The function below strips out stopwords from any string passed to it:

Now, our example url looks like this:

• www.example.com/grey-line-between-whats-right-whats-wrong-in-politics-whats-your-view.html

Of course, you can be as strict or as flexible as you like with your list of stopwords. It would be possible to create url as short as these quite easily using the above function:

• www.example.com/grey-line-between-whats-right-whats-wrong-politics-whats-view.html
• www.example.com/grey-line-between-right-wrong-politics-view.html
• www.example.com/grey-line-right-wrong-politics-view.html

We now have a url that can easily be read by search engines and users alike. It's short, and contains only the words that provide an insight into what the page is about.

Reverse Engineering Titles From URLs

Right, now we have a site capable of creating pages like www.example.com/grey-line-right-wrong-politics-view.html. So, a search engine spider crawls the page and lists it in their index. Someone clicks onto your page via a search engine. What page do you display? After all, you don't have an article called "grey line right wrong politics view", do you? Here's how I solved that problem:

A dynamic MySQL databased site could contain a simple table for page definitions like this:

In order to track which short url corresponds to which article, you need to add a column to your database table which will hold a copy of the short url. This can then be used to cross-reference from one to the other. I'll call this new column 'filename', so our table now looks like this:

This new column, filename, allows you to query the database with a sting like "grey line right wrong politics view" and pull out the correct page entry for the article called "The grey line between what's right & what's wrong in politics - what's your view?".

Limitations

If two entries are created with two different titles, and they both result in the same short url due to the removal of stopwords and punctuation, then the same page will be show for both entries. For example, if one user creates an entry called "A Fairy Tale" and another user creates "Fairy Tale", then they will both result in the short url of "fairy-tale" if 'a' is in your stopword list (I imagine it might be).

The code provided will not distinguish between the strings "example's" and "examples" since it removes all punctuation.

A stopword list is not a one-size-fits-all solution. Sure, you can have a very successful general stopword list that will do for most situations, but not all. The ability to provide specific stopword lists for specific situations would be a good extension to the code above.