Most of the functions were written by 5ubliminal. I use the normalizeHtml() function to convert some Unicode characters to ASCII. Includes/html_functions.php – various utility functions (optional). Includes/porter_stemmer.php – a lousy stemmer script used by the summarizer class. Includes/summarizer.php – the Summarizer PHP class. What this means is, we will have an agent that randomly jumps around different states, with a certain probability of going from each state to another one. Test_summarizer.php – a simple demo of the text summarizer script. A Markov Chain is a stochastic process that models a finite set of states, with fixed conditional probabilities of jumping from a given state to another. For example, if the word “Linux” occurs 4 times overall, and the word “Windows” occurs 3 times, then the sentence “Windows bad, Linux – Linux good!” will get a rating of 11 (assuming “bad” and “good” didn’t make it into the Top 20 word list).

In this case I simply added together the popularity ratings of every “important” word in the sentence.

Rate each sentence by the words it contains.

Also, the “top 20” threshold is a mostly arbitrary choice, so feel free to experiment with other numbers. The idea is that the most common words reflect the main topics of the input text. Sort the words by their “popularity” in the text and keep only the Top 20 most popular words.Skip words that are very common, like “you”, “me”, “is” and so on (I just used the stopword list from the Open Text Summarizer). For each word, stem it and keep count of how many times it occurs in the text.Split the input text into sentences, and split the sentences into words.Here’s the summarization algorithm in a nutshell : But hey, it’s free, and – as far as I know – the only text summarizer implemented in PHP 😉 You can find the download link at the bottom of this post. The summary generator is quite primitive and probably doesn’t compare too favourably to OTS or commercial products. Jamie Brew, writer for parody site Clickhole, has created a predictive text generator that works on Markov-like principles to write. It would be pretty easy to adapt the PHP script to other languages, too. Markov chains can help write prose, as well as poetry. I’ve written a simple text summarizer that can find the most important sentences in any given (English) text and produce a summary of the specified length.

Markov text generator algorithm crummy