Markov Chains

A Markov chain is a sequence of random values whose probabilities at a time interval depends upon the value of the number at the previous time. A simple example is the nonreturning random walk, where the walkers are restricted to not go back to the location just previously visited.

Algorithmic Images

http://dbwilson.com/exact/#gallery (Perfectly Random Sampling with Markov Chains, Gallery, American Mathematical Society, 2002)

Demos

http://www.math.montana.edu/frankw/ccp/multiworld/matrices/markov/applet.htm
(Java Applets, Frank Wattenberg, Dep. Of Mathematics, Montana State University, 1997)
http://peabody.sapp.org/class/dmp2/lab/markov1/ (Digital Music Programming, Midi)

Definitions

http://www.taygeta.com/rwalks/node7.html
http://en2.wikipedia.org/wiki/Markov_chain http://pespmc1.vub.ac.be/ASC/MARKOV_CHAIN.html
http://www.marypat.org/stuff/random/markov.html

Non-Scientific Papers

http://www.saliu.com/Markov_Chains.html
(Markov Chain in Gambling, Game Theory, Lotto and Lottery, Ion Saliu, 2003)
http://www.aloha.com/~craven/Markov.html (War and Markov Chain)

Scientific Papers

http://decision.csl.uiuc.edu/~meyn/pages/book.html
(Markov Chains and Stochastic Stability, Sean Meyn, Richard Tweedie, 1994)
http://www.cs.umd.edu/users/oleary/res00/node1.html
(Markov Chains are used to model processes such as behavior of queuing networks, Dianne OÇLeary, 2001)

Applications in
Computer Science

http://www.cs.toronto.edu/~radford/review.abstract.html (Probabilistic Inference using Markov Chain Monte Carlo Methods, Radford M. Neal, Dept. of Computer Science, University of Toronto, 1993)

Applications in Economy

http://econpapers.hhs.se/paper/wpawuwppe/0205002.htm (The Evolution of Tax Evasion in the Czech Republic: A Markov Chain Analysis, Jan Hanousek (jan.hanousek@cerge-ei.cz) and Filip Palda (filip_palda@enap.uquebec.ca), Public Economics from Economics Working Paper Archive at WUSTL, 2003)
http://faculty.darden.virginia.edu/pfeiferp/Homepage/ModelingCustomersasMarkovChains.pdf (Modeling Custopmer Relationships as Markov Chains, Phillip E. Pfeifer and Robert L. Carraway, Darden School of Business, Charlotteville, VA, Journal of Interactive Marketing, 2000)
http://www.stat.fi/isi99/proceedings/arkisto/varasto/pell0333.pdf (Analysing economic Growth using Panel Data and Markov Chains, Guido Pellegrini, Dep. Of Statistics, University of Bologna)

Applications in Geology

http://www.stieltjes.org/archief/rep9899/node19.html 

Applications in Medicine

http://cnx.rice.edu/content/m10861/latest/ (Introduction to continuous and discrete Markov chains, including the "birth and death" process, The Connexion Project, Rice University)

Applications in Music

http://ccrma-www.stanford.edu/~jacobliu/254report/ (Modeling Music as Markov Chains - Composer Identification, Yi-Wen Liu, Music 254 Final Report, Stanford University, 2002)
http://scholar.lib.vt.edu/theses/available/etd-61098-131249/unrestricted/dmfetd.pdf (Markov Chains as Tool for Jazz Improvisation Analysis, David M. Franz, Faculty of the Virginia Polytechnic, Institute and State University, Blackburg, Virginia)