Beginning with a problem raised by the late Chinese mathematician Loo-Keng Hua, and the author's solution, this book discusses general Markov chains, a subject extremely popular in China in the fifties/sixties. The author emphasizes methodology like the first entrance and last exit decompositions, leading to the most beautiful results by Kolmogorov, Doeblin, and himself. Next he discusses the continuous time case in which names like Paul Lévy and Doob enter and there are hard analytic problems such as differentiability and systems of differential equations which are nicely solved. Next he introduces Brownian motion or Wiener process as the most famous Markov process, relates the probability theory to grand old mathematical-physics associated with Green, Dirichlet, Schrodinger and Feynman. Finally there comes an excursion into general probabilistic methodology. This book contains many examples and exercises with hints and discussions.