Introduction to the Markov Process

In this chapter, we will develop the basic concepts that we need to understand Hidden Markov Models (HMM). We will cover the following topics:

• Random processes
• Markov processes
• Markov chains or discrete-time Markov processes
• Continuous-time Markov chains