## Topic outline

• ### General

• This forum is available for everyone to post messages to. You can raise mathematical and organisational questions related to the module. You are encouraged to post to this forum and I will check it regularly and respond. You should also feel free to reply to other students.

• Key information about how the module will run.

• ### Lecture Notes

Notes covering all the material in the module will appear here. The document of notes will grow as we move through the module, with sections added in advance of the corresponding lectures.

Please let me know if you spot any mistakes or things which could be explained more clearly.

• ### Week 1 - Markov Chain Basics

• This module builds on your first and second year probability modules. Have a go at this sheet to check you are up to speed with the background material you will need. This sheet will be used in the Week 1 seminars.

• Mini-quiz on Markov chain basics (transition probabilities and transition matrices) based on the material in Week 1.

• This sheet will not be assessed. We will discuss selected parts in the seminars in Week 2.

• ### Week 2 - First Step Analysis

• Mini-quiz on absorbing states and first-step analysis based on the material in Week 2.

• This sheet will not be assessed. We will discuss selected parts in the seminars in Week 3.

• ### Week 3 - Long-term Behaviour I

• Mini-quiz on limiting and equilibrium distributions based on the material in Week 3.

• This sheet will not be assessed (but see below for one part to submit). We will discuss selected questions in the seminars in Week 4.

• This is the place to enter your answer to Question 1 from problem Sheet 3 by 5pm Friday 20 October 2023.

• ### Module Description

This is an advanced module in probability. It covers the mathematical analysis of processes which evolve randomly over time. For example, imagine a person wandering randomly through a maze; we can't predict their exact route but we can consider questions such as `What is the expected time it takes to reach the centre?'. One important class of processes studied are Markov chains. Roughly speaking, these are processes where the next step depends only on your current state but not how you got there. The module will cover both discrete-time processes (which evolve at fixed time steps) and continuous-time processes (which evolve continuously through time). The content is mainly theoretical but we will get a taste of how the models discussed do have applications in physical and life sciences and economics.  This module is probabilistic rather than statistical. Analysing processes which evolves over time from a statistical perspective is covered in the module Time Series. The content will build on previous probability modules; you will need to be comfortable with conditional probability and the basic of random variables. We will also see (perhaps surprisingly) that linear algebra and differential equations appear as tools to analyse random processes.

• ### Week 4 - Long-term behaviour II

• Mini-quiz on regular and irreducible chains based on the material in Week 4.

• ### Week 5 - More on limiting and equililbrium distributions/Recurrence and Transience

• Mini-quiz on communicating classes, recurrence and transience based on the material in Week 5.

• This is the first sheet for Assessment. Details of what you need to do are in the document. Submission will be via the tool in the Week 7 topic (to appear). This assessment contributes 10% towards your mark for the module.

• ### Week 6 - Recurrence and Transience

• Mini-quiz on positive recurrence, null recurrence and transience, and the connection with equilibrium distributions based on the material in Week 6.

• ### Week 7 - The Week formerly known as Reading Week

You should submit your work as a PDF file which can be either a scan of a handwritten document or electronically written on a tablet (but not typed in a word processor).

This Assessment component will contribute 10% of your final mark for the module.

Late submissions will not be accepted.

• ### Week 8 - Poisson Process Definition

• Mini-quiz on continuous-time stochastic processes and the basics of the Poisson process based on the material in Week 8.

• ### Week 9 - More on the Poisson Process

• Mini-quiz on more properties of the Poisson process based on the material in Week 9.

• This is the second sheet for Assessment. Details of what you need to do are in the document. Submission will be via the tool in the Week 11 topic (to appear). This assessment contributes 10% towards your mark for the module.

See the Student Forum for posts answering some student questions on this Assessment.

• ### Week 10 - Birth Processes

• Mini-quiz on more properties of the Birth process based on the material in Week 10.

• ### Week 11 - Continuous-time Markov Chains

• Mini-quiz on continuous-time Markov chains based on the material in Week 11 and 12.

• This is the place to enter your answer to Question 1 from Problem Sheet 10 by 5pm Friday 15 December 2023.

You should submit your work as a PDF file which can be either a scan of a handwritten document or electronically written on a tablet (but not typed in a word processor). Make sure that you follow all the instructions about the form your answers should take in the Assessment task document.

This Assessment component will contribute 10% of your final mark for the module.

Late submissions will not be accepted.

• ### Lecture Chat

• Live chat for online and in-person participants.