Week 3

What are Markvos Chains?

  • Use matrices to form increments to numbers.
  • Implement AI to game characters
  • Closest connections with Finite State Machines – Different states causes behaviour of AI to change.
  • Markvos chains are the support of finite state machines.


Classic Example

A Company – 5% of all random people to ask will drink brand a Brandy






B Company – Represents everything that isnt us. 95 % of people drink that.







Want to have more of a percentage of the market, so we go to see smart people and get advice. They predicted 86% of current customers will stick in next month. 5% who don’t drink will drink in the next month.

Markvos Chain


Diagram shows the likelihood someone will leave and go to a different state


Transition Matrix

Use to work out how much of the market Company A will have after a month


How to use in gaming?

  • Made an AI bot for arena map
  • Bot will snipe from a location – Not save once shot
  • Other positions it can go to – But which one?
  • Want the bot to retreat to safety as all positions are compromised



Design a finite state machine that uses transitional values that can be manipulated through markvo chains.